Intelligent coal gangue positioning method and system based on computer vision

Abstract

This application provides a computer vision-based intelligent coal gangue positioning method and system, applied in the field of coal gangue positioning technology. It involves offline acquisition of coal gangue images and environmental parameters, extraction of multi-dimensional image features, and establishment of a mapping relationship between environmental parameters and image degradation to generate an image correction dataset. During online sorting, the system matches correction information based on real-time environmental parameters to adaptively enhance the images, resulting in high-quality images. Based on the obtained high-quality images, the system identifies the coal gangue category and pose parameters, performs multiple trajectory corrections in conjunction with conveyor belt operating parameters and coal gangue characteristics, generates information, and generates control commands based on category and location to drive the sorting equipment to complete coal gangue separation. This method can identify and locate coal gangue under complex and variable environmental conditions, effectively improving sorting efficiency.

Description

Technical Field

[0001] This application relates to the field of coal gangue positioning technology, and in particular to a computer vision-based intelligent coal gangue positioning method and system. Background Technology

[0002] With the development of coal washing and intelligent mine construction, the coal gangue sorting process is gradually shifting from manual sorting to automated visual recognition and actuator linkage control.

[0003] Image recognition-based coal and gangue detection technology has been widely applied in online sorting scenarios on conveyor belts. It acquires coal and gangue image information through camera devices, combines it with classification models to output category results, and drives sorting equipment to complete the rejection operation.

[0004] However, in actual underground or coal washing workshop environments, coal gangue sorting is typically conducted under complex conditions of high dust, high humidity, uneven lighting, and frequent changes in the concentration of suspended particles in the air. Existing coal gangue sorting technologies mainly rely on deep learning models or conventional fixed preprocessing enhancement methods. These methods can achieve good recognition results in the laboratory or under relatively stable conditions, but in actual coal mine production sites, the superposition of multiple environmental factors causes image features to drift significantly with environmental changes. Since existing technologies mostly use fixed enhancement parameters or directly rely on existing models for recognition, the image quality degradation caused by environmental changes cannot be effectively compensated, resulting in a decrease in the accuracy of coal gangue recognition and thus affecting the sorting efficiency of coal gangue. Summary of the Invention

[0005] This application provides a computer vision-based intelligent coal gangue positioning method and system. Its core is as follows: by collecting image frame data of coal gangue on a conveyor belt and corresponding environmental parameters, multi-scale image features are extracted and their degradation relative to preset benchmark features are calculated to form image degradation information. Based on this degradation information, a degradation relationship between environmental parameters and image features is established, and feature correction information for different environmental conditions is generated. A matching image correction dataset is constructed, thereby enhancing and correcting images of coal gangue under corresponding environmental conditions, improving the accuracy of coal gangue identification. Combining the characteristics of the coal gangue itself with motion parameters such as conveyor belt speed and direction of travel, trajectory tracking and position information calculation are performed on the coal gangue, achieving centroid tracking and trajectory prediction. Based on the identified category and position information, control commands are generated to drive the sorting equipment to achieve intelligent coal gangue sorting, improving sorting efficiency under complex working conditions.

[0006] To achieve the above objectives, this application adopts the following technical solution: This application provides a computer vision-based intelligent positioning method for coal gangue, which may include: Acquire the first image frame data of coal gangue on the conveyor belt and the corresponding first environmental parameters; Extract image features from the first image frame data, calculate the degradation amount of the image features relative to a preset reference feature, and obtain image degradation information; Based on the image degradation information, a degradation relationship between the first environmental parameter and the image feature is established, and corresponding feature correction information is generated. Based on the feature correction information and the corresponding first environmental parameter, a corresponding image correction dataset is constructed. During the coal and gangue sorting process, the second image frame data of the coal and gangue on the conveyor belt and the corresponding second environmental parameters are acquired. Based on the second environmental parameters, the image correction dataset is matched to determine the corresponding image correction information. Based on the image correction information, the second image frame data is enhanced to obtain the corresponding third image frame data. Based on the third image frame data, the coal and gangue on the conveyor belt are identified to obtain the corresponding category information and pose parameters. The operating parameters of the conveyor belt are obtained, and based on the operating parameters and pose parameters, the trajectory of the coal and gangue on the conveyor belt is tracked to generate corresponding position information. Based on the category information and location information, corresponding control instructions are generated, and the sorting execution equipment is controlled to sort the coal and gangue on the conveyor belt using the control instructions.

[0007] In some possible implementations, the step of extracting image features from the first image frame data and calculating the degradation of the image features relative to a preset reference feature includes: Multi-scale feature decomposition is performed on the first image frame data to obtain texture features, edge gradient features, grayscale distribution features, color saturation features, and local fogging features. The degradation amount of the texture feature, the edge gradient feature, the grayscale distribution feature, the color saturation feature, and the local haze feature relative to the preset benchmark feature is calculated to obtain the corresponding image degradation information.

[0008] In some possible implementations, calculating the degradation amount of the texture feature, the edge gradient feature, the grayscale distribution feature, the color saturation feature, and the local haze feature relative to a preset reference feature to obtain the corresponding image degradation information includes: For the texture feature, the relative attenuation degree between the texture feature and the preset reference feature is calculated as the texture degradation amount; For the edge gradient feature, the intensity reduction between the edge gradient feature and the preset benchmark feature is calculated as the edge degradation amount; For the grayscale distribution feature, the statistical distribution deviation between the grayscale distribution feature and the preset benchmark feature is calculated as the grayscale degradation amount; For the color saturation feature, the saturation attenuation ratio between the color saturation feature and the preset reference feature is calculated as the color saturation degradation amount; For the local fogging feature, the non-uniformity deviation between the local fogging feature and the preset reference feature is calculated as the scattering degradation amount; The texture degradation, edge degradation, grayscale degradation, color saturation degradation, and scattering degradation are integrated to form the corresponding image degradation information.

[0009] In some possible implementations, establishing the degradation relationship between the first environmental parameter and the image features based on the image degradation information, and generating corresponding feature correction information, includes: Based on the image degradation information, analyze the degradation variation law of the first image frame data under the first environmental parameter, and establish the degradation relationship between the first environmental parameter and the image feature; Based on the degradation relationship, corresponding feature correction information is generated for the first image frame data.

[0010] In some possible implementations, the first environmental parameter includes a first illumination parameter and a first medium parameter. The step of analyzing the degradation variation pattern of the first image frame data under the first environmental parameter based on the image degradation information, and establishing the degradation relationship between the first environmental parameter and the image features, includes: Based on the influence trend of the first illumination parameter on the texture degradation, edge degradation, and color saturation degradation, a corresponding first degradation relationship is established; Based on the influence trend of the first medium parameter on the grayscale degradation and the scattering degradation, a second degradation relationship between the medium parameter and the corresponding degradation is established; By comprehensively analyzing the first degradation relationship and the second degradation relationship, the degradation relationship between the first environmental parameter and the image feature is determined.

[0011] In some possible implementations, generating corresponding feature correction information for the first image frame data based on the degradation relationship includes: Based on the first degradation relationship, the texture degradation amount, the edge degradation amount, and the color saturation degradation amount are respectively calculated in reverse to obtain the corresponding first feature correction information; Based on the second degradation relationship, the grayscale degradation amount and the scattering degradation amount are calculated in reverse to obtain the corresponding second feature correction information; The first feature correction information and the second feature correction information are used as the feature correction information corresponding to the first image frame data.

[0012] In some possible implementations, the second environmental parameter includes a second illumination parameter and a second medium parameter. The step of matching the image correction dataset based on the second environmental parameter to determine the corresponding image correction information includes: The second illumination parameter is compared with the first illumination parameter in the image correction dataset to calculate the parameter difference, and the illumination matching error value is obtained. The media matching error value is obtained by calculating the parameter difference between the second medium parameter and the first medium parameter in the image correction dataset. The illumination matching error value and the medium matching error value are fused to obtain a comprehensive matching error value; The feature correction information corresponding to the first environmental parameter with the smallest comprehensive matching error value is selected from the image correction dataset and used as the image correction information of the second image frame data.

[0013] In some possible implementations, the operating parameters include conveyor belt speed parameters and conveyor belt running direction; the pose parameters include coal and gangue centroid parameters, coal and gangue size parameters, coal and gangue inclination angle parameters, and estimated values ​​of coal and gangue surface normal vectors; and the step of tracking the trajectory of the coal and gangue on the conveyor belt based on the operating parameters and pose parameters to generate corresponding position information includes: Based on the conveyor belt speed parameters and the conveyor belt running direction, calculate the first motion trajectory of the coal and gangue on the conveyor belt; Based on the coal and gangue size parameters and the estimated value of the coal and gangue surface normal vector, calculate the inertial influence coefficient and rolling trend factor of the coal and gangue during the conveyor belt movement process; Based on the coal gangue tilt angle parameter, the rolling trend factor, and the inertial influence coefficient, the attitude deviation in the first motion trajectory is corrected to obtain the second motion trajectory; Based on the changing trend of the coal gangue centroid parameters, the slippage and deviation of the conveyor belt are detected to obtain the operating error. The second motion trajectory is then compensated based on the operating error to obtain the third motion trajectory. Based on the third motion trajectory, the location information of the coal gangue is generated.

[0014] In some possible implementations, calculating the inertial influence coefficient and rolling tendency factor of the coal and gangue during the conveyor belt movement, based on the coal and gangue size parameters and the estimated value of the coal and gangue surface normal vector, includes: Based on the coal gangue size parameters, the motion lag time of the coal gangue on the conveyor belt is calculated, and the motion lag time is normalized as the inertial influence coefficient. Based on the estimated surface normal vector of the coal gangue and the size parameters of the coal gangue, the tendency of the coal gangue to roll on the conveyor belt is calculated, and the tendency is normalized as a rolling trend factor.

[0015] A computer vision-based intelligent coal gangue positioning system, the system comprising: Data acquisition module, image processing module, positioning module, and control module; The data acquisition module is used to acquire the first image frame data of coal and gangue on the conveyor belt and the corresponding first environmental parameters. During the coal and gangue sorting process, it acquires the second image frame data of coal and gangue on the conveyor belt and the corresponding second environmental parameters. The image processing module includes a feature extraction unit, a degradation analysis unit, and a matching enhancement unit; The feature extraction unit is used to extract image features from the first image frame data, calculate the degradation amount of the image features relative to a preset benchmark feature, and obtain image degradation information. The degradation analysis unit is used to establish a degradation relationship between the first environmental parameter and the image feature based on the image degradation information, generate corresponding feature correction information, and construct a corresponding image correction dataset based on the feature correction information and the corresponding first environmental parameter. The matching enhancement unit is used to perform matching in the image correction dataset based on the second environmental parameters, determine the corresponding image correction information, and perform image enhancement on the second image frame data based on the image correction information to obtain the corresponding third image frame data. The positioning module includes an identification unit and a trajectory tracking unit; The identification unit is used to identify the coal and gangue on the conveyor belt based on the third image frame data, and obtain the corresponding category information and pose parameters; The trajectory tracking unit is used to acquire the operating parameters of the conveyor belt, and based on the operating parameters and pose parameters, to track the coal and gangue on the conveyor belt and generate corresponding position information. The control module is used to generate corresponding control commands based on the category information and location information, and use the control commands to control the sorting execution equipment to sort the coal and gangue on the conveyor belt.

[0016] As can be seen from the above technical solution, this application has the following beneficial effects: 1. This application constructs an image correction dataset driven by environmental parameters offline and realizes adaptive image enhancement based on real-time environmental parameters with minimum error matching online. This effectively compensates for image feature drift and degradation caused by multiple interferences such as dust scattering, uneven lighting, and moisture reflection under complex working conditions of coal mine conveyor belts, thereby improving the accuracy of coal gangue identification in extreme environments.

[0017] 2. This application achieves targeted reverse compensation for degradation causes by quantifying multi-dimensional degradation and mapping it independently to environmental parameters, generating parameterized feature correction information, thereby maintaining image quality stability in the variable environment of underground coal mines or coal preparation workshops.

[0018] 3. This application incorporates the physical characteristics of coal and gangue themselves into the trajectory tracking stage for multi-level correction, generating a third motion trajectory, which reduces the phenomena of mis-spraying, empty spraying and delay in sorting execution, and improves the efficiency of coal and gangue sorting. Attached Figure Description

[0019] The present application will be further described below with reference to the accompanying drawings.

[0020] Figure 1 A flowchart of a computer vision-based intelligent coal gangue positioning method provided for this application; Figure 2 A flowchart of another intelligent coal gangue positioning method based on computer vision provided in this application; Figure 3 An example diagram of the computer vision-based intelligent coal gangue positioning system provided in this application. Detailed Implementation

[0021] The terms "first," "second," and "third," etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0022] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the related technologies is given first: In the coal gangue sorting process, images of the coal gangue on the conveyor belt are acquired, and the acquired images are denoised to reduce random interference caused by changes in lighting and dust particles. At the same time, color correction and lighting equalization are used to enhance the overall brightness consistency and local detail discernibility of the images. The processed images extract texture features, edge gradient features, grayscale distribution features, and color features at multiple scales. The extracted features are input into a pre-trained classification model. By comparing the input features with the features learned by the model, the coal gangue category information is output. The two-dimensional position and size of the coal gangue in the image are determined. Based on the conveyor belt speed and running direction, the motion trajectory of the coal gangue on the conveyor belt is calculated using continuous image frames, and its three-dimensional position change and arrival time are predicted. The sorting operation is controlled based on the prediction results to achieve coal gangue sorting.

[0024] Research has revealed that the actual production environment in coal mines is extremely complex. This includes the phased fluctuations in dust concentration due to equipment start-up and shutdown and material impact, dynamic shadows or local overexposure caused by changes in light intensity and direction with the coal accumulation pattern, and specular reflections and highlight spots caused by moisture or mud adhering to the coal gangue surface. The aforementioned image preprocessing and classification methods cannot specifically compensate for these dynamic changes under fixed parameters.

[0025] Specifically, fixed denoising and illumination equalization processes are insufficient to completely eliminate local non-uniform fogging and instantaneous brightness shifts, leading to drift in texture features, edge gradients, and grayscale distributions with environmental changes. Simultaneously, the feature distribution learned by the classification model during training cannot cover these dynamic changes, making the model sensitive to environmental interference, reducing recognition confidence, and increasing false positive and false negative rates. Furthermore, trajectory prediction based on continuous frame calculations also accumulates errors due to centroid position shifts caused by image feature degradation, resulting in a deviation between the sorting execution time and the actual coal arrival time, thus affecting sorting accuracy. As the above analysis shows, existing coal sorting methods lack dynamic correlation between image feature extraction, recognition models, and trajectory prediction for complex environments, leading to low recognition accuracy under different environmental conditions and consequently reducing coal sorting efficiency.

[0026] Example 1: To solve the above problems, this application provides a computer vision-based intelligent coal and gangue positioning method. Please refer to [link to example]. Figure 1 and Figure 2 .

[0027] S101, acquire the first image frame data of coal gangue on the conveyor belt and the corresponding first environmental parameters, extract the image features of the first image frame data, calculate the degradation amount of the image features relative to the preset reference features, and obtain image degradation information.

[0028] The first image frame data refers to two-dimensional images of coal gangue acquired under specific environmental conditions. These images record the visible characteristics of the coal gangue under those conditions, including surface texture, edge contours, grayscale distribution, color saturation, and localized fogging. This image frame data is used to analyze the impact of environmental conditions on image features, rather than data from the real-time sorting process. It primarily serves as a reference basis for establishing the relationship between environmental parameters and image feature degradation.

[0029] The first environmental parameter refers to the environmental state information corresponding to the acquisition of the first image frame data. It is used to describe the external conditions that affect the features of the coal gangue image. The first environmental parameter includes two types: the first illumination parameter and the first medium parameter. The first illumination parameter describes the lighting conditions during acquisition, including light intensity, light source direction, and light uniformity. It is used to quantify the impact of illumination changes on the brightness, contrast, and local shadows of the coal gangue image. The first medium parameter describes the environmental medium factors that affect light propagation and image clarity, including the concentration of suspended particles in the air, humidity, and the amount of water or mud adhering to the surface of the coal gangue. It is used to quantify the impact of medium scattering, fogging, and high light shift on image features.

[0030] Image features refer to quantifiable visual information extracted from coal gangue images, used to describe the texture, edges, grayscale distribution, color, and local fogging degree of the coal gangue. Image features are the core data for subsequent degradation calculations and feature corrections, used to quantify the impact of the environment on visual information.

[0031] Preset baseline features: These refer to the set of coal gangue image features obtained under ideal or standard environmental conditions, including indicators such as texture, edges, grayscale distribution, color saturation, and local fogging. The preset baseline features serve as a reference and are used to compare with the image features of the first image frame data to calculate the degradation amount.

[0032] Degradation: refers to the degree of change of image features relative to preset baseline features under specific environmental parameters, used to quantify the attenuation of visual information in coal gangue images under environmental interference. Degradation can cover multiple aspects such as texture attenuation, edge strength reduction, grayscale distribution shift, color saturation attenuation, and local fogging enhancement.

[0033] Image degradation information refers to a dataset that integrates various image feature degradation parameters, used to describe the overall visual information changes of coal gangue images under specific environmental parameters. Image degradation information is the foundation for establishing the relationship between environmental parameters and image feature degradation, and for generating feature correction information. It can be used for image enhancement and recognition optimization in subsequent sorting processes.

[0034] In some possible implementation methods, first image frame data of coal gangue on the conveyor belt and corresponding first environmental parameters are acquired. The coal gangue is fixed on the conveyor belt and photographed by a high-resolution industrial camera. Before image acquisition, the first environmental parameters are recorded, including first illumination parameters such as illumination intensity and light source direction, and first medium parameters such as concentration of suspended particles in the air, humidity, and moisture or mud adhering to the surface of the coal gangue. In order to fully cover possible environmental changes, multiple first image frame data can be repeatedly acquired under different time periods, different illumination angles, different dust concentrations and humidity conditions to form diverse first image frame data. After each acquisition, the corresponding first environmental parameters are recorded and labeled to ensure that each first image frame data can be associated with accurate environmental conditions.

[0035] In some possible implementation methods, image features can be extracted from the acquired first image frame data through multi-level and multi-angle image analysis to comprehensively quantify the visible feature information of coal gangue under different environmental conditions.

[0036] Specifically, texture features can be calculated using a gray-level co-occurrence matrix at different directions (e.g., 0°, 45°, 90°, and 135°) and distances to determine the co-occurrence probability of pixel gray-level pairs. The occurrence frequency of each gray-level value combination separated by a certain number of pixels is counted to generate a two-dimensional matrix, where each element represents the probability of a particular gray-level combination. Subsequently, statistical indicators are calculated, including contrast (representing the degree of gray-level difference), energy (representing texture consistency), correlation (representing the linear correlation between gray-level pairs), and entropy (representing texture complexity). Furthermore, a local binary model can be used, comparing each pixel's gray-level value with its neighboring pixels; values ​​greater than the center value are encoded as 1, and values ​​less than are encoded as 0, forming a binary model. A statistical histogram reflects the texture distribution. Wavelet decomposition can also be used to separate low-frequency and high-frequency information at different scales. High-frequency coefficients reflect texture details and edge changes, while low-frequency coefficients reflect the overall texture structure.

[0037] Edge gradient features characterize the sharpness and contour information of coal gangue edges by calculating the rate of change of image pixel grayscale. For example, the Sobel operator can be used to calculate horizontal and vertical gradients to obtain gradient magnitude and direction; Gaussian smoothing is first performed for noise reduction, then the gradient magnitude and direction are calculated, and non-maximum values ​​are suppressed by double thresholding to form continuous edge contours. Alternatively, the Laplacian operator can be used to calculate the second derivative to quantify edge sharpness. Statistical analysis includes the mean edge magnitude, edge density (number of edge pixels per unit area), and histogram of edge direction distribution to describe edge sharpness and integrity.

[0038] Gray-level distribution features quantify brightness variations and local contrast by analyzing the statistical distribution of image gray-level values. A gray-level histogram is generated, the frequency of each gray-level value is counted, and then statistical indicators are calculated, including gray-level mean, variance, skewness (indicating distribution skewness), and kurtosis (indicating the sharpness of the gray-level distribution). Through gray-level distribution features, the impact of uneven illumination, shadows, or local brightness shifts on the image can be quantified.

[0039] Color saturation features extract hue, saturation, and brightness information by converting an image to HSV or Lab color space. It can calculate the overall image saturation mean, variance, and local region distribution, statistically analyzing the saturation mean and variance of the entire image or a specific region to reflect overall color intensity and local deviations. Normalizing the color histogram yields the distribution ratio of each grayscale or color interval, used to quantify color attenuation and shift caused by changes in ambient light, humidity, or surface reflections.

[0040] Local fogging features quantify the scattering and fogging effects caused by suspended particles, humidity, and surface moisture in coal gangue by analyzing local image sharpness, edge attenuation, and high-frequency information loss. The high-frequency energy of local image regions can be calculated using Laplacian variance: first, the Laplacian operator is applied to a local window, and the variance of the response value is calculated; a smaller variance indicates more severe local blurring. Alternatively, the degree of local texture attenuation can be analyzed by wavelet decomposition of high-frequency coefficient energy; a decrease in high-frequency coefficient energy indicates enhanced fogging. Furthermore, the variance of local gradient amplitude can be calculated, and changes in local gradient amplitude can determine edge blurring and scattering range. Local fogging features can comprehensively reflect the degree of visual information sharpness loss of coal gangue under specific media conditions.

[0041] In some possible implementations, for the extracted coal gangue image texture features, edge gradient features, grayscale distribution features, color saturation features, and local fogging features, the degradation amount between them and the preset benchmark features is calculated to obtain the corresponding texture degradation amount, edge degradation amount, grayscale degradation amount, color saturation degradation amount, and scattering degradation amount. Then, these degradation amounts are integrated to form comprehensive image degradation information.

[0042] The preset baseline features are obtained by acquiring a set of high-resolution coal gangue images under conditions of uniform lighting, low concentration of suspended particles in the air, moderate humidity, and clean coal gangue surface. During the acquisition process, the intensity and direction of the light source are standardized to ensure that the brightness and shadow distribution of each frame of the image are consistent. At the same time, the exposure, white balance, and gain parameters of the camera equipment are fixed to ensure stable image quality. After the acquisition is completed, the texture features, edge gradient features, grayscale distribution features, color saturation features, and local fogging features are extracted from each image using the above feature extraction methods.

[0043] After obtaining the five types of features for each image, statistical analysis is performed on the features of all images. By calculating the mean, variance, and distribution range of each type of feature, a comprehensive feature set of the specific coal gangue type under standard conditions is formed and defined as the preset benchmark feature.

[0044] In some possible implementations, the texture features of the first image frame data are matched with corresponding indices such as contrast, energy, correlation, and entropy. The relative change of each index between the current image and the baseline features is calculated, and the relative attenuation can be expressed by the formula: ,in Represents the baseline eigenvalue. The image texture feature value represents the first image frame data. The larger the relative attenuation value, the more obvious the texture information is affected by the environment. By comprehensively statistically analyzing all texture feature values, the amount of texture degradation is obtained, which is used to quantify the impact of environmental factors on the texture details and local non-uniformity of coal gangue surface.

[0045] For edge degradation, the average gradient magnitude, edge density, and orientation distribution of the first image frame data are calculated and compared with the corresponding values ​​of the preset benchmark features to calculate the corresponding intensity reduction. For example, the gradient magnitude attenuation rate is equal to the difference between the benchmark gradient mean and the gradient mean of the first image frame data divided by the benchmark gradient mean. The edge density deviation is equal to the difference between the benchmark edge density and the edge density of the first image frame data. The orientation deviation is obtained by calculating the similarity change of the orientation histogram. The above indicators are integrated to form the edge degradation. For example, in a high humidity environment, the gradient magnitude decreases by 30%, the edge density decreases by 10%, the orientation deviation is calculated to be 5%, and the edge degradation is approximately 45%.

[0046] To determine the amount of grayscale degradation, the mean, variance, skewness, and kurtosis of the current image are statistically analyzed and compared with preset benchmark features. The grayscale mean deviation is equal to the difference between the benchmark mean and the mean of the first image frame data, divided by the benchmark mean. The relative differences in skewness and kurtosis can be calculated. The amount of grayscale degradation is obtained by integrating these indicators. For example, if insufficient illumination causes a 12% decrease in mean, an 8% decrease in variance, and a 5% change in skewness and kurtosis, the integrated grayscale degradation amount is approximately 25%.

[0047] To determine the degree of color saturation degradation, the image is first converted to HSV or Lab color space. The saturation channels are then extracted, and saturation feature values ​​such as mean, variance, and local region mean are calculated. These values ​​are then compared with preset benchmark features to determine the saturation attenuation ratio. ,in This represents the saturation feature value in the preset benchmark features. The saturation feature value of the first image frame data can be calculated separately for the overall and local regions, and then weighted to obtain the comprehensive color saturation degradation. Example: Under high humidity and reflective light conditions, the overall saturation decreases by 15%, and the local area decreases by 20%, resulting in a combined color saturation degradation of approximately 45%.

[0048] To determine the scattering degradation, the Laplacian variance of the local window, the energy of the high-frequency wavelet coefficients, and the variance of the local gradient magnitude are calculated and compared with preset benchmark features. The Laplacian variance reduction rate is equal to the difference between the Laplacian variance in the preset benchmark features and the Laplacian variance of the first image frame data, divided by the Laplacian variance in the preset benchmark features. The reduction rate of the high-frequency coefficient energy and the change rate of the local gradient variance are calculated similarly. By integrating these indicators, the local haze degradation is obtained. For example, if the concentration of suspended particles in the air is high, the Laplacian variance decreases by 15%, the high-frequency energy decreases by 18%, and the local gradient variance decreases by 12%, resulting in an integrated scattering degradation of approximately 45%.

[0049] In some possible implementations, texture degradation, edge degradation, grayscale degradation, color saturation degradation, and scattering degradation are treated as independent indicators and normalized to unify their numerical ranges. For example, a normalized degradation value can be obtained by dividing each degradation type by its maximum possible variation, thus eliminating the influence of differences in the dimensions of different feature indicators. During integration, the degradation values ​​can also be retained as vectors to form an image degradation information vector, with each element corresponding to a feature degradation value. This facilitates the subsequent establishment of a mapping relationship between environmental parameters and degradation features, or allows for adaptive adjustment of the image enhancement model and recognition algorithm based on the environment. The image degradation information can be further correlated with a first environmental parameter to form comparative data on environment and image degradation.

[0050] This application acquires images of coal gangue on a conveyor belt and simultaneously records environmental parameters such as illumination, humidity, and dust. Texture, edge, grayscale, color saturation, and local fogging features are extracted from the images and compared with preset benchmark features. Various degradation parameters are then fused to form image degradation information. Based on this information, this application can quantify the impact of the environment on image features and adaptively compensate for these features, thereby maintaining the accuracy of coal gangue identification under different lighting and media conditions.

[0051] S102, Based on the image degradation information, establish the degradation relationship between the first environmental parameters and image features, generate feature correction information, and construct an image correction dataset. (See also...) Figure 2 .

[0052] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant terms is given first: The first degradation relationship refers to the mathematical relationship or fitted curve that quantifies the influence of the first illumination parameters (including illumination intensity, light source direction, and illumination uniformity) on the texture degradation, edge degradation, and color saturation degradation of coal gangue images. This relationship can reflect the changing trends of various image feature degradation under different illumination conditions, providing a basis for feature compensation under illumination conditions.

[0053] The second degradation relationship refers to the mathematical relationship or fitted surface that quantifies the influence of the first medium parameter (including the concentration of suspended particles in the air, humidity, and the moisture or mud on the surface of coal gangue) on the amount of grayscale degradation and scattering degradation. This relationship can describe the local fogging and grayscale change trends of images under different medium conditions, providing a basis for feature compensation under medium interference.

[0054] First feature correction information: This refers to the set of parameters calculated by reverse compensation of texture degradation, edge degradation, and color saturation degradation based on the first degradation relationship. It includes the incremental or proportional adjustment values ​​for each feature. This information is used to recover image features affected by lighting conditions, achieving optimization of the image in terms of texture, edges, and color.

[0055] Second feature correction information: This refers to the set of parameters calculated by inversely correcting the grayscale degradation and scattering degradation based on the second degradation relationship, including the local grayscale adjustment magnitude and the scattering blur enhancement magnitude. This information is used to recover image features under conditions of media interference, thereby optimizing the image in terms of brightness and local sharpness.

[0056] Image correction dataset: refers to a structured data set formed by associating and integrating the image degradation information and corresponding feature correction information of the first image frame data under different first environmental parameters.

[0057] In some possible implementation methods, in order to construct the corresponding first degradation relationship and second degradation relationship, the degradation amount of multiple first image frame data under different environmental conditions is statistically analyzed, and the mean, variance and trend of change are calculated, thereby quantifying the influence of environmental parameters on the degradation of various image features.

[0058] Specifically, for constructing the first degradation relationship, considering the influence of the first illumination parameter on texture, edge, and color saturation, regression analysis, curve fitting, or polynomial fitting methods are used. The first illumination parameter, such as illumination intensity, light source direction, and illumination uniformity, is used as the independent variable, and texture degradation, edge degradation, and color saturation degradation are used as the dependent variables to establish a mathematical model or fitting function. For example, linear regression can obtain a linear mapping relationship between texture degradation and illumination intensity, while polynomial fitting can capture nonlinear trends. By statistically analyzing the effects of illumination direction and uniformity on edge degradation and color saturation degradation, curves relating edge degradation to illumination direction and color saturation to illumination uniformity can be constructed, thus obtaining a complete first degradation relationship. Using the first illumination parameter as the input variable and image degradation as the output variable, a predictable description of the quantitative relationship between illumination conditions and image feature degradation can be achieved by establishing a mapping relationship or fitting curves. For example, texture degradation increases linearly with decreasing illumination intensity. When the illumination intensity decreases from 500 lux to 200 lux, texture degradation increases from 10% to 35%. Edge degradation increases by approximately 8% when the illumination direction deviates from the standard angle by 15 degrees. Color saturation decreases by approximately 12% when illumination uniformity decreases by 20%. By fitting, a relationship curve can be obtained, thus determining the first degradation relationship.

[0059] To construct the second degradation relationship, the influence of the first medium parameter on grayscale degradation and scattering degradation is considered. Through multivariate regression analysis, local weighted regression, or surface fitting, the first medium parameter, such as the concentration of suspended particles in the air, humidity, and surface moisture of coal gangue, is used as the independent variable, and grayscale degradation and scattering degradation are used as the dependent variables. A trend curve of degradation as a function of medium conditions is obtained through fitting. For example, a curve can be established for grayscale degradation, humidity, and dust concentration to describe the change in grayscale degradation under different combinations of humidity and dust concentration. Scattering degradation can be mapped to the first medium parameter through local haze indices to predict the scattering effect. The technical principle lies in obtaining the relationship curve between the environment and degradation by statistically analyzing the image degradation response under different medium conditions, thus achieving a quantitative description of the image feature attenuation by the first medium parameter. For example, under conditions of 80% humidity and an air dust concentration of 0.5 mg / m³... 3 Under the given conditions, the grayscale degradation was 28%, and the scattering degradation was 32%; when the humidity decreased to 60% and the dust concentration decreased to 0.2 mg / m³, the degradation was further reduced. 3 At that time, the grayscale degradation decreased to 18% and the scattering degradation decreased to 22%. The corresponding relationship curves were obtained by surface fitting, thereby determining the second degradation relationship.

[0060] In some possible implementations, for feature correction of the first image frame data, the first degradation relationship and the second degradation relationship constructed in the above steps can be used to perform reverse compensation calculation on various types of image degradation, thereby generating corresponding feature correction information.

[0061] Specifically, for the first degradation relationship, the influence trends of light intensity, light source direction, and light uniformity on texture degradation, edge degradation, and color saturation degradation are used as a reference. Based on the first lighting parameters, the compensation increment or proportional adjustment value for each degradation feature is calculated. For example, if the curve of texture feature and light intensity relationship in the first degradation relationship shows that the texture degradation increases linearly with the decrease of light intensity, and the actual texture degradation of the first image frame data is 30%, the degradation corresponding to the current lighting conditions should be 20%, then the reverse compensation calculation can obtain an incremental adjustment value of 10% for the texture feature, which is used to restore texture clarity. Similarly, for edge degradation and color saturation degradation, the compensation increment or adjustment ratio is calculated according to the degree of deviation of the lighting direction and the decrease in lighting uniformity, respectively. By integrating these, the first feature correction information is formed, covering the correction parameters of texture, edge, and color. For the second degradation relationship, the influence trends of airborne suspended particle concentration, humidity, and surface moisture or mud on the coal gangue on the amount of grayscale degradation and scattering degradation are used as a reference. Based on the first medium parameter, the compensation range of grayscale and scattering is calculated. For example, the relationship curve of grayscale degradation, humidity, and dust concentration shows that when humidity and dust concentration cause a 25% decrease in local grayscale while the actual degradation is 30%, the local grayscale adjustment range can be increased by 5% through reverse correction calculation. Similarly, the scattering degradation is restored by compensating for the local fogging feature to restore edge clarity and local details. Then, the compensation parameters are integrated to form the second feature correction information. Then, the first feature correction information and the second feature correction information are integrated to obtain the corresponding feature correction information.

[0062] In some possible implementations, an image correction dataset is constructed based on the aforementioned generated feature correction information and the corresponding first environmental parameters. Each first image frame data is associated with its corresponding first environmental parameter, and the feature correction information calculated through reverse compensation is applied to the data record of that first image frame data, forming a ternary correspondence between image, environment, and feature correction. For each first image frame data, the texture, edge, color, grayscale, and local haze features of the original image are digitally enhanced or their parameters are adjusted using the feature correction information, so that the image features of the first image frame data under uneven lighting, humidity, or dust effects are restored, generating a corrected image feature vector or enhanced image data. Next, the corrected features or enhanced image of each image, its corresponding original environmental parameters, and the feature correction information used for compensation are integrated and stored to form a unified data structure, which can be represented in tabular form or vectorized matrix. Each record contains the original image identifier, the first environmental parameter, the feature correction information, and the corrected image features. Each data record in the integrated image correction dataset is indexed by the corresponding first environmental parameter. To ensure the dataset covers diverse environmental conditions, images under varying light intensities, light source directions, illumination uniformity, and media parameters can be processed in the same way. All corrected images and their corresponding parameters are then accumulated to form a complete dataset. This corrected image dataset can be directly used as input for image recognition algorithms or visual sorting systems in actual sorting processes.

[0063] This application establishes a first degradation relationship and a second degradation relationship by setting first environmental parameters (light intensity, light source direction, light uniformity, and medium parameters such as dust concentration, humidity, and surface moisture or mud on coal gangue) and corresponding image degradation information. This quantifies the impact of lighting and medium conditions on texture, edge, color, grayscale, and scattering degradation. Based on this degradation relationship, inverse compensation calculations are performed on various degradation features to generate first and second feature correction information. Each first image frame data, its environmental parameters, and corresponding feature correction information are integrated to construct an image correction dataset. Applied to practical sorting, this method can restore image texture, edge, color, grayscale, and local sharpness features under conditions of uneven lighting, varying dust concentration, or surface deposits, thereby improving the recognition accuracy of the visual sorting system.

[0064] S103, acquire the second image frame data of coal gangue on the conveyor belt and the corresponding second environmental parameters, match and determine the image correction information in the image correction dataset, enhance the second image frame data based on the image correction information, and obtain the third image frame data.

[0065] To make the following embodiments clear, the relevant terms will first be explained: Second image frame data: refers to the image data of coal gangue collected in real time by industrial cameras installed above or to the side of the conveyor belt during the actual sorting operation.

[0066] Second environmental parameters: These refer to environmental parameters that are collected synchronously with the second image frame data, including second illumination parameters (illuminance, light source direction, and illumination uniformity) and second medium parameters (concentration of suspended particles in the air, humidity, and the amount of water or mud adhering to the coal gangue surface).

[0067] Illumination matching error value: refers to the difference between the second illumination parameter and the corresponding first illumination parameter in the image correction dataset; Medium matching error value: refers to the difference between the second medium parameter and the corresponding first medium parameter in the image correction dataset; Comprehensive matching error value: refers to the overall difference measure obtained by fusing the illumination matching error value and the medium matching error value.

[0068] Image correction information: refers to the feature correction information corresponding to the first environmental parameter that matches the second environmental parameter.

[0069] Third image frame data: refers to the second image frame data after image correction information enhancement processing.

[0070] For example, during the actual coal and gangue sorting operation, a second image frame data of the coal and gangue on the conveyor belt is acquired by an industrial camera set above or to the side of the conveyor belt, and a second environmental parameter is collected simultaneously.

[0071] In some possible implementations, each data record in the image correction dataset is indexed by a corresponding first environmental parameter, and a one-to-one correspondence has been established between it and the first and second feature correction information. During the matching phase, the image correction dataset is pre-loaded into memory before execution to ensure that error calculation for a single frame image is completed within a preset time window. The matching process sequentially traverses each data record in the image correction dataset, calculating the difference between the second environmental parameter and the first environmental parameter of the current data record. For the second illumination parameter, the illumination intensity is compared with the illumination intensity of the corresponding first illumination parameter to obtain the illumination intensity difference; the light source direction is compared to obtain the direction difference; and the illumination uniformity is compared to obtain the uniformity difference. All these differences are calculated using an absolute difference method to reflect the degree of deviation between the current operating environment and the historical environment. The illumination intensity difference, direction difference, and uniformity difference are directly accumulated to obtain the illumination matching error value corresponding to that data record.

[0072] After calculating the illumination matching error value, the same logic is applied to the second medium parameter. The concentration of suspended particles in the air is compared with the dust concentration in the corresponding first medium parameter to obtain the dust concentration difference; the humidity is compared to obtain the humidity difference; and the moisture or mud adhering to the coal gangue surface is compared to obtain the adhesion state difference. Each difference is calculated using the absolute difference method and directly accumulated to form the medium matching error value.

[0073] After obtaining the illumination matching error value and the medium matching error value, the two are simply summed to obtain the comprehensive matching error value of the current data record. Since the environmental components have already undergone unified scale processing in the early stage, the overall difference measurement can be achieved without introducing weight allocation or complex fusion models. The comprehensive matching error value is calculated for all data records in the image correction dataset, and the error results are sorted. The data record corresponding to the first environmental parameter with the smallest comprehensive matching error value is selected. The feature correction information (first feature correction information and second feature correction information) associated with this data record is determined as the image correction information of the current second image frame data.

[0074] After obtaining the image correction information, the image enhancement stage begins. Based on the texture increment or scaling adjustment values ​​contained in the first feature correction information, high-frequency detail areas in the second image frame data are enhanced to restore texture information attenuated by illumination to standard environmental levels. Based on edge adjustment values, areas with gradient changes in the image are strengthened to improve edge sharpness. Based on color saturation adjustment values, color channels are appropriately gained or suppressed to bring color distribution back to normal range. Simultaneously, based on the local grayscale adjustment magnitude in the second feature correction information, the image brightness is corrected overall or locally to maintain grayscale levels consistent with standard environments. Based on the scattering blur enhancement magnitude, areas with fogging or blurring are sharpened to restore local details and contrast.

[0075] After the above enhancement processing, the third image frame data is obtained. The third image frame data has undergone environmental change compensation in terms of texture, edges, color, grayscale, and local sharpness, maintaining stable image feature representation under complex lighting fluctuations, humidity changes, and dust interference conditions. Because the matching process uses deterministic logic of progressive difference accumulation, it avoids the computational burden of complex model inference, thus enabling real-time processing in actual sorting scenarios with high-speed conveyor belt operation. By directly matching and calling verified feature correction information from historical environments, this application achieves rapid linkage between environmental condition changes and image enhancement strategies, thereby improving the accuracy of coal gangue sorting and identification.

[0076] S104: Identify coal and gangue category information and pose parameters based on the third image frame data, obtain conveyor belt operation parameters, track coal and gangue trajectory based on operation parameters and pose parameters, and generate coal and gangue location information.

[0077] To make the following embodiments clear, the relevant terms will first be explained: The first trajectory refers to the predicted initial path of the coal and gangue along the conveyor belt, based on the conveyor belt speed parameters and direction of travel, without considering the coal's own inertia, rolling, or attitude changes. This trajectory only reflects the ideal path of the coal and gangue moving at a constant speed with the conveyor belt and serves as a benchmark for trajectory correction. The second trajectory refers to the trajectory based on the first trajectory, combined with the estimated values ​​of coal and gangue size parameters, tilt angle parameters, and surface normal vectors, after correcting for inertial lag and rolling tendency. This trajectory reflects the actual movement path of the coal and gangue due to its own characteristics, including attitude deviations and rolling effects.

[0078] The third motion trajectory refers to the final motion path after further compensation for conveyor belt operating errors (such as slippage and deviation) based on the second motion trajectory. This trajectory reflects the movement position of coal and gangue under actual conveyor belt conditions and is used to generate coal and gangue location information and sorting control.

[0079] Inertia effect coefficient: refers to a numerical index that quantifies the degree to which the size and mass of coal gangue lag behind the movement of the conveyor belt.

[0080] Rolling trend factor: A numerical index that quantifies the tendency of coal and gangue to roll on the conveyor belt, used to correct the attitude deviation caused by rolling in the trajectory.

[0081] In some possible implementations, target recognition and pose parameter extraction of coal gangue are performed based on the third image frame data obtained in S103.

[0082] Specifically, the third image frame data is input into a pre-defined deep learning model. This model analyzes the texture features, edge information, color distribution, grayscale changes, and local sharpness of the third image frame data to locate and classify areas in the image that may belong to coal gangue, achieving target region segmentation and category recognition. For each identified coal gangue target, the model outputs corresponding target boundary information and category information (coal or gangue), forming a preliminary recognition result. After obtaining the coal gangue boundary information and category label, the model further extracts the pose parameters of the coal gangue based on the enhanced image features. Specifically, the centroid parameter is obtained by calculating the geometric center coordinates of the target region on the image plane, that is, by analyzing the distribution of segmented pixels to determine the two-dimensional center position of the coal gangue. The size parameter is obtained by fitting the target region with a minimum bounding rectangle or ellipsoid to obtain the length, width, height, or equivalent three-dimensional dimensions of the coal gangue. The tilt angle parameter is determined by fitting the angle between the main direction lines of the coal gangue surface and the horizontal line of the image plane to determine the rotation or tilt state of the coal gangue on the conveyor belt. The surface normal vector estimate is calculated by analyzing local edge gradient, surface curvature, or surface texture direction information. It is used to describe the local directional characteristics of the coal gangue surface, thereby reflecting its orientation in space.

[0083] It should be noted that the aforementioned preset deep learning model is an existing object detection and instance segmentation model (such as YOLOv8n or YOLOv8s). Those skilled in the art can choose according to actual needs, and no specific limitations are made here.

[0084] In some possible implementations, after identifying the coal and gangue category information and pose parameters based on the third image frame data, the coal and gangue trajectory is tracked in combination with the conveyor belt operation parameters to generate corresponding position information.

[0085] In the coal and gangue sorting process, the centroid position of the coal and gangue in the third image frame data is used as the initial point. A linear translation is performed according to the conveyor belt speed and direction to obtain a two-dimensional or three-dimensional position sequence that changes over time, thus obtaining the first motion trajectory of the coal and gangue on the conveyor belt. Based on the coal and gangue size parameters and the estimated surface normal vector, the inertial influence coefficient and rolling trend factor of the coal and gangue during the motion process are calculated. Specific implementation methods include: converting the size parameters of the coal and gangue into a quantitative index of its inertial lag degree; the larger the size or the heavier the mass, the more obvious the lag degree, thus obtaining the inertial influence coefficient; analyzing the surface normal vector direction and the main motion direction of the coal and gangue, and combining it with the size information to calculate its rolling tendency degree, thus obtaining the rolling trend factor.

[0086] The first motion trajectory is combined with the tilt angle parameter of the coal gangue, the rolling trend factor and the inertia influence coefficient to correct the attitude deviation in the trajectory and obtain the second motion trajectory. At each position point along the first motion trajectory, the position of the centroid is finely adjusted and the direction is adjusted to be consistent with the offset direction predicted by the rolling trend factor and the inertia influence coefficient. At the same time, the trajectory is rotated and corrected according to the tilt angle of the coal gangue to make the trajectory closer to the actual motion state.

[0087] Based on the changing trend of the coal gangue centroid parameters over time, the amount of slippage and deviation during conveyor belt operation is detected, and the operating error is calculated. The operating error is applied to the second motion trajectory for compensation to obtain the third motion trajectory. The change of the centroid position in consecutive frames of the third image frame data is compared with the conveyor belt movement to identify the deviation (slippage or deviation). Then, the deviation is smoothly superimposed on the second motion trajectory to achieve trajectory correction and obtain the third motion trajectory. Each position point of the third motion trajectory is associated with the corresponding timestamp to form a real-time motion trajectory record of the coal gangue on the conveyor belt. Combined with the identified category information, the current position, spatial posture, and classification information of each coal gangue are output.

[0088] In some possible implementations, the motion lag time of the coal and gangue is determined based on the coal and gangue's dimensional parameters. These parameters include length, width, height, or the dimensions of an equivalent circumscribed rectangle or ellipsoid. By analyzing the volume, mass distribution, and contact area of ​​the coal and gangue on the conveyor belt, the inertial delay caused by the conveyor belt's start-up or speed change can be estimated. Specifically, the volume and density of the coal and gangue are combined to calculate an approximate mass. Then, based on the conveyor belt's speed change rate and the coal and gangue's contact friction conditions, the time interval required for the coal and gangue to reach synchronous speed with the conveyor belt—the motion lag time—is estimated. After obtaining the motion lag time, it is normalized to a value between 0 and 1, so that it can be directly used as an inertial influence coefficient in subsequent trajectory correction. The logic of normalization is that the maximum possible lag time corresponds to an inertial influence coefficient of 1, and the minimum lag time corresponds to an inertial influence coefficient of 0, thus forming a standardized quantitative index that is easy to combine with other factors to correct the trajectory.

[0089] Based on the estimated surface normal vector and dimensional parameters of the coal gangue, the tendency of the coal gangue to roll on the conveyor belt is calculated. Specifically, by analyzing the angle between the main direction lines of the coal gangue surface and the horizontal plane, as well as the distribution characteristics of the surface normal vector in the longitudinal and transverse directions, and combining this with the size and mass distribution of the coal gangue, the rolling inertia and rolling probability are inferred. If the coal gangue is long and the surface normal vector points towards the direction of movement, the rolling tendency is high; if the coal gangue is wide and thick, and the normal vector deviates from the direction of movement, the rolling tendency is low. The rolling tendency is normalized and converted into a rolling trend factor between 0 and 1, which is used for attitude adjustment and rolling compensation of the first motion trajectory in subsequent trajectory correction.

[0090] This application inputs the enhanced third image frame data into a preset target detection and instance segmentation model to identify coal and gangue targets in the image and extract their pose parameters. Combining the conveyor belt speed and running direction parameters, the trajectory of each coal and gangue is predicted and corrected to generate a first motion trajectory. Then, the trajectory posture is corrected by the inertia influence coefficient and the rolling trend factor to obtain a second motion trajectory. Finally, compensation is made based on the slippage and deviation errors of the conveyor belt to generate a third motion trajectory, thereby forming real-time position and classification information of coal and gangue on the conveyor belt. This enables stable output of classification and position information even under complex lighting, dust interference, and diverse morphological conditions, improving the accuracy of automatic sorting and identification.

[0091] S105 generates control commands based on the coal and gangue category and location information to control the sorting equipment to complete the coal and gangue sorting.

[0092] In some possible implementation methods, control commands are generated based on coal gangue category information and corresponding real-time location information to drive sorting execution equipment (such as pneumatic push rods, robotic arms, or electromagnetic sorting mechanisms) to complete coal gangue sorting.

[0093] Specifically, the current position (centroid coordinates) of each piece of coal gangue is combined with the conveyor belt speed and direction of travel to predict the time window for the coal gangue to reach the sorting point. Based on category information, different categories of coal gangue are mapped to corresponding sorting target areas, such as sending coal to the raw coal channel and sending gangue to the waste channel. Based on the predicted arrival time and the response time of the sorting execution equipment, the required action time, action path, and action amplitude for each piece of coal gangue are calculated, forming a sorting action sequence. In the control command generation, the sorting action sequence is converted into signals executable by the sorting execution equipment, including cylinder extension / retraction pulses, motor rotation angles, and speed control values. Based on kinematic and dynamic analysis, the action trajectory and acceleration requirements of the execution equipment are calculated according to the speed and position of the coal gangue and the mechanical parameters of the sorting device to ensure accurate target positioning when the coal gangue arrives at the sorting point, achieving grabbing or pushing. The control logic can employ time-synchronous control or feedback control, where feedback control fine-tunes the action by real-time detection of the coal gangue position, improving accuracy.

[0094] For example, when a piece of gangue on the conveyor belt is identified based on the location information and is predicted to arrive at the sorting port in 2 seconds, the pneumatic pusher is calculated to start extending at 1.9 seconds and push the gangue into the waste channel at an appropriate speed when it arrives at the sorting port. At the same time, for the moving coal block, the action of another pusher is adjusted according to the category information so that the coal block enters the raw coal channel, ensuring that multiple pieces of material are sorted at the same time without collision or missed sorting.

[0095] This application acquires coal gangue images and environmental parameters offline, quantifies multi-dimensional degradation (texture, edge, grayscale, saturation, scattering), establishes the degradation relationship between environmental parameters and image features, generates targeted feature correction information, and constructs an image correction dataset. In actual coal gangue sorting, the feature correction information is matched online in real time to adaptively enhance the image, and the trajectory tracking is corrected by combining the physical characteristics of coal gangue. This enables accurate identification and positioning of coal gangue in complex environments such as high dust, high humidity, and uneven lighting in coal mine conveyor belts. In actual underground or coal preparation workshop applications, this improves the identification accuracy and thus increases sorting efficiency.

[0096] Example 2: This application provides a computer vision-based intelligent coal gangue positioning system. The system's module functions correspond to the specific implementation steps S101 to S105 in Example 1, including a data acquisition module, an image processing module, a positioning module, and a control module. Please refer to [link to relevant documentation]. Figure 3 .

[0097] The data acquisition module is used to acquire the first image frame data of coal and gangue on the conveyor belt and the corresponding first environmental parameters. During the coal and gangue sorting process, it acquires the second image frame data of coal and gangue on the conveyor belt and the corresponding second environmental parameters. The image processing module includes a feature extraction unit, a degradation analysis unit, and a matching enhancement unit; The feature extraction unit is used to extract image features from the first image frame data, calculate the degradation amount of the image features relative to the preset benchmark features, and obtain image degradation information; The degradation analysis unit is used to establish the degradation relationship between the first environmental parameter and the image features based on the image degradation information, and generate corresponding feature correction information. Based on the feature correction information and the corresponding first environmental parameter, the corresponding image correction dataset is constructed. The matching enhancement unit is used to perform matching in the image correction dataset based on the second environmental parameters, determine the corresponding image correction information, and perform image enhancement on the second image frame data based on the image correction information to obtain the corresponding third image frame data. The positioning module includes an identification unit and a trajectory tracking unit; The identification unit is used to identify the coal and gangue on the conveyor belt based on the third image frame data, and obtain the corresponding category information and pose parameters; The trajectory tracking unit is used to acquire the operating parameters of the conveyor belt. Based on the operating parameters and pose parameters, it tracks the coal and gangue on the conveyor belt and generates corresponding position information. The control module is used to generate corresponding control commands based on category and location information, and to control the sorting equipment to sort the coal and gangue on the conveyor belt using the control commands.

[0098] The functions of the above modules correspond to the method description in Example 1, and will not be described in detail here.

[0099] This application achieves the positioning and intelligent sorting of coal gangue in complex conveyor belt environments through image enhancement, target recognition, trajectory tracking, and sorting control. It can compensate for the influence of changes in light, dust, and humidity on the image, accurately extract the coal gangue category and pose parameters, and track the motion trajectory in real time in combination with the operating characteristics of the conveyor belt, thereby generating corresponding control commands, improving sorting efficiency and accuracy, and realizing efficient intelligent sorting of coal gangue.

[0100] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this application. Various changes and modifications can be made to this application without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of this application as claimed. The scope of protection of this application is defined by the appended claims and their equivalents.

Claims

1. A computer vision-based intelligent coal gangue positioning method, characterized in that, The method includes: Acquire the first image frame data of coal gangue on the conveyor belt and the corresponding first environmental parameters; Extract image features from the first image frame data, calculate the degradation amount of the image features relative to a preset reference feature, and obtain image degradation information; Based on the image degradation information, a degradation relationship between the first environmental parameter and the image feature is established, and corresponding feature correction information is generated. Based on the feature correction information and the corresponding first environmental parameter, a corresponding image correction dataset is constructed. During the coal and gangue sorting process, the second image frame data of the coal and gangue on the conveyor belt and the corresponding second environmental parameters are acquired. Based on the second environmental parameters, the image correction dataset is matched to determine the corresponding image correction information. Based on the image correction information, the second image frame data is enhanced to obtain the corresponding third image frame data. Based on the third image frame data, the coal and gangue on the conveyor belt are identified to obtain the corresponding category information and pose parameters. The operating parameters of the conveyor belt are obtained, and based on the operating parameters and pose parameters, the trajectory of the coal and gangue on the conveyor belt is tracked to generate corresponding position information. Based on the category information and location information, corresponding control instructions are generated, and the sorting execution equipment is controlled to sort the coal and gangue on the conveyor belt using the control instructions.

2. The method according to claim 1, characterized in that, The step of extracting image features from the first image frame data and calculating the degradation amount of the image features relative to a preset benchmark feature includes: Multi-scale feature decomposition is performed on the first image frame data to obtain texture features, edge gradient features, grayscale distribution features, color saturation features, and local fogging features. The degradation amount of the texture feature, the edge gradient feature, the grayscale distribution feature, the color saturation feature, and the local haze feature relative to the preset benchmark feature is calculated to obtain the corresponding image degradation information.

3. The method according to claim 2, characterized in that, The step of calculating the degradation amount of the texture feature, the edge gradient feature, the grayscale distribution feature, the color saturation feature, and the local haze feature relative to the preset benchmark feature to obtain the corresponding image degradation information includes: For the texture feature, the relative attenuation degree between the texture feature and the preset reference feature is calculated as the texture degradation amount; For the edge gradient feature, the intensity reduction between the edge gradient feature and the preset benchmark feature is calculated as the edge degradation amount; For the grayscale distribution feature, the statistical distribution deviation between the grayscale distribution feature and the preset benchmark feature is calculated as the grayscale degradation amount; For the color saturation feature, the saturation attenuation ratio between the color saturation feature and the preset reference feature is calculated as the color saturation degradation amount; For the local fogging feature, the non-uniformity deviation between the local fogging feature and the preset reference feature is calculated as the scattering degradation amount; The texture degradation, edge degradation, grayscale degradation, color saturation degradation, and scattering degradation are integrated to form the corresponding image degradation information.

4. The method according to claim 3, characterized in that, The step of establishing a degradation relationship between the first environmental parameter and the image feature based on the image degradation information, and generating corresponding feature correction information, includes: Based on the image degradation information, analyze the degradation variation law of the first image frame data under the first environmental parameter, and establish the degradation relationship between the first environmental parameter and the image feature; Based on the degradation relationship, corresponding feature correction information is generated for the first image frame data.

5. The method according to claim 4, characterized in that, The first environmental parameters include a first illumination parameter and a first medium parameter. The step of analyzing the degradation variation law of the first image frame data under the first environmental parameters based on the image degradation information, and establishing the degradation relationship between the first environmental parameters and the image features, includes: Based on the influence trend of the first illumination parameter on the texture degradation, edge degradation, and color saturation degradation, a corresponding first degradation relationship is established; Based on the influence trend of the first medium parameter on the grayscale degradation and the scattering degradation, a second degradation relationship between the medium parameter and the corresponding degradation is established; By comprehensively analyzing the first degradation relationship and the second degradation relationship, the degradation relationship between the first environmental parameter and the image feature is determined.

6. The method according to claim 5, characterized in that, The step of generating corresponding feature correction information for the first image frame data based on the degradation relationship includes: Based on the first degradation relationship, the texture degradation amount, the edge degradation amount, and the color saturation degradation amount are respectively calculated in reverse to obtain the corresponding first feature correction information; Based on the second degradation relationship, the grayscale degradation amount and the scattering degradation amount are calculated in reverse to obtain the corresponding second feature correction information; The first feature correction information and the second feature correction information are used as the feature correction information corresponding to the first image frame data.

7. The method according to claim 1, characterized in that, The second environmental parameters include a second illumination parameter and a second medium parameter. The step of matching the image correction dataset based on the second environmental parameters to determine the corresponding image correction information includes: The second illumination parameter is compared with the first illumination parameter in the image correction dataset to calculate the parameter difference, and the illumination matching error value is obtained. The media matching error value is obtained by calculating the parameter difference between the second medium parameter and the first medium parameter in the image correction dataset. The illumination matching error value and the medium matching error value are fused to obtain a comprehensive matching error value; The feature correction information corresponding to the first environmental parameter with the smallest comprehensive matching error value is selected from the image correction dataset and used as the image correction information of the second image frame data.

8. The method according to claim 1, characterized in that, The operating parameters include conveyor belt speed parameters and conveyor belt running direction; the pose parameters include coal and gangue centroid parameters, coal and gangue size parameters, coal and gangue inclination angle parameters, and estimated values ​​of coal and gangue surface normal vectors. Based on the operating parameters and pose parameters, trajectory tracking of the coal and gangue on the conveyor belt is performed to generate corresponding position information, including: Based on the conveyor belt speed parameters and the conveyor belt running direction, calculate the first motion trajectory of the coal and gangue on the conveyor belt; Based on the coal and gangue size parameters and the estimated value of the coal and gangue surface normal vector, calculate the inertial influence coefficient and rolling trend factor of the coal and gangue during the conveyor belt movement process; Based on the coal gangue tilt angle parameter, the rolling trend factor, and the inertial influence coefficient, the attitude deviation in the first motion trajectory is corrected to obtain the second motion trajectory; Based on the changing trend of the coal gangue centroid parameters, the slippage and deviation of the conveyor belt are detected to obtain the operating error. The second motion trajectory is then compensated based on the operating error to obtain the third motion trajectory. Based on the third motion trajectory, the location information of the coal gangue is generated.

9. The method according to claim 8, characterized in that, The step of calculating the inertial influence coefficient and rolling trend factor of the coal and gangue during the conveyor belt movement process based on the coal and gangue size parameters and the estimated value of the coal and gangue surface normal vector includes: Based on the coal gangue size parameters, the motion lag time of the coal gangue on the conveyor belt is calculated, and the motion lag time is normalized as the inertial influence coefficient. Based on the estimated surface normal vector of the coal gangue and the size parameters of the coal gangue, the tendency of the coal gangue to roll on the conveyor belt is calculated, and the tendency is normalized as a rolling trend factor.

10. A computer vision-based intelligent positioning system for coal gangue, characterized in that, The system includes: Data acquisition module, image processing module, positioning module, and control module; The data acquisition module is used to acquire the first image frame data of coal and gangue on the conveyor belt and the corresponding first environmental parameters. During the coal and gangue sorting process, it acquires the second image frame data of coal and gangue on the conveyor belt and the corresponding second environmental parameters. The image processing module includes a feature extraction unit, a degradation analysis unit, and a matching enhancement unit; The feature extraction unit is used to extract image features from the first image frame data, calculate the degradation amount of the image features relative to a preset benchmark feature, and obtain image degradation information. The degradation analysis unit is used to establish a degradation relationship between the first environmental parameter and the image feature based on the image degradation information, generate corresponding feature correction information, and construct a corresponding image correction dataset based on the feature correction information and the corresponding first environmental parameter. The matching enhancement unit is used to perform matching in the image correction dataset based on the second environmental parameters, determine the corresponding image correction information, and perform image enhancement on the second image frame data based on the image correction information to obtain the corresponding third image frame data. The positioning module includes an identification unit and a trajectory tracking unit; The identification unit is used to identify the coal and gangue on the conveyor belt based on the third image frame data, and obtain the corresponding category information and pose parameters; The trajectory tracking unit is used to acquire the operating parameters of the conveyor belt, and based on the operating parameters and pose parameters, to track the coal and gangue on the conveyor belt and generate corresponding position information. The control module is used to generate corresponding control commands based on the category information and location information, and use the control commands to control the sorting execution equipment to sort the coal and gangue on the conveyor belt.

Images