Method for detecting foreign matter on a coal conveying belt in a coal conveying system

Abstract

The present application relates to coal conveying safety monitoring technical field, disclose a kind of detection method of foreign matter on coal conveying belt in coal conveying system.The method includes by collecting laser scanning point cloud and spectral imaging information from multiple azimuth angle, establish the multidimensional light field model of the geometric topology and material reflection characteristic of belt surface, and simultaneously obtain driving motor power spectrum and bearing vibration spectrum.Correlate above-mentioned multi-source information, carry out three-dimensional dynamic reconstruction and material marking to conveying load, and then separate out the non-coal material geometry mixed in coal flow.Perform spectral feature inversion and mechanical response simulation based on material physical properties to the separated foreign matter geometry, and the results are adaptively matched with the non-coal material standard feature library, to accurately identify the type of foreign matter and evaluate the confidence.According to the identification result, the mechanical arm at the specified position is triggered to perform end effector path planning.The present application realizes accurate, robust detection and positioning of foreign matter on coal conveying belt.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coal conveying safety monitoring, in particular to a method for detecting foreign matter on a coal conveying belt in a coal conveying system. BACKGROUND

[0002] In the safe operation of a coal conveying system, foreign matter such as gangue, wood blocks, metal, plastic and the like mixed into the belt conveyor is a major hidden danger. Existing detection technologies mainly rely on visible light cameras for two-dimensional image recognition, or use metal detectors, laser profile scanning and the like single sensor. These methods can only perceive the surface gray scale, color or contour shape of the target, and cannot obtain the accurate three-dimensional geometric structure, depth information and material composition. In a complex and harsh coal conveying environment, coal dust attachment, water stains, light changes and the diversity of the shape and color of coal itself can easily lead to misjudgment or missed reports of algorithms based on two-dimensional images. Although the metal detector is effective for ferromagnetic materials, it is completely ineffective for non-metallic foreign matter, and cannot provide the size, position and shape information of the foreign matter.

[0003] Existing technical solutions mostly use independent processing of sensor data or simple logic parallel connection. This results in the system being difficult to distinguish between normal large coal and foreign matter, and being unable to understand the real embedded state and mechanical influence of the foreign matter in the moving load. Even if three-dimensional scanning is used, it is limited to static geometric reconstruction, and the key dynamic parameters reflecting the load quality distribution and motion state such as the driving power of the belt and bearing vibration are not integrated into the perception model. In the back-end identification link, existing methods mostly rely on foreign matter image template matching or setting a simple vibration threshold, and the identification is based on the appearance characteristics, cannot penetrate the appearance of pollution and obstruction, can identify foreign matter wrapped in coal or similar in material to coal, and lacks analysis and verification mechanisms for the intrinsic physical properties of foreign matter. SUMMARY

[0004] The purpose of the present application is to provide a method for detecting foreign matter on a coal conveying belt in a coal conveying system to solve the problems raised in the background.

[0005] To achieve the above purpose, the present application provides a method for detecting foreign matter on a coal conveying belt in a coal conveying system, which comprises:

[0006] A multi-dimensional light field model of the running surface of the coal conveying belt is established, the multi-dimensional light field model is constructed based on laser scanning point clouds and spectral imaging information collected from multiple azimuth angles, and is used to represent the geometric topology and material reflection characteristics of the running surface of the coal conveying belt;

[0007] Real-time driving motor power spectrum and bearing vibration frequency spectrum associated with the coal conveying belt are obtained;

[0008] According to the geometric topology and material reflection characteristics, and in association with the real-time driving motor power spectrum and bearing vibration spectrum, the volume shape of the conveying load is three-dimensionally dynamically reconstructed and material-labeled;

[0009] Based on the results of the three-dimensional dynamic reconstruction and material labeling, non-coal material geometric bodies attached to the surface of the conveying load or mixed in the conveying coal flow are separated out;

[0010] The non-coal material geometric bodies are subjected to spectrum feature inversion and mechanical response simulation based on material physical properties, and according to the results of the spectrum feature inversion and mechanical response simulation, adaptive mode matching is performed with entries in a non-coal material standard feature library;

[0011] Based on the foreign matter identified through adaptive mode matching and its confidence assessment, an end effector path planning is triggered for a mechanical arm deployed at a specified position along the coal conveying belt.

[0012] Preferably, a multi-dimensional light field model of the running surface of the coal conveying belt is established, including:

[0013] A high-density point cloud coordinate sequence is obtained by continuously scanning the cross section of the coal conveying belt through a linear array laser radar; a continuous spectral image cube within the same field of view range is obtained synchronously through a hyperspectral imager;

[0014] The high-density point cloud coordinate sequence and the continuous spectral image cube are time-stamped aligned and spatial coordinate system registered; according to the registered data, the normal vector, curvature and reflection intensity at different wave bands of each spatial point are calculated, forming a composite feature vector field fusing spatial geometry and spectral reflection characteristics;

[0015] The composite feature vector field is defined as the multi-dimensional light field model.

[0016] Preferably, the volume shape of the conveying load is three-dimensionally dynamically reconstructed and material-labeled, including:

[0017] A dynamic point cloud set and a spectral data stream corresponding to the moving coal flow are extracted from the composite feature vector field; the dynamic point cloud set is time series integrated to generate a continuous volume three-dimensional model of the conveying load within a detection period, in combination with the belt running speed;

[0018] According to the characteristic spectral curves of different pixels in the spectral data stream, each region on the surface of the continuous volume three-dimensional model is labeled for coal quality classification with reference to a known coal spectral database; regions with a difference in spectral feature curve from the known coal spectral feature curve exceeding a threshold value are labeled as regions to be identified;

[0019] The continuous volume three-dimensional model of the conveying load within the detection period is generated, including:

[0020] Based on the fixed period timestamp, snapshots of multiple continuous time points are intercepted from the composite feature vector field, each snapshot containing the current time coal conveying belt running surface spatial point cloud and its corresponding spectral vector;

[0021] According to the belt running speed, the displacement amount of the coal conveying belt surface along the running direction within the time interval between two adjacent snapshots is calculated;

[0022] According to the displacement amount, the three-dimensional coordinates of all spatial points in the snapshot at the subsequent time are inversely translated and compensated, and the spatial point cloud data of all snapshots are converted to the same static coordinate system with the first snapshot as the reference;

[0023] In the static coordinate system, the multiple frames of spatial point cloud data after time series registration are superimposed;

[0024] The superimposed point cloud is subjected to three-dimensional spatial voxelization processing, and a continuous closed triangular mesh model covering the entire detection period is generated based on the Poisson surface reconstruction algorithm, which represents the surface of the conveying load, as the continuous volumetric three-dimensional model.

[0025] Preferably, the non-coal substance geometric body attached to the surface of the conveying load or mixed in the conveying coal flow is separated, including:

[0026] The spatial distribution, boundary morphology, and geometric connection relationship with the surrounding coal quality area of the to-be-discriminated area in the continuous volumetric three-dimensional model are analyzed;

[0027] The spatial region with a closed boundary and a material label different from the surrounding coal quality area is subjected to region growing segmentation, and an independent geometric body point cloud cluster is extracted;

[0028] The bounding box size, centroid position, and point cloud density distribution of each geometric body point cloud cluster are calculated to form the spatial description parameters of the non-coal substance geometric body.

[0029] Preferably, the non-coal substance geometric body is subjected to spectral feature inversion and mechanical response simulation based on the material physical properties, including:

[0030] For each non-coal substance geometric body point cloud cluster, a complete spectral reflectance curve is resampled from the corresponding original hyperspectral image cube;

[0031] The spectral reflectance curve is inverted using a radiation transfer model to estimate the physical property parameters of the surface roughness and dielectric constant of the non-coal substance geometric body;

[0032] The spatial description parameters of the non-coal substance geometric body are input into the discrete element simulation environment to simulate its contact force and motion trajectory with the belt, coal, and roller under the current coal flow motion state, and output the estimated vibration response signal and friction coefficient of the geometric body.

[0033] Preferably, the adaptive pattern matching with the entries in the non-coal substance standard feature library includes:

[0034] Combining the estimated physical property parameters with the simulated output of the estimated vibration response signal and the friction coefficient into a feature vector to be identified;

[0035] Inputting the feature vector to be identified into a pre-trained deep matching network, and the deep matching network is established by a contrast learning method to measure the similarity of each foreign object feature vector in the standard feature library;

[0036] The deep matching network outputs a matching score of the most similar foreign object category in the standard feature library, and the category with a matching score exceeding an activation threshold is determined as the identification result.

[0037] Preferably, the foreign object and its confidence evaluation are identified by adaptive pattern matching, including:

[0038] The deep matching network outputs a similarity score set of the feature vector to be identified and each pre-defined foreign object category reference vector in the standard feature library;

[0039] Applying a softmax function to the similarity score set for normalization, converting it into a probability distribution form, wherein the probability value of each category represents its confidence as an identification result;

[0040] Setting a confidence threshold, and determining the entry with a highest probability value greater than the confidence threshold and a category probability significantly higher than other categories as the final foreign object identification result, and the highest probability value as the confidence evaluation of the identification result.

[0041] Preferably, the radiation transfer model is used to inverse the spectral reflectance curve to estimate the physical property parameters of the surface roughness and dielectric constant of the non-coal substance geometry, including:

[0042] Selecting the Hapke model as the radiation transfer model, which describes the bidirectional reflectance of a non-homogeneous rough surface under specific illumination and observation geometry;

[0043] Inputting the surface normal distribution in the spatial description parameters of the non-coal substance geometry, and the azimuth and zenith angles of the light source and the sensor as known geometric parameters of the Hapke model;

[0044] Taking the complete spectral reflectance curve as the target reflectance data to be fitted by the model;

[0045] Establishing a target function with the surface roughness parameter and the dielectric constant parameter as optimization variables, and the target function is defined as the root mean square error of the theoretical reflectance calculated by the model and the target reflectance data at all wave bands.

[0046] The Levenberg-Marquardt optimization algorithm is used to iteratively solve the objective function until the root mean square error is less than the preset accuracy or the maximum number of iterations is reached, and the surface roughness parameter value and the dielectric constant parameter value obtained at this time are taken as the estimated results of the physical property parameters.

[0047] Preferably, the to-be-identified feature vector is input into a pre-trained deep matching network, and the deep matching network establishes similarity measurement with each foreign object feature vector in the standard feature library through a contrast learning method, including:

[0048] A deep matching network is constructed, and the deep matching network includes two identical sub-networks sharing weights, and each sub-network is composed of multiple fully connected layers and batch normalization layers.

[0049] The to-be-identified feature vector and an entry feature vector in the standard feature library are respectively input into two sub-networks of the deep matching network, and each obtains a corresponding embedding vector through forward propagation.

[0050] The cosine similarity between the embedding vector of the to-be-identified feature vector and the embedding vector of the entry feature vector is calculated as the similarity measurement score of this matching.

[0051] In the training phase, positive sample pairs and negative sample pairs are constructed, the positive sample pairs are composed of different sample feature vectors of the same foreign object category, and the negative sample pairs are composed of sample feature vectors of different foreign object categories.

[0052] The network is trained using a triplet loss function, and the similarity scores between the positive sample pairs are higher than the similarity scores between the negative sample pairs by a set boundary value through optimization, so that the deep matching network learns effective feature representation and similarity measurement criteria.

[0053] Preferably, based on the foreign object identified through adaptive pattern matching and the confidence evaluation, a mechanical arm deployed at a specified position along the coal belt triggers the end effector path planning, including:

[0054] The recognition result output by the adaptive pattern matching is analyzed to obtain the spatial position coordinates, geometric dimensions, category attributes and matching degree scores of the foreign object as the confidence evaluation;

[0055] According to the spatial position coordinates, geometric dimensions and belt running speed of the foreign object, the predicted time and position of the foreign object moving into the working range of the nearest mechanical arm within a future time window are predicted.

[0056] selecting a suitable tool type from an end effector library based on the category attribute of the foreign object, and setting a pose target for the grasping or cleaning task based on the geometric size and predicted position of the foreign object;

[0057] inputting the pose target, the initial pose of the robot arm, and the environmental obstacle information between the current coal flow obtained by three-dimensional dynamic reconstruction and the robot arm base into a path planning algorithm based on a rapidly-explanding random tree;

[0058] running the path planning algorithm to calculate a collision-free motion trajectory of each joint of the robot arm from the initial pose to the pose target, and issuing the trajectory to the corresponding robot arm controller for execution.

[0059] Preferably, the method further comprises:

[0060] After identifying the foreign object through adaptive pattern matching, storing the to-be-identified feature vector, the corresponding identification result and the confidence evaluation as new samples in a dynamic learning sample library;

[0061] periodically using the new samples in the dynamic learning sample library to perform incremental fine-tuning training on the deep matching network to update the network weights;

[0062] adding the feature vectors of successfully identified samples with a confidence evaluation continuously higher than a set threshold to the non-coal substance standard feature library after manual review and confirmation, to realize self-expansion and update of the feature library.

[0063] Preferably, the continuous scanning of the coal conveying belt cross section by the linear array laser radar comprises:

[0064] deploying at least three linear array laser radar scanning heads above the belt and obliquely above both sides of the belt in the transverse direction of the coal conveying belt to ensure that the belt running surface and the side surface of the accumulated coal flow are covered without dead angles;

[0065] controlling the scanning frequency of each scanning head to match the belt running speed, so that the point cloud sampling density in the direction of belt running remains consistent;

[0066] performing real-time fusion and denoising processing on the point cloud data from the at least three scanning heads to generate the high-density point cloud coordinate sequence.

[0067] Preferably, the selecting a suitable tool type from an end effector library comprises:

[0068] The end effector library pre-stores a plurality of types of tools, including electromagnetic suction cups, multi-fingered grippers, hooks and rakes, and high-pressure air nozzles;

[0069] According to the category attribute of the foreign matter, a predefined rule is called to select a tool: an electromagnetic chuck is selected for a metal foreign matter, a multi-fingered gripper is selected for a block-shaped non-metal foreign matter, a hook rake is selected for a woven or belt-shaped foreign matter, and a high-pressure air nozzle is selected for a powder-shaped attached foreign matter.

[0070] Preferably, the method further comprises:

[0071] After the continuous volume three-dimensional model is generated, the total volume and total mass of the conveying load in the current detection period are calculated;

[0072] According to the belt running speed and the detection time length, the average flow rate is calculated;

[0073] The total volume, total mass and average flow rate are cross-verified with the effective power value extracted from the real-time drive motor power spectrum, and if the consistency is lower than a preset threshold, system calibration or alarm is triggered.

[0074] Preferably, the method further comprises:

[0075] A historical foreign matter event database is established to record the category, occurrence time, spatial position, and corresponding conveying load spectrum features and drive motor power spectrum features of each identified foreign matter;

[0076] Based on the historical foreign matter event database, a time series analysis method is used to analyze the periodicity of the occurrence of a specific foreign matter category or the correlation with a specific coal quality spectrum feature;

[0077] According to the analysis result, the data acquisition frequency is actively increased or the sensitivity threshold of the adaptive mode matching is adjusted during a predicted foreign matter high-occurrence period or for a specific marked coal flow.

[0078] Compared with the prior art, the present application has the following beneficial effects:

[0079] By fusing laser scanning point clouds and spectral imaging information from multiple azimuth angles, a multi-dimensional light field model representing the geometric topology and material reflection characteristics of the target is constructed, and the real-time drive motor power spectrum and bearing vibration frequency spectrum are associated to realize three-dimensional dynamic reconstruction and material marking of the conveying load. Based on the fusion information of geometry, material and dynamic characteristics, the system can accurately separate non-coal substances with different forms, reflection characteristics and dynamic responses from the dynamic and complex coal flow background, improving the perception accuracy and robustness of the three-dimensional form and position of the foreign matter under the interference of coal dust and water stains, and overcoming the problem that traditional two-dimensional vision is easily disturbed by the environment and cannot perceive depth.

[0080] For the isolated non-coal material geometry, the spectrum feature inversion based on material physical properties and the mechanical response simulation are performed, the intrinsic material characteristics obtained by inversion and the simulated dynamic behavior are adaptively matched with the non-coal material standard feature library. The identification basis is deepened from the surface color, shape and other changeable characteristics to the internal composition of the material and its theoretical mechanical response under specific working conditions. The spectrum inversion can identify non-metallic foreign matters covered by coal dust or similar in color, and the mechanical response simulation can compare the theoretical vibration characteristics of the foreign matter with the measured frequency spectrum, providing cross verification at the physical level for the identification result, thereby realizing more essential and accurate identification of the foreign matter, reducing the false positive rate and the false negative rate, and being able to distinguish different types of foreign matters. BRIEF DESCRIPTION OF DRAWINGS

[0081] Figure 1 The working principle diagram of the foreign matter detection method in the coal conveying system is described.

[0082] Figure 2 The flowchart for establishing a multi-dimensional light field model is described.

[0083] Figure 3 The flowchart for spectrum feature inversion and mechanical response simulation is described.

[0084] Figure 4 The dynamic learning stage-feature library expansion and network training effect trend are described.

[0085] Figure 5 The scatter plot of the spectrum feature principal component and the foreign matter occurrence probability is described. DETAILED DESCRIPTION

[0086] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0087] Please refer to Figure 1The application provides a foreign matter detection method for a coal conveying belt in a coal conveying system, and the method comprises the following steps: establishing a multi-dimensional light field model of a running surface of the coal conveying belt, the model being constructed based on laser scanning point clouds and spectral imaging information collected from multiple azimuth angles, and being used to represent the geometric topology and material reflection characteristics of the running surface of the coal conveying belt; acquiring a real-time driving motor power spectrum and a bearing vibration frequency spectrum associated with the coal conveying belt; according to the geometric topology and material reflection characteristics and in association with the real-time driving motor power spectrum and the bearing vibration frequency spectrum, performing three-dimensional dynamic reconstruction and material labeling on the volume form of the conveying load; based on the results of the three-dimensional dynamic reconstruction and material labeling, separating out non-coal material geometric bodies attached to the surface of the conveying load or mixed in the conveying coal flow; performing spectral feature inversion and mechanical response simulation based on the material physical properties on the non-coal material geometric bodies, and according to the results of the spectral feature inversion and mechanical response simulation, performing adaptive mode matching with entries in a non-coal material standard feature library; based on the foreign matter and its confidence evaluation identified through the adaptive mode matching, triggering a mechanical arm deployed at a specified position along the coal conveying belt to perform end effector path planning.

[0088] Embodiment 1: refer to Figure 2 The multi-dimensional light field model of the running surface of the coal conveying belt is established, the cross section of the coal conveying belt is continuously scanned by a linear array laser radar to obtain a high-density point cloud coordinate sequence, at least three linear array laser radar scanning heads are deployed above the coal conveying belt and on the two sides of the coal conveying belt, the running surface of the coal conveying belt and the side surface of the accumulated coal flow are ensured to be covered without dead angle, the scanning frequency of each scanning head is controlled to match the running speed of the belt, the point cloud sampling density along the running direction of the belt is kept consistent, the point cloud data from the at least three scanning heads is fused and denoised in real time to generate the high-density point cloud coordinate sequence, and a continuous spectral image cube in the same field of view is obtained by a hyperspectral imager at the same time; the high-density point cloud coordinate sequence and the continuous spectral image cube are time-stamped and aligned and space coordinate systems are registered, the normal vector, the curvature and the reflection intensity at different wave bands of each space point are calculated according to the registered data, and a composite feature vector field of fused space geometry and spectral reflection characteristics is formed; the composite feature vector field is defined as the multi-dimensional light field model.

[0089] The volume shape of the conveying load is three-dimensionally dynamically reconstructed and material marked, a dynamic point cloud set corresponding to the moving coal flow and a spectral data stream are extracted from the composite feature vector field, the time series integral of the dynamic point cloud set is performed in combination with the belt running speed, a continuous volume three-dimensional model of the conveying load in the detection period is generated, based on the fixed period timestamp, snapshots of multiple continuous time instants are intercepted from the composite feature vector field, each snapshot contains the coal conveying belt running surface space point cloud at the current time instant and its corresponding spectral vector, according to the belt running speed, the displacement amount of the coal conveying belt surface along the running direction within the time interval of two adjacent snapshots is calculated, according to the displacement amount, the three-dimensional coordinates of all space points in the snapshot at the subsequent time instant are inversely translated and compensated, the space point cloud data of all snapshots is converted to the same static coordinate system with the first snapshot as the reference, in the static coordinate system, the multiple frames of space point cloud data after time series registration are superimposed, the superimposed point cloud is three-dimensionally voxelized, and based on the Poisson surface reconstruction algorithm, a continuous closed triangular mesh model covering the entire detection period is generated, which represents the surface of the conveying load, as the continuous volume three-dimensional model; according to the characteristic spectral curves of different pixels in the spectral data stream, referring to the known coal spectral database, the coal quality classification marking of each region on the surface of the continuous volume three-dimensional model is performed, and the region with a spectral characteristic curve difference exceeding a threshold value from the known coal spectral characteristic curve is marked as a region to be identified. After the continuous volume three-dimensional model is generated, the total volume and total mass of the conveying load in the current detection period are calculated, the average flow is calculated according to the belt running speed and the detection time length, the total volume, total mass and average flow are cross-verified with the effective power value extracted from the real-time driving motor power spectrum, and if the consistency is lower than a preset threshold, system calibration or alarm is triggered.

[0090] In a specific implementation, a multi-dimensional light field model of the coal conveying belt running surface is established, a high-density point cloud coordinate sequence is obtained by continuously scanning the cross section of the coal conveying belt with a linear array laser radar, three linear array laser radar scanning heads are arranged above the belt and on the two sides of the belt, ensuring that the belt running surface and the side of the accumulated coal flow are covered without dead angle, the scanning frequency of the upper scanning head and the two oblique upper scanning heads is matched with the belt running speed, so that the point cloud sampling density along the belt running direction remains consistent, the point cloud data from the three scanning heads is fused and denoised in real time based on statistical filtering, and the high-density point cloud coordinate sequence is generated; a hyperspectral imager mounted on the same rigid support is used to obtain a continuous spectral image cube in the same field of view range. The high-density point cloud coordinate sequence and the continuous spectral image cube are time-stamped based on a hardware trigger signal and space coordinate system registered based on a calibration board, and based on the registered data, the normal vector, curvature and reflection intensity at different wavebands of each spatial point are calculated to form a composite feature vector field that fuses spatial geometry and spectral reflection characteristics; the composite feature vector field is defined as the multi-dimensional light field model.

[0091] In some embodiments, the volume morphology of the conveying load is three-dimensionally dynamically reconstructed and material marked, a dynamic point cloud set corresponding to the moving coal flow and a spectral data stream are extracted from the composite feature vector field, the dynamic point cloud set is time series integrated based on the belt running speed to generate a continuous volume three-dimensional model of the conveying load within a detection period, a plurality of snapshots at consecutive time instants are intercepted from the composite feature vector field based on a fixed period time stamp, each snapshot contains a spatial point cloud of the coal conveying belt running surface at the current time instant and its corresponding spectral vector, the displacement of the coal conveying belt surface along the running direction within the time interval of two adjacent snapshots is calculated based on the belt running speed, and the three-dimensional coordinates of all spatial points in the snapshot at the subsequent time instant are inversely translated and compensated based on the displacement, and the compensation formula is:

[0092]

[0093] wherein: represents the coordinate set of the point cloud in the snapshot at the i-th time instant after compensation in the static coordinate system, represents the original coordinate set of the point cloud in the snapshot at the i-th time instant, represents the displacement of the coal conveying belt surface along the running direction within the time interval of two adjacent snapshots, represents the displacement of the coal conveying belt surface along the running direction within the time interval of two adjacent snapshots, represents the displacement of the coal conveying belt surface along the running direction within the time interval of two adjacent snapshots, The total displacement vector of the coal conveying belt along the running direction at the moment, converts all the snapshot spatial point cloud data to the same static coordinate system with the first snapshot as the reference, in the static coordinate system, superimposes the multi-frame spatial point cloud data after time series registration, performs three-dimensional spatial voxelization processing on the superimposed point cloud, and generates a continuous closed triangular mesh model covering the entire detection period and representing the outer surface of the conveying load based on the Poisson surface reconstruction algorithm, as the continuous volumetric three-dimensional model. According to the characteristic spectral curve of different pixels in the spectral data stream, referring to the known coal spectral database, each region on the surface of the continuous volumetric three-dimensional model is marked for coal quality classification, and the region with a spectral characteristic curve difference exceeding the threshold value from the known coal spectral characteristic curve is marked as a region to be identified.

[0094] In a specific implementation, after the continuous volumetric three-dimensional model is generated, the total volume and total mass of the conveying load in the current detection period are calculated, the continuous closed triangular mesh model generated according to the Poisson surface reconstruction algorithm is used to calculate the total volume of the conveying load by calculating the volume of the cylinder surrounded by each triangular facet and the model bottom surface and summing up, and the total mass of the conveying load is estimated by multiplying the total volume by the average density obtained by spectral data inversion. According to the belt running speed and the detection time, the average flow rate is calculated, and the total volume, total mass and average flow rate are cross-verified with the effective power value extracted from the real-time driving motor power spectrum. The consistency of the load mass calculated from the driving motor power and the total mass estimated from the three-dimensional model is compared in the cross-verification process. If the consistency is lower than the preset threshold, the system calibration or alarm is triggered.

[0095] It can be understood that the scanning frequency of each scanning head is matched with the belt running speed, which is specifically manifested as setting the scanning frequency of the line array laser radar so that the line array laser radar can complete at least one complete transverse section scanning within a preset longitudinal resolution distance of the belt running. Optionally, the point cloud data from at least three scanning heads is fused in real time, and a coordinate transformation method based on the relative position of the known scanning head is adopted to unify all the point clouds to a global coordinate system with the center line of the belt conveyor as the reference.

[0096] In some embodiments, the high-density point cloud coordinate sequence is spatially registered with a continuous spectral image cube using a stereo calibration board with hyperspectral reflective features, and three-dimensional corner point coordinates of the calibration board in the point cloud data and pixel coordinates of the calibration board in the hyperspectral image are obtained simultaneously, and a transformation matrix between the coordinate systems is calculated by solving a perspective n-point problem. It can be understood that the normal vector and curvature of each spatial point are calculated, a principal component analysis-based method is used to calculate a covariance matrix and eigenvalue decomposition for each point and points in its neighborhood, and the eigenvector corresponding to the minimum eigenvalue is the normal vector direction, and the curvature is calculated from the distribution relationship of the eigenvalues. Optionally, a snapshot is taken based on a fixed period timestamp, and the fixed period is dynamically adjusted according to the maximum running speed of the belt and the required longitudinal resolution of the three-dimensional model. In a specific implementation, the superimposed point cloud is subjected to three-dimensional voxelization processing, the size of the voxel grid is set, and all point clouds falling into the same voxel are replaced by the centroid thereof, thereby reducing the data amount while retaining the geometric features. The total volume, total mass, and average flow rate are cross-verified with the effective power value extracted from the real-time driving motor power spectrum, and the preset threshold is set to fifteen percent of the relative error of the mass.

[0097] Embodiment 2: see Figure 3The non-coal substance geometry attached to the conveying load surface or mixed in the conveying coal flow is separated, the spatial distribution, boundary shape and geometric connection relationship with the surrounding coal quality area of the to-be-discriminated area in the continuous volume three-dimensional model are analyzed, the spatial area with a closed boundary and a material label of being different is regionally grown and segmented, an independent geometry point cloud cluster is extracted, the bounding box size, centroid position and point cloud density distribution of each geometry point cloud cluster are calculated, and the spatial description parameters of the non-coal substance geometry are formed. The spectrum feature inversion and mechanical response simulation based on the material physical properties are performed on the non-coal substance geometry, the complete spectral reflectance curve is resampled from the original hyperspectral image cube corresponding to each non-coal substance geometry point cloud cluster, the spectral reflectance curve is inverted by using a radiation transfer model, the physical property parameters of the surface roughness and dielectric constant of the non-coal substance geometry are estimated, the Hapke model is selected as the radiation transfer model, the model describes the bidirectional reflectance of a non-homogeneous rough surface under specific illumination and observation geometry, the surface normal distribution in the spatial description parameters of the non-coal substance geometry and the azimuth and zenith angles of the light source and the sensor are input as the known geometric parameters of the Hapke model, the complete spectral reflectance curve is input as the target reflectance data to be fitted by the model, a target function is established with the surface roughness parameter and the dielectric constant parameter as the optimization variables, the target function is defined as the root mean square error of the theoretical reflectance calculated by the model and the target reflectance data in all wave bands, the Levenberg-Marquardt optimization algorithm is used to iteratively solve the target function until the root mean square error is less than a preset accuracy or the maximum iteration number is reached, and the surface roughness parameter value and the dielectric constant parameter value obtained at this time are taken as the estimation results of the physical property parameters; the spatial description parameters of the non-coal substance geometry are input into a discrete element simulation environment, the contact force and motion trajectory of the geometry between the belt, coal and roller under the current coal flow motion state are simulated, and the estimated vibration response signal and friction coefficient of the geometry are output.

[0098] In a specific implementation, a non-coal substance geometry attached to the surface of the conveying load or mixed in the conveying coal flow is separated, the spatial distribution, boundary shape and geometric connection relationship with the surrounding coal quality region of the to-be-discriminated region in the continuous volume three-dimensional model are analyzed, a region growing segmentation is performed on the spatial region with a closed boundary and a material label being different, an independent geometry point cloud cluster is extracted, the bounding box size, centroid position and point cloud density distribution of each geometry point cloud cluster are calculated, and the spatial description parameters of the non-coal substance geometry are formed. The spectral feature inversion and mechanical response simulation based on the material physical properties are performed on the non-coal substance geometry, a complete spectral reflectance curve is resampled from the original hyperspectral image cube corresponding to the point cloud cluster of each non-coal substance geometry, the spectral reflectance curve is inverted by using a radiation transfer model, the physical property parameters of the surface roughness and dielectric constant of the non-coal substance geometry are estimated, the Hapke model is selected as the radiation transfer model, the model describes the bidirectional reflectance of a non-homogeneous rough surface under specific illumination and observation geometry, the surface normal distribution in the spatial description parameters of the non-coal substance geometry and the azimuth and zenith angles of the light source and the sensor are input as the known geometric parameters of the Hapke model, the complete spectral reflectance curve is input as the target reflectance data to be fitted by the model, a target function is established with the surface roughness parameter and the dielectric constant parameter as the optimization variables, the target function is defined as the root mean square error of the theoretical reflectance calculated by the model and the target reflectance data in all wave bands, the Levenberg-Marquardt optimization algorithm is used to iteratively solve the target function until the root mean square error is less than a preset precision or the maximum iteration number is reached, and the surface roughness parameter value and the dielectric constant parameter value obtained at this time are taken as the estimation results of the physical property parameters; the spatial description parameters of the non-coal substance geometry are input into a discrete element simulation environment to simulate the contact force and motion trajectory of the geometry between the belt, the coal and the roller under the current coal flow motion state, and the estimated vibration response signal and the friction coefficient of the geometry are output.

[0099] In some embodiments, the region growing segmentation is performed on the spatial region with a closed boundary and a material label being different, the initial seed point is selected as a pixel point with the greatest difference in spectral features from coal in the to-be-discriminated region, the similarity criterion of the region growing considers the adjacency of the spatial three-dimensional coordinates and the Euclidean distance of the spectral vectors in the feature space, and the segmentation is stopped when there is no longer a point meeting the criterion in the neighborhood of the boundary point of the growing region. The bounding box size of each geometry point cloud cluster is calculated, the principal axis direction of the point cloud cluster is determined by using a principal component analysis method, a smallest cuboid capable of completely enclosing the point cloud cluster is constructed along the principal axis direction, and the length, width and height of the cuboid are the bounding box size.

[0100] In a specific implementation, a complete spectral reflectance curve is resampled from the original hyperspectral image cube. According to the pixel coordinates corresponding to each three-dimensional point in the point cloud cluster of the geometric body on the two-dimensional hyperspectral image, the reflectance value of the pixel at all wavebands is extracted. The reflectance values of all pixels covered by the point cloud cluster of the geometric body are averaged in the spatial dimension to obtain an average spectral reflectance curve representing the entire geometric body. It can be understood that the azimuth and zenith angles of the light source and the sensor are determined through joint calibration when the system is installed and are stored as fixed parameters. A target function is established with the surface roughness parameter and the dielectric constant parameter as optimization variables, which is expressed by the formula:

[0101]

[0102] wherein: is the target function value, is the surface roughness parameter, is the dielectric constant parameter, is the total number of hyperspectral wavebands, is the waveband index, is the theoretical reflectance calculated by the Hapke model based on the current and at the waveband, is the observed reflectance value of the resampled spectral reflectance curve at the waveband. The Levenberg-Marquardt optimization algorithm is used to iteratively solve the target function, and the preset precision is set to and the maximum number of iterations is set to 200.

[0103] It can be understood that the spatial description parameters of the non-coal substance geometric body are input into the discrete element simulation environment, and the spatial description parameters include the bounding box size, the center of mass position, the estimated equivalent density of the point cloud density distribution, and the contact mechanics parameters converted from the surface roughness and dielectric constant parameters obtained by spectral inversion. In some embodiments, the discrete element simulation environment establishes a simulation scene according to the belt running speed, the coal flow accumulation form, and the roller layout, and simplifies the non-coal substance geometric body into a polyhedron or a cluster of spheres model with equivalent mass, volume and contact properties for dynamic simulation. First, based on the spatial description parameters of the non-coal substance geometric body, including the bounding box size, the center of mass position and the point cloud density distribution, the equivalent volume and mass thereof are calculated, wherein the bounding box size is used to determine the circumscribed cube volume of the geometric body, the point cloud density distribution is used to estimate the average density by counting the ratio of the number of point clouds in the grid to the grid volume, so as to derive the equivalent mass in combination with the volume; the contact properties are converted from the surface roughness parameters and the dielectric constant parameters obtained by spectral feature inversion, and are used to set the friction coefficient and the elastic parameter of the model surface. Then, according to the shape complexity of the geometric body, a simplified model form is selected, for a regular geometric body such as a block-shaped foreign matter, a polyhedron model based on a bounding box is adopted, and for an irregular geometric body, a cluster of spheres model is used to approximate the shape thereof.

[0104] Optionally, the estimated vibration response signal of the geometric body and the average dynamic friction coefficient between the non-coal substance geometric body and the belt surface in the simulation process are output, the estimated vibration response signal is given in the form of acceleration power spectral density, and the average dynamic friction coefficient is the friction coefficient. Optionally, the reflectivity values of all pixels covered by the geometric body point cloud cluster are averaged in the spatial dimension, and a weighted average method after removing obvious noise bands in the spectral curve is adopted. The point cloud density distribution of each geometric body point cloud cluster is calculated, the bounding box space is divided into uniform cubic grids, the number of point clouds in each grid is counted, and the ratio of the number of point clouds in the grid to the grid volume is the point cloud density of the region.

[0105] In embodiment 3, adaptive pattern matching is performed with entries in the non-coal substance standard feature library, the estimated physical property parameters are combined with the simulated output pre-estimated vibration response signal and the friction coefficient to form a feature vector to be identified, the feature vector to be identified is input into a pre-trained deep matching network, the deep matching network establishes similarity measurement with feature vectors of various foreign substances in the standard feature library through a contrast learning method, the deep matching network outputs a matching score of the most similar foreign substance category in the standard feature library, and a category with a matching score exceeding an activation threshold is determined as the recognition result. Through adaptive pattern matching, the foreign substance and its confidence evaluation are identified, the deep matching network outputs a similarity score set of the feature vector to be identified and each pre-defined foreign substance category reference vector in the standard feature library, a softmax function is applied to the similarity score set for normalization processing, and the similarity score set is converted into a probability distribution form, wherein the probability value of each category represents its confidence as the recognition result, a confidence threshold is set, an entry with a highest probability value greater than the confidence threshold and a category probability significantly higher than other categories is determined as the final foreign substance recognition result, and the highest probability value is used as the confidence evaluation of the recognition result. After identifying the foreign substance through adaptive pattern matching, the feature vector to be identified, the corresponding recognition result and the confidence evaluation are stored as new samples in a dynamic learning sample library, and the deep matching network is periodically fine-tuned and trained incrementally using the new samples in the dynamic learning sample library to update the network weights. The feature vectors of successfully identified samples with a confidence evaluation continuously higher than a set threshold are added to the non-coal substance standard feature library after being manually reviewed and confirmed, so as to realize self-expansion and update of the feature library.

[0106] In a specific implementation, adaptive pattern matching is performed with entries in the non-coal substance standard feature library, the estimated physical property parameters are combined with the simulated output estimated vibration response signal and the friction coefficient to form a feature vector to be identified, the feature vector to be identified is input into a pre-trained deep matching network, the deep matching network establishes similarity measurement with feature vectors of various foreign matter categories in the standard feature library through a contrast learning method, the deep matching network outputs a matching score of the most similar foreign matter category in the standard feature library, and a category with a matching score exceeding an activation threshold is determined as a recognition result. The foreign matter and its confidence evaluation are identified through adaptive pattern matching. The deep matching network outputs a similarity score set of the feature vector to be identified and each pre-defined foreign matter category reference vector in the standard feature library. A softmax function is applied to the similarity score set for normalization, which is converted into a probability distribution form. The probability value of each category represents its confidence as a recognition result. A confidence threshold is set. An entry with a highest probability value greater than the confidence threshold and a category probability significantly higher than other categories is determined as a final foreign matter recognition result. The highest probability value is used as the confidence evaluation of the recognition result. After the foreign matter is identified through adaptive pattern matching, the feature vector to be identified, the corresponding recognition result and the confidence evaluation are stored as new samples in a dynamic learning sample library. The new samples in the dynamic learning sample library are periodically used to perform incremental fine-tuning training on the deep matching network to update the network weights. The feature vectors of successfully identified samples with a confidence evaluation continuously higher than a set threshold are added to the non-coal substance standard feature library after being confirmed by manual review, so as to realize self-expansion and update of the feature library.

[0107] In some embodiments, the estimated physical property parameters, the simulated output estimated vibration response signal and the friction coefficient are combined to form a feature vector to be identified. The physical property parameters include a surface roughness parameter and a dielectric constant parameter. The estimated vibration response signal is represented by acceleration power spectral density values in multiple frequency bands. The friction coefficient is a single scalar value. These parameters are spliced into a one-dimensional real vector in a fixed order to form the feature vector to be identified. The deep matching network outputs a matching score of the most similar foreign matter category in the standard feature library. The matching score is a cosine similarity value between the feature vector to be identified and an average feature vector of a certain category in the standard feature library. The activation threshold is set according to the accuracy rate statistical results of historical recognition data.

[0108] In a specific implementation, a softmax function is applied to the similarity score set for normalization, and the formula is:

[0109]

[0110] wherein: Pn represents the normalized probability of the nth foreign matter category, Pn represents the normalized probability of the nth foreign matter category, The original similarity score of the first foreign matter category represented by the deep matching network output, is the total number of preset foreign matter categories in the non-coal substance standard feature library, is the temperature parameter used to control the smoothing degree of the probability distribution, is the category index. A confidence threshold is set, and the confidence threshold is set to 0.85. Entries with a highest probability value greater than 0.85 and a difference between the highest probability value and the second highest probability value greater than 0.3 are determined as the final foreign matter identification result.

[0111] It can be understood that the feature vector of the successfully identified sample with the confidence evaluation continuously higher than the set threshold is added to the non-coal substance standard feature library after being confirmed by manual review. The set threshold is higher than the determination threshold, for example, it is set to 0.95. The deep matching network is periodically incrementally fine-tuned using new samples in the dynamic learning sample library, and the training period is set to start once every 100 new samples accumulated.

[0112] In some embodiments, the deep matching network establishes similarity measurement with each foreign matter feature vector in the standard feature library through a contrast learning method. The standard feature library stores multiple feature vector samples composed of historical data for each foreign matter category. When matching, the average similarity between the to-be-identified feature vector and all sample feature vectors of each category is calculated as the matching degree score of the category. Optionally, the dynamic learning sample library is managed using a first-in-first-out queue structure. When the number of new samples reaches the storage upper limit, the oldest sample is automatically deleted. Optionally, the softmax function is applied to the similarity score set for normalization processing, and the value of the temperature parameter is set to 1.0. In specific implementation, the to-be-identified feature vector, the corresponding identification result and the confidence evaluation are stored as new samples. The stored information also includes the timestamp of the appearance of the foreign matter, the spatial position coordinates and the batch identification of the source coal flow.

[0113] Referring to Figure 4 , which is a two-axis line chart for showing the relationship between "feature library sample quantity", "network identification accuracy" and "network training loss" and the training period in the coal belt foreign matter detection system. As the training period continues to grow, the self-expansion mechanism of the feature library is reflected. It shows a steady upward trend, indicating that the expansion of the feature library and the incremental training effectively improve the accuracy of foreign matter identification. It shows a continuous downward trend, corresponding to the increase in identification accuracy, reflecting good model convergence effect. The chart directly reflects the effectiveness of the "dynamic learning + feature library self-expansion" mechanism: through sample accumulation, feature library expansion and incremental training, the identification ability of the model is continuously improved, which can be used to evaluate the iterative optimization effect of the coal belt foreign matter detection system and guide the adjustment of subsequent training period and sample accumulation strategy.

[0114] In embodiment 4, the feature vector to be identified is input into a pre-trained deep matching network, the deep matching network is constructed by comparing the similarity of each feature vector of the foreign matter in the standard feature library through a contrast learning method, the deep matching network includes two identical sub-networks sharing weights, each sub-network is composed of multiple fully connected layers and batch normalization layers, the feature vector to be identified and an entry feature vector in the standard feature library are input into the two sub-networks of the deep matching network respectively, and the corresponding embedding vectors are obtained through forward propagation respectively, the cosine similarity between the embedding vector of the feature vector to be identified and the embedding vector of the entry feature vector is calculated as the similarity measurement score of this matching, in the training stage, positive sample pairs and negative sample pairs are constructed, the positive sample pairs are composed of different sample feature vectors of the same foreign matter category, the negative sample pairs are composed of sample feature vectors of different foreign matter categories, the network is trained using a triplet loss function, and through optimization, the similarity score between the positive sample pairs is higher than the similarity score between the negative sample pairs by a set boundary value, so that the deep matching network learns effective feature representation and similarity measurement criteria.

[0115] Based on the foreign matter identified by adaptive pattern matching and the confidence evaluation, the mechanical arm deployed at a specified position along the coal conveying belt triggers the end effector path planning, analyzes the recognition result output by the adaptive pattern matching, obtains the spatial position coordinates, geometric dimensions, category attributes and matching degree score of the foreign matter as the confidence evaluation, predicts the expected time and position of the foreign matter moving into the working range of the nearest mechanical arm within a future time window according to the spatial position coordinates, geometric dimensions and belt running speed of the foreign matter, selects the appropriate tool type from the end effector library based on the category attributes of the foreign matter, and sets the pose target of the grabbing or cleaning task based on the geometric dimensions and predicted position of the foreign matter, inputs the pose target, the initial pose of the mechanical arm, and the environmental obstacle information between the current coal flow and the mechanical arm base obtained by three-dimensional dynamic reconstruction into the path planning algorithm based on the rapid expansion random tree, runs the path planning algorithm, calculates the collision-free motion trajectory of each joint of the mechanical arm from the initial pose to the pose target, and sends the trajectory to the corresponding mechanical arm controller for execution.

[0116] In practice, the feature vector to be identified is input into a pre-trained deep matching network. The deep matching network establishes a similarity measure with the feature vectors of various foreign objects in the standard feature library through contrastive learning. The deep matching network consists of two identical sub-networks with shared weights. Each sub-network is composed of multiple fully connected layers and batch normalization layers. The feature vector to be identified and an entry feature vector from the standard feature library are respectively input into the two sub-networks of the deep matching network. Each sub-network obtains its corresponding embedding vector through forward propagation. The cosine similarity between the embedding vector of the feature vector to be identified and the embedding vector of the entry feature vector is calculated as the similarity measure score for this match. During the training phase, positive sample pairs and negative sample pairs are constructed. Positive sample pairs consist of different sample feature vectors of the same foreign object category, and negative sample pairs consist of sample feature vectors of different foreign object categories. The network is trained using a triplet loss function. Through optimization, the similarity score between positive sample pairs is made higher than the similarity score between negative sample pairs by a set boundary value, thereby enabling the deep matching network to learn effective feature representations and similarity measurement criteria.

[0117] Based on the foreign objects identified through adaptive pattern matching and their confidence assessment, the robotic arms deployed at designated locations along the coal conveyor belt are triggered to perform end-effector path planning. The identification results output by the adaptive pattern matching are analyzed to obtain the spatial coordinates, geometric dimensions, category attributes, and matching score of the foreign objects as confidence assessment. Based on the spatial coordinates, geometric dimensions, and belt speed of the foreign objects, the expected time and location of the foreign objects moving to the nearest robotic arm's working range within a future time window are predicted. Based on the category attributes of the foreign objects, an appropriate tool type is selected from the end-effector library. Based on the geometric dimensions and predicted location of the foreign objects, the pose target for grasping or cleaning operations is set. The pose target, the initial pose of the robotic arm, and the environmental obstacle information between the current coal flow and the robotic arm base obtained by 3D dynamic reconstruction are input into a path planning algorithm based on a fast expanding random tree. The path planning algorithm is run to calculate the collision-free motion trajectory of each joint of the robotic arm from the initial pose to the pose target, and the trajectory is sent to the corresponding robotic arm controller for execution.

[0118] In some embodiments, each subnetwork of the deep matching network consists of three fully connected layers, each followed by a batch normalization layer and a ReLU activation function. The last fully connected layer outputs a fixed-dimensional embedding vector. The cosine similarity between the embedding vector of the feature vector to be identified and the embedding vector of the entry feature vector is calculated using the following formula:

[0119]

[0120] in: The cosine similarity score is represented by the score. denotes the embedding vector obtained after the forward propagation of the feature vector of an entry in the standard feature library through the subnetwork, denotes the embedding vector obtained after the forward propagation of the feature vector of an entry in the standard feature library through the subnetwork, denotes the vector dot product operation, denotes the L2 norm of a vector. The network is trained using a triplet loss function with a margin set to 0.5.

[0121] In a specific implementation, the predicted time and position of the foreign object moving into the working range of the nearest robot arm within a future time window, the position of the base coordinate system of the robot arm in the global coordinate system, the maximum working radius of the robot arm, and the running direction and speed of the belt are known, the time required for the foreign object to move from the current coordinate along the running direction of the belt to the boundary of the working space of the robot arm is calculated, and the accurate coordinate at the arrival time is calculated combined with the kinematic model. Set the pose target of the grabbing or cleaning task, the pose target includes the target position (X, Y, Z) of the tool center point of the end effector in the global coordinate system and the target attitude (Rx, Ry, Rz), the target position is obtained by offset compensation according to the predicted arrival position and geometric size of the foreign object, and the target attitude is determined according to the surface normal vector of the foreign object and the preset task approach direction. Refer to Table 1.

[0122] Table 1: An example table of setting a pose target

[0123]

[0124] It can be understood that the pose target, the initial pose of the robot arm, and the environmental obstacle information between the current coal flow obtained by three-dimensional dynamic reconstruction and the base of the robot arm are input into the path planning algorithm based on the rapid expansion random tree, the environmental obstacle information is provided in the form of a three-dimensional occupancy grid map, and the rapid expansion random tree algorithm randomly samples and tree expands in the joint space of the robot arm until a path connecting the initial pose and the target pose without collision with obstacles is found.

[0125] In some embodiments, a path planning algorithm is run to calculate a collision-free motion trajectory of each joint of the robot arm from the initial pose to the target pose, the planning algorithm time-parameterizes the found path to generate a smooth trajectory containing the position, velocity, and acceleration of each joint as a function of time, and the trajectory is issued to the robot arm controller in the form of a sequence of points. Optionally, during training of the triplet loss function, each training batch is composed of multiple triplets, each of which includes an anchor sample, a positive sample, and a negative sample. Optionally, an appropriate tool type is selected from the end effector library, the selection rules are pre-configured in a configuration file, and the mapping relationship table between the foreign object category attribute and the tool type is directly determined by querying the mapping relationship table. In a specific implementation, when expanding the tree nodes of the path planning algorithm based on the rapidly expanding random tree, a scoring function is used to guide the search direction, and the scoring function considers the joint space distance from the node to the target pose and the density of obstacles near the node. It can be understood that the collision-free motion trajectory of each joint of the robot arm from the initial pose to the target pose is calculated, the planning process is set with a maximum planning time limit, and if a feasible path is not found within the limit time, a planning failure information is output and a backup alarm process is triggered.

[0126] In embodiment 5, an appropriate tool type is selected from an end effector library, the end effector library pre-stores multiple types of tools including an electromagnetic chuck, a multi-fingered gripper, a hook rake, and a high-pressure air nozzle, and according to the category attribute of the foreign object, a predefined rule is called to select the tool: an electromagnetic chuck is selected for a metal foreign object, a multi-fingered gripper is selected for a block-shaped non-metal foreign object, a hook rake is selected for a woven or strip-shaped foreign object, and a high-pressure air nozzle is selected for a powder-shaped attached foreign object. A historical foreign object event database is established to record the category of each identified foreign object, the time of occurrence, the spatial position, and the corresponding conveying load spectrum features and drive motor power spectrum features, based on the historical foreign object event database, a time series analysis method is used to analyze the periodicity of a specific foreign object category or the correlation with a specific coal spectrum feature, and according to the analysis result, the data acquisition frequency is actively increased or the sensitivity threshold of the adaptive mode matching is adjusted during the predicted high-occurrence period of foreign objects or for specific marked coal flows.

[0127] In a specific implementation, an appropriate tool type is selected from an end effector library, the end effector library pre-stores multiple types of tools including electromagnetic chuck, multi-fingered gripper, hook rake, and high-pressure air nozzle, a predefined rule is invoked for tool selection according to the foreign matter category attribute, the predefined rule is stored in a configuration file in the form of a mapping table, the mapping table explicitly specifies the preferred tool type corresponding to different foreign matter category attributes, the electromagnetic chuck is selected for metal foreign matter, the multi-fingered gripper is selected for block non-metal foreign matter, the hook rake is selected for woven or strip foreign matter, and the high-pressure air nozzle is selected for powder-like attached foreign matter. A historical foreign matter event database is established to record the identified foreign matter category, occurrence time, spatial position, and corresponding conveying load spectrum feature and drive motor power spectrum feature each time, based on the historical foreign matter event database, a time series analysis method is used to analyze the periodicity of a specific foreign matter category or the correlation with a specific coal quality spectrum feature, and according to the analysis result, the data acquisition frequency is actively increased or the sensitivity threshold of adaptive mode matching is adjusted during a predicted foreign matter high-occurrence period or for a specific marked coal flow.

[0128] In some embodiments, a predefined rule is invoked for tool selection, specifically by querying an internal data structure, which associates a foreign matter category attribute string with a tool type identifier, after the category attribute string of the foreign matter is obtained, the system performs an exact matching query in the data structure, returns the corresponding tool type identifier, and the control system selects and loads the specified end effector according to the returned tool type identifier.

[0129] In a specific implementation, a historical foreign matter event database is established, each record in the database contains multiple fields, including foreign matter category, occurrence timestamp, three-dimensional spatial position coordinates, foreign matter geometric size, matching confidence, conveying load average spectrum curve feature vector at the corresponding moment, and drive motor power spectrum energy value in a specific frequency band. A time series analysis method is used to analyze the periodicity of a specific foreign matter category, the historical foreign matter event database is sequenced according to the occurrence timestamp of a certain category of foreign matter, Fourier transform or autocorrelation function is applied to analyze the regularity of the time interval, and it is identified whether there is a periodic pattern in units of hours, days, or weeks. The correlation between foreign matter occurrence and specific coal quality spectrum features is analyzed, the distance or similarity in the feature space between the conveying load average spectrum curve feature vector corresponding to the foreign matter occurrence period and the historical normal coal flow spectrum feature vector is calculated, and a statistical test method is used to determine whether there is a significant association between foreign matter occurrence and the spectrum feature vector.

[0130] It can be understood that the data acquisition frequency of the line array laser radar and the hyperspectral imager is actively increased from the regular 10 Hz to 20 Hz or higher during the predicted high foreign matter period or for the specific marked coal flow. The sensitivity threshold of the adaptive pattern matching is adjusted, that is, the confidence threshold for determining the recognition result is temporarily adjusted from 0.85 to 0.70, so that the system is more sensitive to the detection of potential foreign matters. The formula defines one way of adjusting the sensitivity threshold:

[0131]

[0132] wherein: represents the adjusted sensitivity threshold, represents the basic sensitivity threshold, is an adjustment coefficient set according to the historical false positive rate, represents the deviation measure between the current coal flow spectral characteristics and the average spectral characteristics of the coal flow during the historical high foreign matter period.

[0133] In some embodiments, the historical foreign matter event database is stored in a time series database to support efficient query and analysis operations on time series. Optionally, a high-pressure gas nozzle is selected for powder-like attached foreign matters, which are marked as “wet coal mud attachment” or “dust lump” in the category attribute. Optionally, the data acquisition frequency is actively increased during the predicted high foreign matter period, and the predicted high period is determined according to the periodicity obtained by time series analysis, for example, if the analysis finds that the frequency of “woven fabric” foreign matters significantly increases during a specific shift period every day, the system automatically increases the acquisition frequency during that period every day. In specific implementation, the sensitivity threshold of the adaptive pattern matching is adjusted according to the analysis result, and the adjustment operation is dynamic and reversible. When the high period ends or the specific marked coal flow is delivered, the system automatically restores the sensitivity threshold to the basic value. It can be understood that the corresponding conveying load spectral characteristics and drive motor power spectral characteristics are recorded, the conveying load spectral characteristics are extracted as the first three principal component scores after principal component analysis dimensionality reduction, and the drive motor power spectral characteristics are extracted as the vibration energy proportion of the 0-10 Hz, 10-50 Hz, and 50-100 Hz frequency bands. Based on the historical foreign matter event database, the time series analysis method is used to analyze the periodicity of specific foreign matter categories or the correlation with specific coal spectral characteristics. The analysis task is periodically automatically performed by the background server, and the analysis period is set to once every 24 hours.

[0134] Referring to Figure 5This is a scatter plot of the principal component of spectral characteristics and the probability of foreign matter occurrence, used to analyze the correlation between the spectral characteristics of coal quality in the coal conveying system and the occurrence of foreign matter. When PC1 is on the right side (value >= 4), the color of the data point is light (the probability of foreign matter occurrence >= 0.6), indicating that the spectral characteristics of this region correspond to a higher risk of foreign matter occurrence; when PC1 is on the left side (value <= -4), the color of the data point is dark (the probability of foreign matter occurrence <= 0.3), corresponding to a lower risk of foreign matter occurrence. The influence of PC2: PC2 has weak discrimination on the probability of foreign matter occurrence, and the main correlation dimension is PC1. This chart can be used for foreign matter risk prediction in the coal conveying system: when the principal component of the real-time collected coal quality spectral characteristics falls into the right side of PC1, the system can actively increase the data acquisition frequency or adjust the matching sensitivity, to early warn the risk of foreign matter and improve the detection efficiency.

[0135] While embodiments of the present application have been shown and described, it is to be understood that the embodiments described are merely divergences, modifications, substitutions and variations of the embodiments and can be made by one of ordinary skill in the art without departing from the spirit and principles of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for detecting foreign matter on a coal conveying belt in a coal conveying system, characterized by, The method comprises: establishing a multi-dimensional light field model of the coal conveying belt running surface, the multi-dimensional light field model being constructed based on laser scanning point clouds and spectral imaging information collected from multiple azimuth angles, and being used to represent the geometric topology and material reflection characteristics of the coal conveying belt running surface; acquiring real-time driving motor power spectrum and bearing vibration frequency spectrum associated with the coal conveying belt; according to the geometric topology and material reflection characteristics, and in association with the real-time driving motor power spectrum and bearing vibration frequency spectrum, performing three-dimensional dynamic reconstruction and material labeling on the volume shape of the conveying load; based on the results of the three-dimensional dynamic reconstruction and material labeling, separating out non-coal substance geometries attached to the conveying load surface or mixed in the conveying coal flow; performing spectral feature inversion and mechanical response simulation based on the material physical properties on the non-coal substance geometries, and performing adaptive pattern matching with entries in a non-coal substance standard feature library according to the results of the spectral feature inversion and mechanical response simulation; based on the foreign matter identified through the adaptive pattern matching and the confidence assessment thereof, triggering an end effector path planning performed by a mechanical arm deployed at a specified position along the coal conveying belt; establishing a multi-dimensional light field model of the coal conveying belt running surface, comprising: continuously scanning the cross section of the coal conveying belt by a linear array laser radar to obtain a high-density point cloud coordinate sequence; simultaneously acquiring a continuous spectral image cube within the same field of view range by a hyperspectral imager; aligning the time stamps and registering the spatial coordinate systems of the high-density point cloud coordinate sequence and the continuous spectral image cube; according to the registered data, calculating the normal vector, curvature and reflection intensity at different wave bands of each spatial point to form a composite feature vector field that fuses spatial geometry and spectral reflection characteristics; defining the composite feature vector field as the multi-dimensional light field model; performing three-dimensional dynamic reconstruction and material labeling on the volume shape of the conveying load, comprising: extracting a dynamic point cloud set corresponding to the moving coal flow and a spectral data stream from the composite feature vector field; in combination with the belt running speed, performing time series integration on the dynamic point cloud set to generate a continuous volume three-dimensional model of the conveying load within a detection period; according to the feature spectral curves of different image elements in the spectral data stream, referring to a known coal spectral database, performing coal quality classification labeling on each region of the surface of the continuous volume three-dimensional model; marking the regions that differ from the known coal spectral feature curves by more than a threshold value as regions to be identified; the generation of the continuous volume three-dimensional model of the conveying load within the detection period comprises: based on fixed-period time stamps, intercepting snapshots of multiple continuous time instants from the composite feature vector field, each snapshot containing the spatial point cloud of the coal conveying belt running surface at the current time instant and the corresponding spectral vector; according to the belt running speed, calculating the displacement amount of the coal conveying belt surface along the running direction within the time interval between two adjacent snapshots; according to the displacement amount, performing reverse translation compensation on the three-dimensional coordinates of all spatial points in the snapshots at subsequent time instants, and converting the spatial point cloud data of all snapshots to the same static coordinate system with the first snapshot as the reference; in the static coordinate system, superimposing the multiple frames of spatial point cloud data after time series registration; The superimposed point cloud is subjected to three-dimensional spatial voxelization processing, and based on a Poisson surface reconstruction algorithm, a continuous closed triangular mesh model covering the entire detection period and representing the outer surface of the conveying load is generated as the continuous volumetric three-dimensional model; The non-coal substance geometries attached to the surface of the conveying load or mixed in the conveying coal flow are separated, including: The spatial distribution, boundary shape and geometric connection relationship with the surrounding coal quality area of the to-be-discriminated area in the continuous volumetric three-dimensional model are analyzed; The spatial area with a closed boundary and a material label of being different is subjected to region growing segmentation, and an independent geometric body point cloud cluster is extracted; The bounding box size, centroid position and point cloud density distribution of each geometric body point cloud cluster are calculated to form the spatial description parameters of the non-coal substance geometries; The non-coal substance geometries are subjected to spectral feature inversion and mechanical response simulation based on material physical properties, including: For each non-coal substance geometry point cloud cluster, a complete spectral reflectance curve is resampled from the corresponding original hyperspectral image cube; The spectral reflectance curve is inverted using a radiation transfer model to estimate the physical property parameters of the surface roughness and dielectric constant of the non-coal substance geometry; The spatial description parameters of the non-coal substance geometry are input into a discrete element simulation environment to simulate the contact force and motion trajectory between the geometry and the belt, coal and roller under the current coal flow motion state, and output the estimated vibration response signal and friction coefficient of the geometry.

2. The method of claim 1, wherein the method further comprises: Adaptive pattern matching is performed with the entries in the non-coal substance standard feature library, including: The estimated physical property parameters, the simulated output estimated vibration response signal and the friction coefficient are combined into a to-be-identified feature vector; The to-be-identified feature vector is input into a pre-trained deep matching network, which establishes a similarity measurement with each type of foreign object feature vector in the standard feature library through a contrast learning method; The deep matching network outputs a matching score with the most similar foreign object category in the standard feature library, and determines the category with a matching score exceeding an activation threshold as the recognition result.

3. The method of claim 2, wherein the method further comprises: determining the position of the foreign object on the coal conveying belt based on the detected signal. The foreign object and its confidence evaluation are identified through adaptive pattern matching, including: The deep matching network outputs a similarity score set of the to-be-identified feature vector and each pre-defined foreign object category reference vector in the standard feature library; The similarity score set is normalized by applying a softmax function to convert it into a probability distribution form, where the probability value of each category represents its confidence as a recognition result; A confidence threshold is set, and the entry with a highest probability value greater than the confidence threshold and a category probability significantly higher than other categories is determined as the final foreign object recognition result, and the highest probability value is taken as the confidence evaluation of the recognition result.

4. The method of claim 3, wherein the method further comprises: determining the type of the foreign object based on the detected foreign object. The radiation transfer model is selected as the Hapke model, which describes the bidirectional reflectance of a non-homogeneous rough surface under specific illumination and observation geometry; ​ Input the surface normal distribution in the spatial description parameter of the non-coal substance geometric body, and the azimuth and zenith angle of the light source and the sensor as known geometric parameters of the Hapke model; Input the complete spectral reflectance curve as the target reflectivity data to be fitted by the model; Establish a target function with the surface roughness parameter and the dielectric constant parameter as optimization variables, and the target function is defined as the root mean square error of the theoretical reflectivity calculated by the model and the target reflectivity data at all wave bands; Solve the target function iteratively by using the Levenberg-Marquardt optimization algorithm until the root mean square error is less than the preset accuracy or the maximum number of iterations is reached, and the surface roughness parameter value and the dielectric constant parameter value obtained at this time are taken as the estimation results of the physical property parameters.

5. The method of claim 4, wherein the method further comprises: determining the type of the foreign object based on the detected foreign object. The feature vector to be identified is input into a pre-trained deep matching network, and the deep matching network establishes similarity measurement with each foreign matter feature vector in the standard feature library through a contrast learning method, comprising: A deep matching network is constructed, and the deep matching network includes two identical sub-networks sharing weights, and each sub-network is composed of multiple fully connected layers and batch normalization layers; The feature vector to be identified and an entry feature vector in the standard feature library are input into two sub-networks of the deep matching network respectively, and each obtains a corresponding embedding vector through forward propagation; The cosine similarity between the embedding vector of the feature vector to be identified and the embedding vector of the entry feature vector is calculated as the similarity measurement score of this matching; In the training stage, positive sample pairs and negative sample pairs are constructed, and the positive sample pairs are composed of different sample feature vectors of the same foreign matter category, and the negative sample pairs are composed of sample feature vectors of different foreign matter categories; The network is trained using a triplet loss function, and the similarity score between the positive sample pairs is higher than the similarity score between the negative sample pairs by a set boundary value through optimization, so that the deep matching network learns effective feature representation and similarity measurement criteria.

6. The method of claim 5, wherein the method further comprises: determining the type of the foreign object based on the detected foreign object. Based on the foreign matter identified through adaptive pattern matching and the confidence evaluation, a mechanical arm deployed at a specified position along the coal conveying belt is triggered to perform end effector path planning, comprising: Analyze the recognition result output by the adaptive pattern matching to obtain the spatial position coordinates, geometric dimensions, category attributes and matching degree score of the foreign matter as the confidence evaluation; According to the spatial position coordinates, geometric dimensions and belt running speed of the foreign matter, predict the predicted time and position of the foreign matter moving into the working range of the nearest mechanical arm within a future time window; Based on the category attributes of the foreign matter, select an appropriate tool type from the end effector library, and based on the geometric dimensions and predicted position of the foreign matter, set the pose target of the grabbing or cleaning task; Input the pose target, the initial pose of the mechanical arm, and the environmental obstacle information between the current coal flow obtained by three-dimensional dynamic reconstruction and the base of the mechanical arm into the path planning algorithm based on the rapidly expanding random tree; Running the path planning algorithm, calculating the collision-free motion trajectory of each joint of the robot arm from the initial pose to the target pose, and issuing the trajectory to the corresponding robot arm controller for execution.

7. The method of claim 6, wherein the method further comprises: determining the type of the foreign object based on the detected signal. Also comprising: After identifying the foreign matter through adaptive pattern matching, the to-be-identified feature vector, the corresponding identification result and the confidence evaluation are stored as new samples in a dynamic learning sample library. Periodically, the new samples in the dynamic learning sample library are used to perform incremental fine-tuning training on the deep matching network to update the network weights. The feature vectors of successfully identified samples with a confidence evaluation continuously higher than a set threshold are added to the non-coal substance standard feature library after being confirmed by manual review, realizing self-expansion and updating of the feature library.

8. The method of claim 1, wherein the method further comprises: determining the type of the foreign object based on the detected foreign object. The continuous scanning of the coal conveying belt cross section by the linear array laser radar comprises: At least three linear array laser radar scanning heads are deployed along the coal conveying belt in the transverse direction, directly above the belt and obliquely above both sides, to ensure that the belt running surface and the side of the accumulated coal flow are covered without dead angles; The scanning frequency of each scanning head is controlled to match the belt running speed, so that the point cloud sampling density along the belt running direction remains consistent; The point cloud data from the at least three scanning heads are fused and denoised in real time to generate the high-density point cloud coordinate sequence.

9. The method of claim 7, wherein the method further comprises: determining the type of the foreign object based on the detected signal. The selection of an appropriate tool type from the end effector library comprises: The end effector library pre-stores multiple types of tools, including electromagnetic suction cups, multi-fingered grippers, hooks and high-pressure nozzles; According to the category attributes of the foreign matter, predefined rules are invoked for tool selection: electromagnetic suction cups for metal foreign matters, multi-fingered grippers for block-shaped non-metal foreign matters, hooks for woven or strip-shaped foreign matters, and high-pressure nozzles for powder-like attached foreign matters.

10. The method of claim 1, wherein the method further comprises: determining a location of the foreign object on the coal conveying belt. Also comprising: After the continuous volumetric three-dimensional model is generated, the total volume and total mass of the conveying load within the current detection period are calculated; According to the belt running speed and the detection duration, the average flow rate is calculated; The total volume, total mass and average flow rate are cross-verified with the effective power value extracted from the real-time drive motor power spectrum, and if the consistency is lower than the preset threshold, the system calibration or alarm is triggered.

11. The method of claim 1, wherein the method further comprises: determining a location of the foreign object on the coal conveying belt. Also comprising: A historical foreign matter event database is established to record the category, occurrence time, spatial position, and corresponding conveying load spectral features and drive motor power spectrum features of each identified foreign matter; Based on the historical foreign matter event database, time series analysis methods are used to analyze the periodicity of specific foreign matter categories or the correlation with specific coal quality spectral features; According to the analysis results, the data acquisition frequency is actively increased or the sensitivity threshold of the adaptive pattern matching is adjusted during the predicted high incidence period of foreign matters or for specific marked coal flows.

Images