Deep learning based ore sorting method and system

Abstract

The application discloses a mineral sorting method and system based on deep learning, and belongs to the field of intelligent mineral sorting. In order to solve the problem that the appearance of ore and waste rock combination is similar to that of waste rock, the local material difference is not obvious, and the changes of working conditions such as illumination, dust and silt lead to unstable recognition in the online sorting of a conveying belt, the application synchronously collects RGBD images, multi-line laser radar point clouds and dynamic weighing data in a detection area, establishes cross-modal mapping and instance association, adopts instance segmentation driven by Transform target detection and prompt to obtain a mineral body mask, performs TSDF volume estimation based on the mask constraint depth data, combines the weighing to calculate the density characteristics, compensates and normalizes the distance and incident angle of the point cloud reflection intensity, extracts the local high reflection abnormal distribution and multi-scale intensity statistical characteristics, combines the progressive discrimination mechanism of uncertainty gating to output the category and drive the sorting execution, and realizes the technical effects of online rapid sorting and high-robust and high-accurate recognition of difficult mineral bodies.

Description

Technical Field

[0001] This invention relates to the field of intelligent ore sorting, and more particularly to an ore sorting method and system based on deep learning. Background Technology

[0002] Ore sorting is a crucial step in mineral processing production lines, aiming to separate valuable ores from waste rock and inclusions to reduce the load on subsequent crushing and grinding and improve resource utilization. Existing conveyor belt online sorting technology has evolved from early methods such as manual picking, mechanical screening, and density sorting to intelligent sorting systems based on sensor detection and automated actuators. In recent years, with the widespread adoption of industrial cameras and computing platforms, deep learning-based visual recognition methods have been widely applied. These typically identify ore targets on the conveyor belt through target detection or instance segmentation, and then control the blowing or separating mechanism to complete the sorting. Simultaneously, depth cameras and LiDAR, among other 3D sensors, are being used to provide shape and spatial information, and some systems are also incorporating dynamic weighing to acquire mass data to support more refined sorting decisions.

[0003] However, existing technologies still have shortcomings in online sorting under complex working conditions, mainly including:

[0004] 1. When relying solely on the appearance information of color images, ores, mineral waste combinations and waste rocks are similar in color and texture. Furthermore, due to the influence of changes in lighting, dust obstruction, and surface mud and sand coverage, the recognition results are prone to fluctuation, resulting in misclassification and omission.

[0005] 2. Multi-source data fusion often lacks a stable alignment mechanism for the same ore body instance. Depth data, point cloud data, and weighing data are prone to mismatch with the target, resulting in unreliable features such as volume, mass, and density, which in turn affects sorting decisions.

[0006] 3. The reflection intensity of multi-line lidar is significantly affected by factors such as ranging distance and incident angle. Using the original intensity directly for material identification is prone to deviation. At the same time, it is insufficient to characterize the local high reflection anomaly areas on the ore body surface and their spatial distribution, making it difficult to reliably distinguish difficult targets with indistinct local material differences.

[0007] Therefore, there is a need for an ore sorting method and system that can overcome the shortcomings of the existing technologies. Summary of the Invention

[0008] One objective of this invention is to propose a deep learning-based ore sorting method. Addressing the problems of instability in existing technologies for online conveyor belt sorting due to the similar appearance of ore, ore-waste composites, and waste rock, the difficulty in accurately aligning multi-source data for the same ore body instance, and the difficulty in reliably characterizing material differences due to the influence of distance and incident angle on the reflection intensity of multi-line lidar, this invention proposes a cross-modal progressive discrimination technique based on ore body instances: Color images, depth images, multi-line lidar point clouds, and dynamic weighing data are simultaneously acquired in the detection area, and time alignment and spatial calibration are completed; Transformer-based target detection is used to obtain candidate regions and initial confidence scores, and these candidate regions are used as prompts for instance segmentation and cross-time association to obtain instance identifiers; Volume estimation is performed using instance mask constraints on depth data, and density features are calculated by combining weighing data; Simultaneously, corresponding instance point clouds are extracted, and reflection intensity is compensated and normalized to generate local high-reflection anomaly area distribution features and multi-scale intensity statistical features; Based on confidence scores, a gated fusion discrimination is triggered to output the final category and drive the sorting process. This invention has the technical advantages of fast online processing speed, strong consistency of instance-level fusion, robustness to changes in working conditions, and high accuracy in identifying difficult-to-distinguish ore bodies.

[0009] This invention provides a deep learning-based ore sorting method, comprising:

[0010] S1. Acquire color images, depth images, multi-line LiDAR point cloud data, and dynamic weighing mass data in the detection area, perform time alignment to obtain a synchronized dataset. The multi-line LiDAR point cloud data includes the spatial coordinates and original reflection intensity of the points. S2. Establish a cross-modal mapping relationship between the pixel coordinates of the color images and the spatial coordinates of the multi-line LiDAR point cloud data. S3. Perform target detection on the color images in the synchronized dataset, output candidate regions of the ore body and initial category confidence scores, perform instance segmentation of the ore body based on the candidate regions, output instance masks, and perform instance association between the target detection results and instance segmentation results at consecutive time steps to obtain ore body instance identifiers. S4. Extract ore body instance depth data from the depth images based on the instance masks and estimate the ore body instance volume. Determine the ore body from the dynamic weighing mass data based on the ore body instance identifiers. S5. Calculate the orebody instance density characteristics and extract orebody instance point cloud data from multi-line lidar point cloud data based on instance masks and cross-modal mapping relationships. S6. Perform physical normalization processing on the original reflection intensity in the orebody instance point cloud data based on ranging distance and incident angle to obtain normalized reflection intensity, generating reflection intensity features, including local high-reflection anomaly area distribution features and multi-scale reflection intensity distribution statistical features. S7. Output orebody instance categories based on a progressive discrimination mechanism and drive the sorting mechanism to perform sorting: when the initial category confidence meets the preset sorting conditions, output the orebody instance category corresponding to the target detection and drive the sorting mechanism to perform sorting; otherwise, input the orebody instance density features and reflection intensity features into the fusion discrimination model for classification and discrimination, output the final orebody instance category, and drive the sorting mechanism to perform sorting.

[0011] Optionally, S1 includes:

[0012] The color image, the depth image, the multi-line lidar point cloud data, and the dynamic weighing mass data are collected respectively within the detection area of ​​the conveyor belt, and the color image, the depth image, the multi-line lidar point cloud data, and the dynamic weighing mass data are time-aligned using a unified trigger signal or timestamp.

[0013] The color image and the depth image under the same time reference after time alignment are paired to form image data. The multi-line lidar point cloud data and the dynamic weighing mass data within the time period corresponding to the image data are associated with the image data on the same time axis to obtain the synchronous dataset.

[0014] Each point in the multi-line lidar point cloud data includes the spatial coordinates and the original reflection intensity.

[0015] Optionally, S2 includes:

[0016] The trigger signal or timestamp of the acquisition device is calibrated to complete time synchronization;

[0017] The imaging parameters of the acquired color image are obtained, and the pixel coordinate system of the color image and the spatial coordinate system of the multi-line lidar point cloud data are spatially calibrated to obtain the coordinate transformation parameters from the spatial coordinate system to the pixel coordinate system.

[0018] Based on the coordinate transformation parameters, the spatial coordinates in the multi-line lidar point cloud data are transformed to the pixel coordinates of the color image, and a coordinate mapping relationship is established between the pixel coordinates of the color image and the spatial coordinates of the multi-line lidar point cloud data, thus obtaining the cross-modal mapping relationship.

[0019] Optionally, S3 includes:

[0020] Target detection is performed on the color images in the synchronous dataset. The Transformer-based target detection network outputs multiple detection candidates and performs overlap suppression on the detection candidates to determine the candidate region corresponding to each ore body instance. At the same time, the initial class confidence corresponding to the candidate region is determined to obtain the target detection result.

[0021] Based on the target detection results, the prompt information is generated for each candidate region, and the prompt information includes at least the regional location parameters of the candidate region;

[0022] The instance segmentation network is driven by the prompt information to perform pixel-level segmentation in the candidate region and output the instance mask that corresponds one-to-one with the pixel coordinates of the color image to obtain the instance segmentation result; when the target detection result and the instance segmentation result at consecutive time steps are associated to obtain the ore body instance identifier, the process includes: predicting the motion of the ore body instance in the candidate region position at adjacent time steps based on the conveyor belt speed.

[0023] Construct a cross-time matching cost, which includes the intersection-union ratio of candidate regions and the distance between the center points of candidate regions;

[0024] Based on the matching cost, a one-to-one correspondence across time periods is determined, and the correspondence is updated to reflect the trajectory, outputting the ore body instance identifier.

[0025] Optionally, S4 includes:

[0026] Based on the instance mask in the instance segmentation result, the depth value corresponding to the pixel covered by the instance mask is selected in the depth image, and outlier values ​​are removed from the selected depth value to obtain the ore body instance depth data corresponding to the instance mask.

[0027] Based on the depth data of the ore body instance, a truncated signed distance field is constructed within a preset voxel grid. The volume of the ore body instance is obtained by volume integration of voxel cells that satisfy preset integration conditions within the voxel grid. A weighing time window for the ore body instance is determined based on its identifier. This weighing time window is the time interval during which the ore body instance passes the weighing sensor. Determining the weighing time window includes: acquiring the distance along the conveyor belt from the detection area to the weighing sensor and the conveyor belt speed; calculating the time offset of the ore body instance from the detection area to the weighing sensor based on the distance and the conveyor belt speed; mapping the detection time corresponding to the ore body instance identifier to the weighing time based on the time offset; and determining the weighing time window. Within the weighing time window, the dynamic weighing mass data is truncated, and baseline correction is performed on the truncated mass data to obtain the ore body instance mass.

[0028] The density characteristic of the ore body instance is calculated based on the mass and volume of the ore body instance, whereby the density characteristic of the ore body instance is the ratio of the mass to the volume of the ore body instance.

[0029] Based on the cross-modal mapping relationship, the spatial coordinates in the multi-line lidar point cloud data are mapped to the pixel coordinates of the color image, and points whose mapped pixel coordinates fall within the scope of the instance mask are selected to form ore body instance point cloud data corresponding to the instance mask.

[0030] Furthermore, after extracting the point cloud data of the ore body instance, cross-modal consistency verification is also included: the point cloud data of the ore body instance is mapped to the pixel coordinates of the color image through coordinate transformation, and the proportion of the mapped points falling within the range of the instance mask is calculated. When the proportion is less than a preset consistency threshold, the instance association is re-executed or the point cloud data of the current ore body instance is removed to avoid erroneous association.

[0031] Optionally, S5 includes:

[0032] The distance to be measured at each point is calculated based on the spatial coordinates of each point in the point cloud data of the ore body instance.

[0033] The neighboring points of the point are obtained and plane fitting is performed to obtain the surface normal vector of the ore body corresponding to the point. The incident angle corresponding to the point is determined by the angle between the laser emission direction and the surface normal vector of the ore body.

[0034] The reflection intensity compensation coefficient is calculated based on the ranging distance and the incident angle, and the original reflection intensity is compensated to obtain the compensated reflection intensity;

[0035] The compensated reflection intensity is normalized to obtain the normalized reflection intensity, forming a normalized point cloud containing the spatial coordinates and the normalized reflection intensity;

[0036] The normalized point cloud is subjected to point cloud feature encoding, which is implemented by a Transformer-based point cloud feature encoding network. During the encoding process, local high-reflectance anomaly regions on the ore body surface are determined based on the normalized reflection intensity, and anomaly marker vectors are generated. The anomaly marker vectors are used to characterize the distribution characteristics of the local high-reflectance anomaly regions. The determination of the local high-reflectance anomaly regions includes: determining an anomaly judgment threshold based on the normalized reflection intensity distribution of the ore body instance point cloud. The anomaly judgment threshold includes at least one of the upper quantile threshold and the mean plus standard deviation threshold.

[0037] When the normalized reflection intensity of a point exceeds the anomaly determination threshold, the local neighborhood to which the point belongs is marked as a high-reflection anomaly region.

[0038] Simultaneously, a multi-scale intensity statistical head is used to statistically aggregate the normalized reflection intensity distribution at different spatial scales to form multi-scale intensity statistical features. The multi-scale intensity statistical head performs statistical aggregation of normalized reflection intensity based on neighborhoods of at least two different spatial scales. The spatial scales include at least one of spherical neighborhoods of different radii and voxel grids of different sizes, and outputs the reflection intensity features. The reflection intensity features include the distribution features of the local high reflection anomaly region and the multi-scale intensity statistical features.

[0039] Furthermore, the distribution characteristics of the local high reflectance anomaly region include regional statistical features. The generation of the regional statistical features includes: spatially clustering points that exceed the anomaly determination threshold to form an anomaly region set, and calculating at least one of the following: the number of anomaly regions, the area ratio of anomaly regions, the maximum normalized reflectance intensity of anomaly regions, and the distribution bias of anomaly regions along the main axis of the ore body.

[0040] Optionally, S6 includes:

[0041] The first candidate class confidence score is the maximum class confidence score output by the target detection network, the second candidate class confidence score is the second largest class confidence score output by the target detection network, and the confidence score difference is the difference between the first candidate class confidence score and the second candidate class confidence score.

[0042] When the confidence level of the first candidate category is less than the preset sorting threshold or the confidence level difference is less than the preset difference threshold, the ore body instance density feature and the reflection intensity feature are concatenated to obtain a fused discriminant feature, and the fused discriminant feature is input into a multilayer perceptron classifier or a Transformer-based classifier to output the final ore body instance category.

[0043] On the other hand, the present invention also provides a deep learning-based ore sorting system, comprising:

[0044] The data acquisition module is used to acquire color images, depth images, multi-line LiDAR point cloud data, and dynamic weighing mass data in the detection area and perform time alignment to obtain a synchronized dataset. The multi-line LiDAR point cloud data includes the spatial coordinates of points and the original reflection intensity. The cross-modal mapping module is used to establish a cross-modal mapping relationship between the pixel coordinates of the color images and the spatial coordinates of the multi-line LiDAR point cloud data. The target detection and segmentation module is used to perform target detection on the color images, output candidate regions and initial category confidence scores, perform instance segmentation based on the candidate regions, output instance masks, and perform instance association to obtain ore body instance identifiers. The density and point cloud module is used to extract ore body instance depth data based on the instance mask and estimate the ore body instance volume, and based on the ore body instance... The system identifies and determines the quality of orebody instances and calculates their density features. It also extracts point cloud data of orebody instances based on instance masks and cross-modal mapping relationships. A reflection intensity feature module performs physical normalization on the original reflection intensity based on the ranging distance and incident angle to obtain normalized reflection intensity and generates reflection intensity features, including local high-reflection anomaly area distribution features and multi-scale reflection intensity distribution statistical features. A discrimination and sorting module outputs orebody instance categories based on a progressive discrimination mechanism and drives a sorting mechanism to perform sorting. When the initial category confidence meets preset sorting conditions, the module outputs the orebody instance category corresponding to the target detection; otherwise, it inputs the orebody instance density features and reflection intensity features into a fusion discrimination model to output the final orebody instance category.

[0045] The beneficial effects of this invention are:

[0046] 1. This invention performs time alignment on color images, depth images, multi-line lidar point cloud data, and dynamic weighing mass data, and establishes a cross-modal mapping relationship between pixel coordinates and point cloud spatial coordinates. Combined with instance association at consecutive time points, it obtains ore body instance identifiers, so that volume estimation, mass acquisition, and point cloud intensity feature extraction are all performed on the same ore body instance, reducing misjudgments caused by multi-source data mismatch and improving the reliability and consistency of sorting decisions.

[0047] 2. This invention utilizes instance mask to constrain depth data and employs truncated signed distance fields for volume integration, combined with dynamic weighing mass data to calculate density characteristics. In scenarios where appearances are similar and texture differences are not obvious, it provides stable physical quantity support for ore classification, thereby improving the ability to distinguish between ore, mineral waste combination and waste rock and the ability to resist working condition disturbances.

[0048] 3. This invention performs compensation and normalization processing on the original reflection intensity of multi-line lidar based on ranging distance and incident angle, and further extracts the distribution characteristics of local high reflection anomaly areas and the statistical characteristics of multi-scale reflection intensity. At the same time, it adopts a confidence-gated progressive discrimination mechanism to ensure rapid online initial judgment while triggering subsequent fusion discrimination only on uncertain samples, thereby improving the recognition accuracy of difficult targets and the overall real-time performance of the system. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0050] Fig. 1 This is a flowchart of the deep learning-based ore sorting method proposed in this invention;

[0051] Fig. 2 This is a flowchart of the progressive discrimination mechanism in step S6 of the present invention. Detailed Implementation

[0052] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0053] refer to Figs. 1-2 Deep learning-based ore sorting methods include:

[0054] S1. Acquire color images, depth images, multi-line LiDAR point cloud data, and dynamic weighing mass data in the detection area, perform time alignment to obtain a synchronized dataset. The multi-line LiDAR point cloud data includes the spatial coordinates and original reflection intensity of the points. S2. Establish a cross-modal mapping relationship between the pixel coordinates of the color images and the spatial coordinates of the multi-line LiDAR point cloud data. S3. Perform target detection on the color images in the synchronized dataset, output candidate regions of the ore body and initial category confidence scores, perform instance segmentation of the ore body based on the candidate regions, output instance masks, and perform instance association between the target detection results and instance segmentation results at consecutive time steps to obtain ore body instance identifiers. S4. Extract ore body instance depth data from the depth images based on the instance masks and estimate the ore body instance volume. Determine the ore body from the dynamic weighing mass data based on the ore body instance identifiers. S5. Calculate the orebody instance density characteristics and extract orebody instance point cloud data from multi-line lidar point cloud data based on instance masks and cross-modal mapping relationships. S6. Perform physical normalization processing on the original reflection intensity in the orebody instance point cloud data based on ranging distance and incident angle to obtain normalized reflection intensity, generating reflection intensity features, including local high-reflection anomaly area distribution features and multi-scale reflection intensity distribution statistical features. S7. Output orebody instance categories based on a progressive discrimination mechanism and drive the sorting mechanism to perform sorting: when the initial category confidence meets the preset sorting conditions, output the orebody instance category corresponding to the target detection and drive the sorting mechanism to perform sorting; otherwise, input the orebody instance density features and reflection intensity features into the fusion discrimination model for classification and discrimination, output the final orebody instance category, and drive the sorting mechanism to perform sorting.

[0055] In this specific embodiment, S1 includes:

[0056] A color camera, a depth camera, and a multi-line LiDAR are sequentially installed above the detection area of ​​the conveyor belt. A dynamic weighing sensor and signal conditioning circuit are installed in the weighing section below the conveyor belt. The controller uses a master clock supporting IEEE 1588PTP to provide unified time synchronization for the color camera, depth camera, LiDAR, and weighing acquisition card, and outputs a hardware trigger pulse with a fixed frequency of 60Hz as a unified trigger signal. This allows the color camera and depth camera to synchronously expose under the same trigger pulse and each generate a frame of data. The PTP timestamp of each frame is recorded and defined as the reference time of that frame. ,in A trigger sequence number used to uniquely identify a group of synchronized data under the same time base;

[0057] The color camera output resolution is The color image, the depth camera output resolution is The depth image is generated and the distance from the camera's optical center to the measured surface is represented by a depth value. The controller uses the same trigger sequence number. Same reference time The corresponding color image and depth image are directly paired to form image data and written to the synchronous data cache;

[0058] The multi-line lidar continuously outputs point cloud data packets using a rotating scanning method. The controller writes a PTP timestamp to each point cloud point and ensures that each point contains spatial coordinates and the original reflection intensity. The spatial coordinates are three-dimensional coordinates in the radar coordinate system. Indicates and Represents the coordinate components along the direction of the conveyor belt's movement. Represents the horizontal coordinate components, The vertical coordinate components are represented by , and the original reflection intensity is denoted as . It also represents the uncompensated intensity value of the lidar echo after quantization within the device;

[0059] The load signal generated by the dynamic weighing sensor at the weighing section is acquired by the data acquisition card at a sampling frequency of 2000Hz and written to the PTP timestamp to form a dynamic weighing mass data stream. The single-point mass value in the dynamic weighing mass data is denoted as... It also indicates the instantaneous mass reading of the weighing segment under that timestamp;

[0060] The controller uses the reference time of each set of image data Construct a time window corresponding to this set of image data, centered on the image data:

[0061] ;

[0062] in This indicates the time interval used to correlate LiDAR point cloud data and dynamic weighing mass data on the same time axis. This indicates the period between two consecutive trigger pulses, set to 16.67ms. The controller will then set the timestamp to this value. Extracting the point cloud set of the LiDAR within and matching it with the trigger sequence number The data is bound to form a multi-line LiDAR point cloud data segment corresponding to this set of image data, and the timestamp is added to it. Extracting the dynamic weighing mass data sample sequence and matching it with the trigger sequence number The dynamic weighing mass data segment corresponding to this set of image data is bound together, ultimately based on the trigger sequence number. The color image, depth image, point cloud data segment, and quality data segment are encapsulated into a synchronous dataset for indexing and output to subsequent steps for instance-level processing and sorting decisions.

[0063] In this specific embodiment, S2 includes:

[0064] The controller performs time synchronization calibration on the unified timing link. The calibration method involves reading the PTP slave clock offsets from the color camera, depth camera, multi-line LiDAR, and weighing acquisition card, and performing offset compensation until the residual offsets of all four relative to the master clock are less than [value missing]. This ensures that the timestamps of each sensor can be directly used for alignment during subsequent spatial calibration data acquisition.

[0065] Subsequently, the imaging parameters of the color camera were acquired and a pixel coordinate system was established. The color camera employed a pinhole imaging model and a Brown-Conrady distortion model. The pinhole model consisted of an intrinsic parameter matrix. Characterized and:

[0066] ;

[0067] in and Representing the focal length pixel values ​​in the horizontal and vertical directions respectively, and taking... , and Let x and y represent the x and y coordinates of the principal point in the pixel coordinate system, respectively, and take... The distortion parameters include the radial distortion coefficient. , With tangential distortion coefficient The controller performs distortion correction on each frame of the color image and uses the distorted pixel coordinates as the pixel coordinates for subsequent mapping. ,in Represents the horizontal pixel coordinates and their values ​​range from 1 to 2. Represents the vertical pixel coordinate and its value range is ;

[0068] During the spatial calibration phase, a rigid planar calibration plate is placed within the detection area, and four reflective corner point targets are fixed on the plate to ensure that the targets can be stably extracted from both the color image and the point cloud. The controller collects 200 sets of synchronous data while the conveyor belt is stopped, and extracts the pixel coordinates of the four corner points from the color image in each set of data. and This indicates the corner point number, and the spatial coordinates of the four corner points in the radar coordinate system are extracted from the multi-line lidar point cloud data. And outliers are removed by plane consistency constraints, where With steps The coordinate components along the conveyor belt direction, the horizontal direction, and the vertical direction are respectively represented.

[0069] The controller aims to minimize the reprojection error at the four corner points and employs Levenberg-Marquardt nonlinear least squares to solve for the coordinate transformation parameters from the multi-line lidar coordinate system to the color camera coordinate system. ,in for Homogeneous transformation matrix and rotation matrix With translation vector It constitutes and satisfies:

[0070] ;

[0071] in for An orthogonal matrix that represents the coordinate system attitude transformation. for Vector and represents the translation of the origin of the coordinate system and takes for Zero vector;

[0072] In obtaining and After establishing a cross-modal mapping relationship, the controller constructs homogeneous coordinates in the radar coordinate system for any cloud point in each frame. ,in and The homogeneous coordinates in the camera coordinate system are calculated by representing the three-dimensional spatial coordinate components of the point in the radar coordinate system. And denote its three-dimensional non-homogeneous coordinates as ,in Let each represent the three-dimensional coordinate components of the point in the camera coordinate system, and when The point is determined to be behind the camera's imaging plane and discarded directly. Then, it is projected onto the pixel coordinate system according to the pinhole model and the pixel coordinates are calculated using the following formula:

[0073] ;

[0074] in Describe the projection scale factor and satisfy and These represent the horizontal and vertical pixel coordinates after distortion correction. For the intrinsic parameter matrix of the color camera, These are the coordinate components of the point in the camera coordinate system;

[0075] The controller sets the coordinates of each valid point cloud point to its projected pixel coordinates. Write it to the mapping table in key-value pairs and match it with the original reflection intensity of that point. Shared storage, in which and The uncompensated reflection intensity value of the point is consistently represented, and a one-to-one mapping relationship between the pixel coordinates of the color image and the spatial coordinates of the multi-line lidar point cloud data is obtained, which is then used as the cross-modal mapping relationship for subsequent instance-based mask filtering of point cloud points.

[0076] In this specific embodiment, S3 includes:

[0077] The controller performs ore body target detection on each frame of the color image to output candidate regions and initial class confidence scores. The target detection network adopts a Transformer-based DETR structure and uses the distortion-corrected color image as input. The input image is scaled to... And apply the average value to each channel. With variance For normalization, the backbone network uses ResNet-50 and outputs feature maps with a stride of 32. The Transformer encoder has 6 layers, the decoder has 6 layers, the feature dimension is 256, the multi-head attention head has 8 layers, and the number of target queries is 100. The classification head uses Softmax to output a set of ore body categories. Ore, mineral waste combination, waste rock Based on the probability distribution of the background class, the regression head outputs the candidate region location parameters for each detection candidate and uses them in conjunction with the probability distribution of the background class. This represents the pixel coordinates of its top-left and bottom-right corners, where All are within the pixel coordinate system of the color image and are pixel coordinates Maintain consistency;

[0078] The controller sorts the detection candidates from high to low according to the maximum class probability and performs overlap suppression to determine the candidate region corresponding to each ore body instance. The overlap suppression adopts an IoU threshold of 0.50 and only retains the candidate regions with a maximum class probability of not less than 0.30. At the same time, the maximum class probability corresponding to each candidate region is defined as the initial class confidence of the candidate region and bound to the candidate region to form the target detection result.

[0079] The controller generates prompt information for each candidate region based on the target detection results. The prompt information is composed of the candidate region location parameters. The spatial range of instance segmentation is constrained by a cue-driven encoder-decoder structure, consisting of an image encoder, a cue encoder, and a mask decoder. The image encoder uses ViT-B and contains 12 Transformer layers with an embedding dimension of 768 and a patch size of 16. The cue encoder encodes the candidate region location parameters into cue vectors of the same dimension as the image encoder and fuses them with image features. The mask decoder contains 2 Transformer layers and outputs instance masks that correspond one-to-one with the pixel coordinates of the input color image. After obtaining a pixel-level foreground probability map by Sigmoid, the instance mask is binarized with a threshold of 0.50 to obtain the pixel-level mask of the ore body instance, thus obtaining the instance segmentation result.

[0080] The controller performs instance association between the target detection results and instance segmentation results at consecutive time intervals to generate ore body instance identifiers. The instance association is based on the conveyor belt speed. The time interval between two adjacent frames Based on motion prediction, where Real-time output from conveyor belt speed encoder and and The triggering period is consistent with 16.67ms. The motion prediction shifts the center point of the candidate region of the existing trajectory in the previous moment along the direction of the conveyor belt to obtain the predicted center point of the trajectory at the current moment.

[0081] Subsequently, the cross-time-sequence matching cost is constructed, and the matching cost is calculated for each trajectory and each candidate region at the current time. Calculate using the following formula:

[0082] ;

[0083] in This represents the trajectory index at the previous time step and corresponds to an existing ore body instance identifier. This represents the candidate region index at the current time. Representing the trajectory Predicted candidate regions and candidate regions The intersection and union ratio, Indicates candidate region The center point pixel coordinate vector and And it is calculated from the location parameters of the candidate region. Representing the trajectory The predicted center point pixel coordinate vector at the current moment and The 1-norm is used to calculate the Euclidean distance between the center points. Indicates the cost weights of intersection and union ratios and takes The weight represents the distance cost from the center point and is taken as follows: This represents the distance normalization constant, set to 200 pixels.

[0084] The controller uses the Hungarian algorithm to solve for the one-to-one correspondence in the cost matrix and sets a maximum allowable cost threshold of 0.80. When the minimum cost does not exceed this threshold, the candidate region is selected. With trajectory Associate and inherit trajectories The orebody instance is identified, and candidate regions are used simultaneously. Its corresponding instance mask update trajectory state, when the candidate region When no trajectory matches, a new orebody instance identifier is assigned and a new trajectory is created. When a trajectory fails to match a candidate region for three consecutive frames, the trajectory is terminated and its state is released, thus ensuring consistency at each reference time. Output the target detection results and instance mask with ore body instance identifiers for subsequent instance-level feature extraction and sorting decisions.

[0085] In this specific embodiment, S4 includes:

[0086] The controller uses the same reference time The corresponding instance mask performs instance-level constraints on the depth image to extract ore body instance depth data. The depth image is a registered depth map that corresponds one-to-one with the pixel coordinates of the color image, and each pixel... The depth value represents the distance from the optical center of the RGBD camera to the surface of the corresponding mineral body. The controller traverses the foreground pixels in the instance mask and reads the corresponding depth values ​​to form a depth sample set and performs outlier removal. The outlier removal process first performs... Median filtering is used to suppress isolated noise. The median and median absolute deviation of the depth samples are then recalculated, and depth values ​​that deviate from the median by more than three times the median absolute deviation are removed to obtain the orebody instance depth data.

[0087] The controller backprojects the depth data of the ore body instance into a three-dimensional point set in the camera coordinate system and establishes a preset voxel grid with the bounding box aligned with the axis of the three-dimensional point set as the boundary. The voxel side length is 2mm and the bounding box is extended by 20mm in each of the three axes to cover the uncertain boundary area. Then, a truncated signed distance field (TSDF) is constructed in the voxel grid with a truncated distance of 10mm. The TSDF is updated by pixel-by-pixel ray integration. For each ray pointing from the camera optical center to the three-dimensional point, the signed distance of the voxel is updated within the truncated distance range and the weight is accumulated. Voxels with a weight of 3 and a TSDF value of less than zero are determined to be located inside the ore body and participate in the volume integration. The volume integration is obtained by summing the voxel volumes of all participating voxels to obtain the ore body instance volume.

[0088] The controller determines the weighing time window for each ore body instance based on its identifier. The weighing time window is calculated by the distance along the conveyor belt from the detection area to the weighing sensor and the conveyor belt speed. The arrival time offset is determined and the detection time corresponding to the ore body instance is mapped to the weighing time. At the same time, the physical length of the instance along the conveyor belt direction is obtained by combining the pixel length of the instance in the candidate region of the adjacent frame along the conveyor belt direction with the depth back projection, and then divided by the conveyor belt speed to obtain the duration of passing through the weighing sensor, thereby determining the start and end times of the weighing time window.

[0089] Within the weighing time window, the controller extracts a sequence of mass samples from the dynamic weighing mass data and performs baseline correction. The baseline for baseline correction is the median of the mass samples within 0.20 seconds before the start of the weighing time window, and this baseline is subtracted from each mass sample within the weighing time window to eliminate conveyor belt self-weight drift and zero-point drift. The mass of the ore body instance is the maximum value of the mass sample sequence after baseline correction to characterize the effective mass of the ore body instance when it passes through the weighing section.

[0090] The controller calculates the density feature of the orebody instance based on its quality and volume, and writes it to the feature cache of that orebody instance identifier. The density feature is then processed according to... Calculation, where Represents the density characteristics of orebody instances, with units of Indicates the mass of an ore body instance, with units of Represents the volume of an ore body instance, with units of 1. ;

[0091] The controller further utilizes cross-modal mapping relationships from the same reference time. Extracting orebody instance point cloud data from associated multi-line lidar point cloud data segments: The extraction process involves calculating the projected pixel coordinates of each point cloud point according to the mapping relationship and determining whether the projected pixel coordinates fall within the instance mask foreground region of the orebody instance. Point cloud points that fall within the foreground region are collected as orebody instance point cloud data, and their spatial coordinates and original reflection intensity are preserved. ;

[0092] After the point cloud data of the ore body instance is extracted, a cross-modal consistency check is performed. The consistency check reprojects the point cloud data of the ore body instance onto the pixel coordinates of the color image and calculates the proportion of the projection point falling into the foreground area of ​​the instance mask. When the proportion is less than 0.65, the controller triggers the re-execution of instance association for the ore body instance identifier. When re-executing instance association, the matching cost of the ore body instance trajectory and the candidate area is reconstructed within the last 5 frames, and the one-to-one correspondence is updated using the Hungarian algorithm to correct the ore body instance identifier and the instance mask. When the proportion is still less than 0.65 after re-executing instance association, the point cloud data of the current ore body instance is removed, and the ore body instance identifier is marked as invalid point cloud to avoid erroneous association affecting subsequent reflection intensity feature extraction and sorting decisions.

[0093] In this specific embodiment, S5 includes:

[0094] The controller calculates the distance to each point in the point cloud data of each ore body instance. Define it as the spatial coordinates of that point. The Euclidean distance relative to the origin of the multi-line lidar coordinate system, where These represent the coordinate components along the direction of the conveyor belt, in the horizontal and vertical directions, respectively, with units in meters (m).

[0095] The controller is built around each point within the point cloud data of the ore body instance. Nearest neighbor and neighbor domain take The least-squares plane is fitted to the neighborhood point set, and the unit normal vector of the fitted plane is used as the surface normal vector of the ore body corresponding to that point. Simultaneously, the unit vector pointing from that point to the origin of the multi-line lidar coordinate system is taken as the laser emission direction. and with and The included angle is defined as the angle of incidence. And the range of values ​​is ;

[0096] In obtaining and After the original reflection intensity Perform physical compensation based on the ranging distance and incident angle, and then normalize to obtain the normalized reflection intensity. The Calculate using the following formula:

[0097] ;

[0098] in Represents the normalized reflection intensity and its value range is With steps Consistently represents the uncompensated original reflection intensity quantization value at that point. Indicates the distance measured. Indicates the angle of incidence. The cosine value of the incident angle is used to characterize the effect of the incident angle on the echo intensity. This indicates that the 1st percentile of all compensated intensity values ​​in the point cloud data of this orebody instance is used to suppress low-end outliers. This indicates that the 99th percentile of all compensated intensity values ​​in the point cloud data of this ore body instance is used to suppress high-end outliers. Indicates the numerical value Cut off to interval The cutoff function within;

[0099] The controller will store the spatial coordinates of each point. Its normalized reflection intensity A normalized point cloud is formed by combining the features, and a reflection intensity feature is generated on the normalized point cloud. The reflection intensity feature is composed of the distribution features of local high reflection anomaly areas and the statistical features of multi-scale reflection intensity distribution.

[0100] The determination of local high-reflectance anomaly regions adopts a distribution-based threshold method, and the controller analyzes the point cloud data of this ore body instance. Calculate the mean and standard deviation, and then calculate the upper quantile threshold. The final anomaly detection threshold is the larger of the "upper quantile threshold" and the "mean plus twice the standard deviation threshold." The controller will then meet these requirements. Points exceeding the anomaly detection threshold are marked as anomalies, and the binary labels of the anomalies are arranged in order to form an anomaly label vector. The anomaly label vector is used to characterize the discrete distribution of anomalies on the surface of the ore body.

[0101] The controller performs spatial clustering on outliers to form a set of outlier regions. The spatial clustering uses the DBSCAN algorithm with parameters set to... The clustering results are used to calculate regional statistical features as the distribution features of local high reflectance anomaly areas. The regional statistical features include the number of anomaly areas, the proportion of anomaly points, the maximum normalized reflectance intensity of anomaly points, and the distribution bias of anomaly areas along the main axis of the ore body. The main axis of the ore body is obtained by performing PCA on the spatial coordinates of the point cloud data of the ore body instance to obtain the first principal component direction, and the distribution bias is obtained by subtracting the mean projection of anomaly points in the main axis direction from the mean projection of all points in the main axis direction.

[0102] The statistical characteristics of multi-scale reflection intensity distribution are generated through a multi-scale intensity statistical head. The controller constructs spherical neighborhoods with radii of 0.02m and 0.05m centered on each point and then analyzes the values ​​within the neighborhoods. Calculate the mean, standard deviation and Quantiles yield local statistics, and then mean aggregation and maximum aggregation are performed on the local statistics of all points to obtain the intensity statistical characteristics of the ore body instance at two spatial scales. Simultaneously, the controller performs analysis on all points... Construct a histogram of 16 equal-width intervals and normalize it to obtain the global intensity distribution characteristics;

[0103] To jointly encode spatial structure and intensity distribution, the controller performs point cloud feature encoding on the normalized point cloud. This point cloud feature encoding is implemented by a Transformer-based point cloud feature encoding network, with the network input being a normalized point cloud with a fixed number of points. The point count is fixed by resampling the farthest point to 4096, and padding by index when the number of points is insufficient. The network's input embedding layer will... Mapping to a 128-dimensional feature vector through a fully connected layer and then superimposing it based on... The sinusoidal position encoding is then performed, followed by self-attention encoding through a 4-layer Transformer encoder with 4 multi-head attention heads per layer, a feature dimension of 128, a feedforward network hidden layer dimension of 256, and Dropout of 0.10. The encoded output is then subjected to global max pooling to obtain a 256-dimensional global point cloud intensity structure representation vector, which is then concatenated with the anomaly marker vector statistical results and multi-scale intensity statistical features to form the reflection intensity features of the ore body instance.

[0104] In this specific embodiment, S6 includes:

[0105] The controller performs progressive discrimination for each ore body instance and drives the sorting mechanism to complete the sorting.

[0106] controller from The initial class confidence score of the orebody instance is read from the target detection results and sorted in descending order to obtain the first candidate class confidence score. confidence level with the second candidate category ,in This represents the maximum class confidence score output by the object detection network for this ore body instance, corresponding to the first candidate class. This represents the second-largest category confidence score of the target detection network for this ore body instance, corresponding to the second candidate category. The controller calculates the confidence score difference. ,in This represents the difference between the confidence scores of the first candidate category and the second candidate category, and is used to characterize the initial discrimination.

[0107] The controller sets the preset sorting threshold to Set the preset difference threshold to ,when and When the controller selects the first candidate category as the output category for the ore body instance, it generates a sorting execution instruction. or At that time, the controller will use the density characteristics of the ore body instances. The reflection intensity feature is concatenated with the reflection intensity feature in a fixed order to form a fusion discriminant feature vector, which is then input into the fusion discriminant model for classification. The first dimension of the fusion discriminant feature vector is the density feature. Each subsequent dimension is a reflection intensity feature, which is composed of a 256-dimensional global representation vector output by point cloud feature encoding, a regional statistical feature vector of local high reflection anomaly area, and a multi-scale intensity statistical feature vector concatenated sequentially. The controller performs dimension-wise standardization on the fused discrimination feature vector based on the statistics of the training set and inputs the standardized vector into the multilayer perceptron classifier to output the final ore body instance category.

[0108] The multilayer perceptron classifier is a feedforward neural network with a defined structure and an input dimension equal to the dimension of the fused discriminative feature vector. The network consists of three fully connected hidden layers and one fully connected output layer. The number of neurons in the three hidden layers is as follows: Each hidden layer uses the ReLU activation function, and a dropout rate of 0.20 is applied after the output of each hidden layer. The output layer has 3 neurons, and the number of neurons is the same as the class set. Ore, mineral waste combination, waste rock The probability of each category is obtained through a one-to-one correspondence and Softmax. The controller takes the category with the highest probability as the final ore body instance category and binds it to the ore body instance identifier.

[0109] After obtaining the output category, the controller queries the latest center point location of the ore body instance in the detection area based on the ore body instance identifier and combines it with the conveyor belt speed. The trigger time for the ore body instance to reach the sorting actuator is calculated based on the distance along the conveyor belt from the detection area to the sorting actuator. When the trigger time arrives, the controller outputs the corresponding action code to the solenoid valve drive plate of the sorting actuator and maintains the valve open time for 30ms. When the output category is ore, the action code is to pass without triggering the jetting. When the output category is mineral waste mixture, the action code is to trigger the first nozzle to push it into the medium-grade material trough. When the output category is waste rock, the action code is to trigger the second nozzle to push it into the waste rock trough. Thus, when the initial judgment confidence is high, the target detection result is directly used to achieve rapid sorting. When the initial judgment is uncertain, the final category is determined by the fusion of density characteristics and reflection intensity characteristics to drive the sorting mechanism to complete online sorting.

[0110] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0111] This invention constructs a progressive intelligent discrimination link for online sorting scenarios on conveyor belts, enabling information from various modalities to form a closed loop around the same ore body instance. First, target detection using color images achieves rapid initial judgment and outputs confidence scores. Then, instance segmentation based on candidate region hints obtains pixel-level masks, achieving instance-level constraint purification of depth and point cloud data. On this basis, depth-volume estimation constrained by the mask is used, combined with dynamic weighing mass calculation to calculate density features, compensating for the difficulty in consistently distinguishing similar-looking targets under two-dimensional vision. Simultaneously, reflection intensity is extracted from the instance point cloud and physically compensated and normalized to generate intensity features that characterize material differences. Finally, a confidence-gated progressive discrimination mechanism is adopted, triggering the fusion classification of density and reflection intensity features only when the initial judgment is uncertain. This improves the accuracy and robustness of identifying difficult-to-distinguish ore bodies under complex lighting, dust, and surface mud and sand conditions, while also meeting the needs of online real-time processing.

[0112] Compared to existing technologies that rely on single appearance features or coarse-grained fusion, this invention improves the algorithm structure to address the technical problems of this case: First, it introduces cross-time instance association and cross-modal mapping, ensuring that features such as volume, mass, and point cloud intensity strictly correspond to the same ore body instance, reducing misjudgments caused by mismatches in multi-source data; Second, it adds physical normalization processing of reflection intensity based on ranging distance and incident angle on the laser side, and further explicitly characterizes the distribution of local high-reflection anomaly areas and multi-scale intensity statistics, thereby enhancing the sensitivity and interpretability of locally differentiated targets such as mineral waste aggregates; Third, it adopts a gated triggering progressive discrimination architecture, focusing the high-cost multi-feature fusion calculation on uncertain samples, achieving a more stable sorting effect without sacrificing real-time performance.

Claims

1. A deep learning-based ore sorting method, characterized in that, include: S1. Acquire color images, depth images, multi-line LiDAR point cloud data, and dynamic weighing quality data in the detection area, perform time alignment, and obtain a synchronous dataset. The multi-line LiDAR point cloud data includes the spatial coordinates of the points and the original reflection intensity. S2. Establish a cross-modal mapping relationship between the pixel coordinates of the color images and the spatial coordinates of the multi-line LiDAR point cloud data. S3. Perform target detection on the color images in the synchronous dataset, output the candidate regions of the ore body and the initial category confidence, perform instance segmentation on the ore body based on the candidate regions, output the instance mask, and perform instance association between the target detection results and the instance segmentation results at consecutive time steps to obtain the ore body instance identifier. S4. Extract ore body instance depth data from depth images based on instance masks and estimate ore body instance volume. Determine ore body instance mass from dynamic weighing mass data based on ore body instance identifiers. Calculate ore body instance density features. Extract ore body instance point cloud data from multi-line lidar point cloud data based on instance masks and cross-modal mapping relationships. S5. Perform physical normalization processing on the original reflection intensity in the point cloud data of the ore body instance based on the ranging distance and the incident angle to obtain the normalized reflection intensity and generate reflection intensity features, including the distribution features of local high reflection anomaly areas and the statistical features of multi-scale reflection intensity distribution. S6. Output the ore body instance category based on the progressive discrimination mechanism and drive the sorting mechanism to perform sorting: When the initial category confidence meets the preset sorting conditions, output the ore body instance category corresponding to the target detection and drive the sorting mechanism to perform sorting; otherwise, input the ore body instance density feature and reflection intensity feature into the fusion discrimination model for classification and discrimination, output the final ore body instance category and drive the sorting mechanism to perform sorting.

2. The ore sorting method based on deep learning according to claim 1, characterized in that, S1 includes: The color image, the depth image, the multi-line lidar point cloud data, and the dynamic weighing mass data are collected respectively within the detection area of ​​the conveyor belt, and the color image, the depth image, the multi-line lidar point cloud data, and the dynamic weighing mass data are time-aligned using a unified trigger signal or timestamp. The color image and the depth image under the same time reference after time alignment are paired to form image data. The multi-line lidar point cloud data and the dynamic weighing mass data within the time period corresponding to the image data are associated with the image data on the same time axis to obtain the synchronous dataset. Each point in the multi-line lidar point cloud data includes the spatial coordinates and the original reflection intensity.

3. The ore sorting method based on deep learning according to claim 1, characterized in that, S2 include: The trigger signal or timestamp of the acquisition device is calibrated to complete time synchronization; The imaging parameters of the acquired color image are obtained, and the pixel coordinate system of the color image and the spatial coordinate system of the multi-line lidar point cloud data are spatially calibrated to obtain the coordinate transformation parameters from the spatial coordinate system to the pixel coordinate system. Based on the coordinate transformation parameters, the spatial coordinates in the multi-line lidar point cloud data are transformed to the pixel coordinates of the color image, and a coordinate mapping relationship is established between the pixel coordinates of the color image and the spatial coordinates of the multi-line lidar point cloud data, thus obtaining the cross-modal mapping relationship.

4. The ore sorting method based on deep learning according to claim 1, characterized in that, S3 includes: Target detection is performed on the color images in the synchronous dataset. The Transformer-based target detection network outputs multiple detection candidates and performs overlap suppression on the detection candidates to determine the candidate region corresponding to each ore body instance. At the same time, the initial class confidence corresponding to the candidate region is determined to obtain the target detection result. Based on the target detection results, the prompt information is generated for each candidate region, and the prompt information includes at least the regional location parameters of the candidate region; The instance segmentation network is driven by the prompt information to perform pixel-level segmentation in the candidate region and output the instance mask that corresponds one-to-one with the pixel coordinates of the color image to obtain the instance segmentation result; when the target detection result and the instance segmentation result at consecutive time steps are associated to obtain the ore body instance identifier, the process includes: predicting the motion of the ore body instance in the candidate region position at adjacent time steps based on the conveyor belt speed. Construct a cross-time matching cost, which includes the intersection-union ratio of candidate regions and the distance between the center points of candidate regions; Based on the matching cost, a one-to-one correspondence across time periods is determined, and the correspondence is updated to reflect the trajectory, outputting the ore body instance identifier.

5. The ore sorting method based on deep learning according to claim 1, characterized in that, S4 includes: Based on the instance mask in the instance segmentation result, the depth value corresponding to the pixel covered by the instance mask is selected in the depth image, and outlier values ​​are removed from the selected depth value to obtain the ore body instance depth data corresponding to the instance mask. Based on the depth data of the ore body instance, a truncated signed distance field is constructed within a preset voxel grid. The volume of the ore body instance is obtained by volume integration of voxel cells that satisfy preset integration conditions within the voxel grid. A weighing time window for the ore body instance is determined based on its identifier. This weighing time window is the time interval during which the ore body instance passes the weighing sensor. Determining the weighing time window includes: acquiring the distance along the conveyor belt from the detection area to the weighing sensor and the conveyor belt speed; calculating the time offset of the ore body instance from the detection area to the weighing sensor based on the distance and the conveyor belt speed; mapping the detection time corresponding to the ore body instance identifier to the weighing time based on the time offset; and determining the weighing time window. Within the weighing time window, the dynamic weighing mass data is truncated, and baseline correction is performed on the truncated mass data to obtain the ore body instance mass. The density characteristic of the ore body instance is calculated based on the mass and volume of the ore body instance, whereby the density characteristic of the ore body instance is the ratio of the mass to the volume of the ore body instance. Based on the cross-modal mapping relationship, the spatial coordinates in the multi-line lidar point cloud data are mapped to the pixel coordinates of the color image, and points whose mapped pixel coordinates fall within the range of the instance mask are selected to form ore body instance point cloud data corresponding to the instance mask.

6. The ore sorting method based on deep learning according to claim 1, characterized in that, S5 include: The distance to be measured at each point is calculated based on the spatial coordinates of each point in the point cloud data of the ore body instance. The neighboring points of the point are obtained and plane fitting is performed to obtain the surface normal vector of the ore body corresponding to the point. The incident angle corresponding to the point is determined by the angle between the laser emission direction and the surface normal vector of the ore body. The reflection intensity compensation coefficient is calculated based on the ranging distance and the incident angle, and the original reflection intensity is compensated to obtain the compensated reflection intensity; The compensated reflection intensity is normalized to obtain the normalized reflection intensity, forming a normalized point cloud containing the spatial coordinates and the normalized reflection intensity; The normalized point cloud is subjected to point cloud feature encoding, which is implemented by a Transformer-based point cloud feature encoding network. During the encoding process, local high-reflectance anomaly regions on the ore body surface are determined based on the normalized reflection intensity, and anomaly marker vectors are generated. The anomaly marker vectors are used to characterize the distribution characteristics of the local high-reflectance anomaly regions. The determination of the local high-reflectance anomaly regions includes: determining an anomaly judgment threshold based on the normalized reflection intensity distribution of the ore body instance point cloud. The anomaly judgment threshold includes at least one of the upper quantile threshold and the mean plus standard deviation threshold. When the normalized reflection intensity of a point exceeds the anomaly determination threshold, the local neighborhood to which the point belongs is marked as a high-reflection anomaly region. Simultaneously, a multi-scale intensity statistical head is used to statistically aggregate the normalized reflectance intensity distribution at different spatial scales to form multi-scale intensity statistical features. The multi-scale intensity statistical head performs statistical aggregation of normalized reflectance intensity based on neighborhoods of at least two different spatial scales. The spatial scales include at least one of spherical neighborhoods of different radii and voxel grids of different sizes, and outputs the reflectance intensity features. The reflectance intensity features include the distribution features of the local high reflectance anomaly region and the multi-scale intensity statistical features.

7. The ore sorting method based on deep learning according to claim 1, characterized in that, S6 include: The first candidate class confidence score is the maximum class confidence score output by the target detection network, the second candidate class confidence score is the second largest class confidence score output by the target detection network, and the confidence score difference is the difference between the first candidate class confidence score and the second candidate class confidence score. When the confidence level of the first candidate category is less than the preset sorting threshold or the confidence level difference is less than the preset difference threshold, the ore body instance density feature and the reflection intensity feature are concatenated to obtain a fused discriminant feature, and the fused discriminant feature is input into a multilayer perceptron classifier or a Transformer-based classifier to output the final ore body instance category.

8. The ore sorting method based on deep learning according to claim 5, characterized in that, After extracting the point cloud data of the ore body instance, cross-modal consistency verification is also included: the point cloud data of the ore body instance is mapped to the pixel coordinates of the color image through coordinate transformation, and the proportion of the mapped points falling within the range of the instance mask is calculated. When the proportion is less than a preset consistency threshold, the instance association is re-executed or the point cloud data of the current ore body instance is removed to avoid false association.

9. The ore sorting method based on deep learning according to claim 6, characterized in that, The distribution characteristics of the local high reflectance anomaly region include regional statistical features. The generation of the regional statistical features includes: spatial clustering of points exceeding the anomaly determination threshold to form an anomaly region set, and calculating at least one of the following: the number of anomaly regions, the area ratio of anomaly regions, the maximum normalized reflectance intensity of anomaly regions, and the distribution bias of anomaly regions along the main axis of the ore body.

10. A deep learning-based ore sorting system for performing the deep learning-based ore sorting method according to any one of claims 1 to 9, comprising: The data acquisition module is used to collect color images, depth images, multi-line lidar point cloud data and dynamic weighing mass data in the detection area and perform time alignment to obtain a synchronous dataset. The multi-line lidar point cloud data includes the spatial coordinates of the points and the original reflection intensity. The system includes a cross-modal mapping module for establishing a cross-modal mapping relationship between the pixel coordinates of a color image and the spatial coordinates of multi-line LiDAR point cloud data; a target detection and segmentation module for performing target detection on the color image, outputting candidate regions and initial category confidence scores, performing instance segmentation based on the candidate regions, outputting instance masks, and associating instances to obtain ore body instance identifiers; a density and point cloud module for extracting ore body instance depth data and estimating ore body instance volume based on the instance mask, determining ore body instance quality and calculating ore body instance density features based on the ore body instance identifier, and extracting ore body instance point cloud data based on the instance mask and the cross-modal mapping relationship; and a reflection intensity feature module for performing physical normalization processing on the original reflection intensity based on the ranging distance and incident angle to obtain normalized reflection intensity, and generating reflection intensity features, which include local high reflection anomaly area distribution features and multi-scale reflection intensity distribution statistical features. The discrimination and sorting module is used to output the ore body instance category based on the progressive discrimination mechanism and drive the sorting mechanism to perform sorting. When the initial category confidence meets the preset sorting conditions, the ore body instance category corresponding to the target detection is output. Otherwise, the ore body instance density feature and reflection intensity feature are input into the fusion discrimination model to output the final ore body instance category.

Images