Belt ore drawing collaborative control method based on visual recognition and three-dimensional laser fusion

Abstract

This invention discloses a belt conveyor ore discharge collaborative control method based on visual recognition and 3D laser fusion, specifically relating to the field of mine automation control technology. The method involves acquiring continuous image sequences of the ore discharge area and corresponding 3D laser point cloud data; extracting the ore accumulation edge contour based on the image sequences; and using a semantic segmentation network to identify the effective ore accumulation area. Combining the area and point cloud data, a 3D ore pile surface model is extracted and a fitted boundary point set is generated. A spatial registration algorithm is used to construct an ore accumulation geometric model. The ore accumulation centroid, volume change rate, and edge gradient features are extracted from this model and used as inputs to drive the collaborative control model, adjusting the belt speed and the ore discharge valve opening. Based on real-time control errors and ore pile state feedback, the weight parameters in the control model are dynamically corrected to achieve adaptive ore discharge control. This method can achieve accurate identification and efficient control of the ore pile state, improving the automation, intelligence, and stability of ore discharge operations.

Description

Technical Field

[0001] This invention relates to the field of mine automation control technology, specifically to a belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion. Background Technology

[0002] Traditional belt conveyor ore discharge operations typically achieve precise ore or material discharge by controlling the start and stop of the belt conveyor and coordinating its operation with the feeding device. However, in complex mining areas with limited space or drastically changing environments, the following key issues often arise: Large spatial coupling error between the ore discharge port and the receiving conveyor belt: Especially during the operation of the conveyor belt, due to the uncertainty of ore accumulation, settlement or slight displacement of the foundation of the ore discharge equipment, it is easy to cause ore discharge deviation, material spillage, or even belt deviation.

[0003] There is a lack of accurate means of sensing the real-time geometric state of ore: current methods rely mainly on monocular vision or infrared detection, which are significantly affected by dust obstruction and changes in illumination, making it difficult to stably extract the ore accumulation boundary and volume contour features.

[0004] The efficient fusion of 3D laser point clouds and visual images is difficult: existing algorithms rely heavily on external parameter calibration and fixed scenes, making it difficult to cope with fusion drift caused by dynamic vibrations of equipment or slight perturbations of the viewpoint, resulting in the control system being unable to obtain an accurate ore pile model that can be used for feedback in real time.

[0005] In complex mining areas, even minor errors in ore discharge can lead to serious hazards such as ore pile collapse, material jamming, and equipment damage. How to achieve stable identification, accurate modeling, and highly reliable fusion perception of the geometric features of ore piles, and to feed the perception results back to the collaborative control system in real time, is a key problem that urgently needs to be solved in the field of automated control of conveyor belt ore discharge. Summary of the Invention

[0006] The purpose of this invention is to provide a belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion to address the shortcomings of the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion, comprising: Acquire continuous image sequences of the mining area and corresponding 3D laser point cloud data to construct a time-series registration dataset; The contours of ore deposits are extracted from image sequences, and the effective ore deposit areas are identified using a semantic segmentation network. Based on the identified effective ore accumulation area, and using the three-dimensional laser point cloud data, a three-dimensional ore pile surface model at the corresponding time is extracted to generate a set of fitted boundary points. The ore accumulation edge contour and boundary point set are spatially registered, and the ore accumulation geometric model is constructed by the minimum error fusion algorithm. In the ore accumulation geometry model, the centroid position, volume change rate, and edge gradient features of the ore accumulation are extracted; The ore accumulation center of gravity position, volume change rate and edge gradient characteristics are used as inputs to drive the collaborative control model of the ore discharge belt and the ore feeding device, and adjust the belt speed and the opening of the ore discharge valve. Based on real-time control error and ore pile state feedback, the weight parameters in the cooperative control model are dynamically corrected to achieve adaptive cooperative control.

[0008] Preferably, the step of extracting the ore deposit edge contour based on the image sequence and identifying the effective ore deposit area using a semantic segmentation network includes: Edge enhancement preprocessing was performed on each frame of the constructed temporal registration dataset, and the initial contour of the ore accumulation edge was extracted using the multi-scale Sobel operator. A semantic segmentation neural network with U-Net structure as its core is constructed to perform pixel-level classification on preprocessed images and output a probability map containing mineral regions and non-mineral regions. The probability map is fused with the initial edge contour, and the segmentation result is optimized based on contour similarity and region connectivity to generate a mask map of the effective ore accumulation area. The mask image is used to extract the set of boundary points of the ore accumulation area in the image coordinate system.

[0009] Preferably, the step of generating the fitted boundary point set includes: Based on the boundary point set of the identified effective ore accumulation area in the image coordinate system, it is projected to the three-dimensional laser point cloud coordinate system through the external parameter mapping relationship, and the point cloud subset of the corresponding area is selected. Outlier removal and noise filtering are performed on the point cloud subset, and uniform sampling is performed using a voxel grid filtering method. The moving cube algorithm is used to construct a continuous 3D surface model of the ore pile from a subset of filtered point clouds, extract its surface boundary contour, and generate a set of fitted boundary points.

[0010] Preferably, the step of constructing the ore accumulation geometric model using the minimum error fusion algorithm includes: The coordinates of the ore accumulation edge contour point set in the image coordinate system are normalized, and then transformed to the three-dimensional laser point cloud coordinate system based on the camera extrinsic matrix. An initial rigid registration algorithm is used to calculate the initial rotation matrix and translation vector between the edge contour point set and the three-dimensional boundary point set in order to establish the spatial mapping relationship between the two. Fine registration is performed based on the iterative nearest point algorithm to minimize the Euclidean distance error between corresponding point pairs and optimize spatial alignment accuracy. By jointly interpolating the registered edge contours and boundary point sets, a continuous closed geometric model of ore accumulation is constructed.

[0011] Preferably, in the ore accumulation geometry model, the steps of extracting the centroid location, volume change rate, and edge gradient features of the ore accumulation include: Based on the completed geometric model of ore accumulation, the model is divided into regular three-dimensional voxel units. Each voxel is assigned corresponding spatial coordinates and occupancy state. The overall centroid position of the ore accumulation is calculated based on the voxel distribution. The centroid position is the spatial average value of the center coordinates of each voxel after being weighted by the voxel volume. At consecutive time points, the number of voxels corresponding to the ore accumulation geometric model in two adjacent frames is calculated differentially, and combined with the sampling time interval, the volume change rate of the ore accumulation volume with time is obtained. Along the outer surface boundary of the ore accumulation geometric model, the change of normal vectors of adjacent surface units is extracted. By calculating the spatial gradient magnitude, the edge gradient feature characterizing the steepness of the ore accumulation edge is obtained.

[0012] Preferably, the step of adjusting the belt speed and the opening of the ore discharge valve includes: The extracted ore accumulation centroid position, volume change rate and edge gradient features are uniformly mapped into a standardized control input vector, and normalized according to the preset feature range. A collaborative control model based on the fusion of fuzzy rules and linear weights is constructed. The model adjusts the belt speed by the direction of the center of gravity offset, adjusts the opening of the ore discharge valve by the rate of volume change, and adjusts the ore discharge rhythm by the change of edge gradient. The feature input is dynamically updated within the control cycle to calculate the belt speed adjustment and the target opening value of the ore discharge valve, thus forming a control output command. Control commands are sent to the belt drive and feed actuator in real time to achieve closed-loop coordinated control of ore flow rate, accumulation pattern and ore discharge stability.

[0013] Preferably, the step of dynamically correcting the weight parameters in the collaborative control model includes: in each control cycle, collecting the actual belt speed, ore discharge valve opening and ore pile geometric change results, calculating the error value between them and the control output of the previous cycle, and forming a control error vector; constructing an error sensitivity factor based on the control error vector and historical change trends, and dynamically updating the weight parameters corresponding to each input feature in the collaborative control model using an exponential moving average algorithm.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. The conveyor belt ore discharge collaborative control method based on visual recognition and 3D laser fusion provided by this invention constructs a high-precision image and point cloud registration dataset, integrates multiple perception technologies such as edge detection, deep semantic segmentation, and 3D modeling, and achieves continuous monitoring of ore accumulation status and accurate spatial structure modeling, significantly improving the real-time performance, completeness, and stability of ore pile identification. Simultaneously, the spatial registration algorithm integrating 2D image boundaries and 3D point cloud models ensures the geometric consistency of the perceived data, providing a reliable basis for subsequent control strategies.

[0015] 2. This invention uses the location of the accumulation center of gravity, the rate of change of volume, and edge gradient characteristics as control inputs to construct a collaborative control model that integrates fuzzy logic and linear weighted scaling. It also introduces a weighted adaptive mechanism based on error feedback, which automatically optimizes control parameters according to the actual control effect, achieving stable ore discharge control under different ore particle size distributions, changes in accumulation morphology, and disturbances in the operating environment. This method effectively improves the intelligence level, adaptability, and operational safety of the ore discharge system, and has good industrial application value. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] For examples, please refer to Figure 1 As shown in this embodiment, the belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion includes: Acquire continuous image sequences of the mining area and corresponding 3D laser point cloud data to construct a time-series registration dataset.

[0020] In this invention, in order to achieve high-precision modeling and dynamic monitoring of the ore accumulation state in the belt conveyor mining area, it is first necessary to acquire continuous image sequences and three-dimensional laser point cloud data of the area, and then perform synchronous registration of the two types of data based on timestamps in order to construct the temporal registration dataset required for fusion sensing.

[0021] Specifically, an industrial vision acquisition unit and a lidar scanning device are installed above the ore discharge area. Among them: The industrial vision acquisition unit uses an industrial camera with high frame rate and anti-interference capability, preferably set above or to the side of the ore dumping area to obtain a large field of view coverage and complete image information of the ore accumulation surface. The lidar scanning device adopts a rotating or linear array scanning structure and is deployed in adjacent positions to periodically collect three-dimensional structural information of the ore surface in the mining area and generate dense point cloud data.

[0022] To achieve synchronization and spatial consistency between the two types of data, the system constructs the registration dataset through the following steps: Time synchronization: A unified clock source is used to calibrate the image acquisition unit and the laser scanning device, so that each frame of image and the point cloud data at the corresponding time point are consistent and data mismatch caused by timing deviation is avoided.

[0023] Initial extrinsic parameter calibration: During the deployment phase, spatial extrinsic parameter calibration is performed between the industrial camera and the LiDAR to obtain the position and attitude mapping relationship between the two in the three-dimensional coordinate system, forming an initial registration matrix.

[0024] Dynamic attitude compensation: Considering the minute displacement changes caused by factors such as mine vibration and equipment thermal drift, the system introduces an IMU (Inertial Measurement Unit) module to perform real-time compensation for the attitude of the acquisition unit, thereby improving data stability.

[0025] Construction of temporal registration dataset: Based on the completion of time synchronization and spatial registration, the system matches the continuously acquired image frames with their corresponding point cloud frames one by one to construct a temporal registration dataset in the form of {I(t), P(t)}, where I(t) represents the image frame at time t and P(t) represents the corresponding point cloud frame.

[0026] This dataset serves as the foundational input for subsequent image feature extraction and 3D fusion modeling in this invention, providing crucial perceptual support for achieving high-precision identification and control of the geometric features of the mine pile.

[0027] The contours of ore deposits are extracted from image sequences, and the effective ore deposit areas are identified using a semantic segmentation network.

[0028] In this invention, to further achieve accurate identification of ore accumulation areas, after obtaining the time-series registration dataset, the following steps are performed on each frame of the image in the dataset: First, edge enhancement preprocessing is performed on each frame of the image sequence. The specific steps are as follows: The original image is converted to grayscale to enhance the brightness difference between the ore and the background; A multi-scale edge detection operator group was constructed, and Sobel operators with convolution window sizes of 3×3, 5×5 and 7×7 were used to extract gradient maps in the horizontal and vertical directions, respectively. The gradient maps at the three scales are normalized in magnitude, and the comprehensive edge response value is calculated by weighted average fusion, where the weights are allocated according to the mean square error of the local gradient at each scale.

[0029] Set an edge response threshold, and extract the initial edge contour of the ore accumulation from pixel regions that exceed the threshold to obtain a candidate boundary set.

[0030] To achieve accurate identification of mineral regions in images, this invention employs a fully convolutional neural network based on an encoder-decoder structure, wherein: The encoding part consists of a five-layer convolutional structure, each layer containing two convolutional operations and one pooling operation, used to extract multi-scale spatial features of the image; The decoding part gradually restores the image spatial resolution through deconvolution operations and uses skip connections to fuse high-level features with low-level features. The network output is connected to a Softmax layer, which is used to classify each pixel as a "mineral region" or a "non-mineral region". The output is a two-dimensional probability map with values ​​ranging from 0 to 1, representing the probability that each pixel belongs to the mineral region. The network uses cross-entropy as the loss function and employs a stochastic gradient descent algorithm with a momentum term for iterative training. The training samples are obtained from actual ore discharge images and manually labeled semantic tags.

[0031] To improve semantic segmentation accuracy and correct misclassified regions, the obtained initial edge contours are fused with the ore probability map, as follows: Construct a contour similarity metric function and use the Hausdorff distance to measure the spatial consistency between the initial boundary and the high-confidence region in the probability map; Construct a regional connectivity function to determine whether continuous high-confidence regions in the probability graph meet the area threshold and shape compactness requirements, and remove discrete noise regions; Combining the above two criteria, a weighted integral strategy is adopted to construct a fusion loss function, minimize contour offset and region splitting error, and output an optimized binary mask image; The white area in the mask image represents the effective ore accumulation area after determination, while the black area represents the background or invalid area.

[0032] Finally, the boundary point set is extracted from the optimized ore accumulation mask image, as follows: A contour extraction operation is performed on the binary mask image, and an edge tracking algorithm based on eight-neighborhood connectivity is used to identify the boundaries of all closed regions. Each boundary curve is simplified by using the Douglas-Peucker algorithm to remove redundant inflection points and retain key inflection points; The preserved boundary points are mapped to the image coordinate system to form a structured boundary point set, and the row and column position coordinates of each point are recorded. The boundary point set serves as the target region calibration basis for spatial registration with the 3D laser point cloud model in subsequent steps, and is used to construct the image-point cloud fusion model.

[0033] Based on the identified effective ore accumulation area, and using the three-dimensional laser point cloud data, a three-dimensional ore pile surface model at the corresponding time is extracted to generate a set of fitted boundary points.

[0034] To achieve accurate modeling of the effective ore accumulation area in three-dimensional space, the following steps are performed, combining the identified image boundaries with the acquired three-dimensional laser point cloud data: The image boundary point set is projected onto the 3D laser point cloud coordinate system, and a subset of the point cloud is selected. In this step, the extrinsic parameter relationship between the image coordinate system and the laser point cloud coordinate system is used to accurately map the ore accumulation boundary points in the image to 3D space, and the corresponding subset of the point cloud is selected: Assuming the two-dimensional coordinates of a boundary point in the image coordinate system are (u,v), the normalized direction vector of the pixel is calculated by normalizing the camera intrinsic parameters (including focal lengths fx and fy and principal point coordinates cx and cy). Using the camera extrinsic matrix [R|T], where R is a 3×3 rotation matrix and T is a 3×1 translation vector, the pixel direction vector is back-projected onto the laser coordinate system; In the 3D laser point cloud data, a spherical neighborhood with a radius of 0.05 meters is constructed with each projection point as the center. All point clouds falling into the neighborhood are selected to form a subset of the point cloud.

[0035] The projection and filtering operations are repeated for all boundary points in each frame of the image, and finally a subset of the 3D point cloud of the ore accumulation area corresponding to that frame of the image is obtained.

[0036] To improve point cloud quality and modeling accuracy, the following preprocessing operations are performed on a subset of the point cloud: Outlier removal: For each point Pi, count the 20 nearest points in its neighborhood (typically set to a radius of 0.1 meters); Calculate the average distance Di between this point and all points in its neighborhood; Calculate the mean distance D̄ and standard deviation σ of all points; like If a value is found to be outlier, it is removed from the data.

[0037] Noise filtering: Set a height threshold range (e.g., 0 meters to 2.5 meters) to remove outliers with heights outside this range, thus preventing high-reflectivity false points from participating in the modeling.

[0038] Voxel grid filtering: Construct a three-dimensional voxel grid structure, dividing the space into cubic voxels with a side length of 0.02 meters; in each voxel, calculate the center position coordinates of all points, retain the center point as the representative point, and discard the other points; output a new uniform sampling point cloud set, which reduces the data volume while maintaining the overall contour structure.

[0039] 3D ore heap surface model construction and boundary point set generation. After point cloud preprocessing, a 3D surface reconstruction is performed, and the ore heap boundary point set is extracted, as detailed below: 3D Surface Reconstruction: The moving cube algorithm is used to construct a surface model from the preprocessed point cloud. The point cloud is mapped to a voxel mesh, and each voxel is recorded as to whether it is occupied by the point cloud. The isosurface threshold is set to 0.5, which means that a voxel occupied by more than half of the points is a "solid", and the rest is empty. Local triangular mesh patches are generated at the boundary between solid and empty voxels, and these patches are spliced ​​together to form a continuous ore pile surface.

[0040] Boundary contour extraction: Calculate the rate of change of the normal vector of the curved surface model and extract the boundary regions where the difference in normal vectors is greater than 60 degrees; use the region growing algorithm to search for continuous contour lines at the edge of the model to form a set of boundary point clouds.

[0041] Boundary fitting processing: Boundary points are arranged according to their order in three-dimensional space; a B-spline curve fitting method is applied, with a control point set at fixed intervals to construct a parametric boundary curve; the control point positions are optimized using the least squares fitting method to obtain a smooth and geometrically preserved set of fitted boundary points. The final fitted boundary point set retains the true spatial shape of the ore pile and possesses smooth curvature properties, serving as the core geometric basis for subsequent volume calculations, accumulation trend analysis, and control strategy input.

[0042] The ore accumulation edge contour and boundary point set are spatially registered, and a geometric model of the ore accumulation is constructed using a minimum error fusion algorithm.

[0043] To achieve accurate fusion of ore accumulation edge information between two-dimensional images and three-dimensional point clouds, thereby constructing a complete ore accumulation geometric model, this invention is based on spatial registration between edge contour point sets and boundary point sets, specifically including the following steps: After completing image semantic segmentation and boundary extraction, the set of ore accumulation edge contour points in the image coordinate system is obtained, denoted as the two-dimensional pixel coordinate set {(u1,v1),(u2,v2),…,(un,vn)}. To map this point set to the three-dimensional laser point cloud coordinate system, the following processing is performed: For each point in the image coordinate system, normalization is performed using camera intrinsic parameters, where: The pixel coordinates (u,v) are converted to normalized camera coordinates (x,y) using the following calculation method: , Where fx and fy are the camera's focal lengths, and cx and cy are the principal point positions; a normalized direction vector (x, y, 1) is constructed; this vector is projected onto the laser coordinate system using the camera extrinsic parameter matrix [R|T], with the transformation method being: PL = R × PC + T; where PC is the normalized point's position vector in the camera coordinate system, R is the rotation matrix, T is the translation vector, and PL is the transformed position vector in the laser point cloud coordinate system. The transformed point set is the projection estimation result of the ore edge contour in the image in three-dimensional space, denoted as Pedge.

[0044] To establish the spatial alignment between the image contour point set Pedge and the 3D boundary point set Plidar extracted by laser scanning, an initial rigid registration algorithm is used for initial parameter estimation: The geometric centroids of Pedge and Plidar are calculated using the centroid alignment method. Subtract the centroid of each of the two point sets to obtain the centered point sets; Construct the covariance matrix between two point sets; Perform singular value decomposition on the covariance matrix to obtain the optimal rotation matrix R0; Calculate the translation vector , where C represents the centroid of the point set.

[0045] The resulting (R0,T0) is the initial rigid transformation between the edge contour and the boundary point set, which is used as the initial input for the subsequent fine registration process.

[0046] To further improve registration accuracy and minimize spatial errors between point sets, an iterative nearest-point algorithm is used for precise registration. The steps are as follows: Using R0 and T0 as initial transformations, apply them to the edge contour point set; For each transformed point, find the nearest neighbor in the boundary point set; Constructing the error function , where Pi is the transformed point and Qi is its nearest neighbor; The rotation matrix and translation vector are iteratively optimized using the least squares method until the convergence condition is met. Typically, an error variation threshold is set. rice; Output the final transformation matrix (Ropt, Topt) to achieve high-precision spatial alignment of the two point sets.

[0047] This algorithm can effectively eliminate the residuals caused by the initial registration, ensuring that the correspondence between the two-dimensional contour and the three-dimensional boundary is accurately established in space.

[0048] After completing the point set registration, the geometric model is constructed as follows: The finely registered edge contour point set and boundary point set are merged to form a joint point set, Punion. A spatial surface model is constructed on Punion, and the surface is reconstructed using a triangular meshing algorithm. The surface boundary is closed to ensure that the stacking model has a closed topological structure. Finally, the ore stacking geometric model is output, which can be used to calculate parameters such as ore volume, center of gravity position, and stacking morphology change trend, and provide input data support for subsequent control models.

[0049] In the ore accumulation geometry model, the centroid position, volume change rate, and edge gradient features of the ore accumulation are extracted.

[0050] In this invention, based on the constructed geometric model of ore accumulation, key parameters that characterize the spatial structure changes of the ore accumulation are extracted, including the centroid position, volume change rate, and edge gradient features of the ore accumulation. The specific implementation steps are as follows: To obtain the three-dimensional centroid position of the ore accumulation, the constructed geometric model of the ore accumulation is divided into regular voxels: The three-dimensional space of the ore accumulation model is divided into cubic voxel units with fixed side lengths, preferably with a voxel side length of 0.05 meters; Traverse all triangular meshes in the model. For each voxel, determine whether it is surrounded or passed through by a triangular mesh. If it is occupied, mark its occupation status as 1; otherwise, mark it as 0. For all occupied voxels, calculate their geometric center coordinates, denoted as (xi,yi,zi), and their corresponding voxel volume is V (this value is the cube of the voxel's side length, i.e., 0.05 to the power of cubic meters). The overall centroid coordinates (Xc, Yc, Zc) of the ore accumulation are calculated using a voxel-weighted spatial averaging method, specifically: Xc = (ΣV × xi) / (ΣV), Yc = (ΣV × yi) / (ΣV), Zc = (ΣV × zi) / (ΣV). The centroid position reflects the overall distribution center of the current ore accumulation mass in three-dimensional space.

[0051] To obtain the volume change trend of the ore pile during the dynamic process, volume difference analysis is performed based on the voxel occupancy state at continuous time points: Obtain the ore accumulation geometry model corresponding to time t1 and t2, and perform the above voxel partitioning process respectively to obtain the voxel occupancy set at the two time points; Compare voxels at the same spatial location at two different time points and count the number of voxels whose state changes from "unoccupied" to "occupied". And the number of voxels that changed from "occupied" to "unoccupied". ; Calculate the volume change value Where V is the volume of a single voxel; the sampling time interval is denoted as Δt (in seconds), and the volume change rate Rv is expressed as ΔV / Δt, in cubic meters per second. This volume change rate can be used to determine the ore pile accumulation rate, unloading rate, or other dynamic evolution trends.

[0052] To extract the steepness of the ore accumulation edges, surface gradient analysis is performed on the outer surface of the accumulation model, as follows: For all outer surface triangular patches in the ore accumulation model, calculate their unit normal vector Ni; perform a dot product operation on the normal vectors between two adjacent patches to obtain the cosine of the included angle, denoted as cosθ; calculate the spatial gradient magnitude Gi between adjacent normal vectors, defined as: The value, measured in radians, reflects the degree of abrupt change in the normal direction. The entire model boundary region is traversed, and all boundary points with Gi values ​​greater than a set threshold θ0 are extracted as edge abrupt change regions. A typical threshold is set at 0.52 radians (approximately 30 degrees). For all boundary patches that meet the conditions, their total area or number of points is counted to form an edge gradient feature index, used to reflect the local slope change trend of the ore pile. This edge gradient feature has significant reference value for determining whether there is abnormal accumulation, landslide risk, or local blockage.

[0053] Using the ore accumulation center of gravity location, volume change rate, and edge gradient characteristics as inputs, a collaborative control model for the ore discharge conveyor belt and the ore feeding device is driven to adjust the belt speed and the opening of the ore discharge valve.

[0054] In this invention, to achieve precise linkage control between the ore accumulation state and the ore discharge execution equipment, a control model is constructed using the aforementioned extracted ore accumulation center of gravity position, volume change rate, and edge gradient features. This model can drive the ore discharge conveyor belt and the ore feeding device to operate in coordination. Specifically, the model includes the following steps: To ensure a unified approach to handling multi-dimensional ore pile features in the control logic, the original feature parameters must first be standardized and transformed into a normalized control input vector. Feature parameter definition: The center of gravity offset is set as ΔC=[ΔX,ΔY,ΔZ], in meters, which represents the spatial offset between the current center of gravity and the ideal stacking center; Volume change rate Vrate, in cubic meters per second; The mean edge gradient Gedge, in radians, ranges from 0 to π.

[0055] Normalization method: For each dimension of centroid offset ΔX, ΔY, ΔZ, normalize with ±1.0 meters as the maximum offset to obtain the normalized value Cnorm; The volume change rate was linearly normalized to Vnorm within a range of ±0.5 cubic meters per second. The edge gradient features are normalized to Gnorm by the maximum value π.

[0056] Constructing the input vector: The final control input vector is represented as: U=[Cx,Cy,Cz,Vnorm,Gnorm], where each term is a dimensionless normalized value, ranging from... Or [0,1].

[0057] The control model is constructed by fusing fuzzy logic rules with linear weighting functions, exhibiting good adaptability and dynamic response capabilities. Control target allocation principle: Cx and Cy components are used to control the direction and amplitude of belt speed; Vnorm is used to adjust the opening of the ore discharge valve; Gnorm is used to adjust the ore discharge frequency or rhythm (such as controlling the valve opening and closing interval or step size).

[0058] Fuzzy logic rule definition: Each input is divided into 3 language levels (low, medium, high); a control rule base is constructed. Example: If Cx is positive high, Vnorm is medium, and Gnorm is high → the conveyor belt is adjusted to the left at a faster speed, the valve is opened at a moderate degree, and the ore release pace is slowed down; rule inference uses the Centroid Method for fuzzy solution. Linear weight fusion mechanism: In the linear fusion module, each input dimension is multiplied by a weight factor Wi, and the weights are set based on experience or adaptive strategies; the control output vector... , where W is the weight matrix and Q is the vector transpose.

[0059] Within each control cycle (e.g., every 0.5 seconds), the system performs the following operations: Obtain the input vector U at the current moment; calculate the output control vector through a fuzzy inference and linear fusion model: Vbelt represents the target speed of the belt, in meters per second; Θvalve represents the opening degree of the ore discharge valve, in percentage; Tinterval represents the valve opening and closing rhythm interval, in seconds; Limiting control: Belt speed control range: 0.3 m / s to 1.0 m / s; Valve opening range: 10% to 90%; Rhythm interval range: 0.2 seconds to 3.0 seconds. Example of output control command format (structured command).

[0060] The calculated control commands are directly sent to the field execution equipment to achieve closed-loop control of the ore discharge operation: the belt speed command is sent to the belt drive frequency converter, and speed control is achieved by adjusting the frequency output; the valve opening command is converted into an analog signal or a pulse control signal to drive the electric actuator to adjust the opening of the ore discharge valve; the rhythm control signal is used to set the execution frequency of valve opening and closing or the trigger frequency of control signals to achieve intermittent flow regulation; information such as belt running speed and actual valve opening is collected through the feedback channel, and the deviation is compared with the control target for correction reference in the next control cycle. Through the above process, the system can realize a collaborative control strategy driven by the ore accumulation state based on visual recognition and laser point cloud fusion, ensuring the stability, safety, and adaptability of the ore discharge process.

[0061] Based on real-time control error and ore pile state feedback, the weight parameters in the cooperative control model are dynamically corrected to achieve adaptive cooperative control.

[0062] To further enhance the robustness and adaptability of the ore discharge process control, in a preferred embodiment of the present invention, the weight parameters of each feature input in the collaborative control model are dynamically adjusted based on real-time control error and ore accumulation state feedback information, thereby achieving adaptive collaborative control for complex scenarios. Specifically, this includes the following steps: Within each control cycle (preferably set to 0.5 seconds), the changes in ore accumulation status after the previous cycle are collected along with the execution feedback value of the control output command, and the control error is calculated. The control output feedback items include: actual belt speed vactual (in meters per second); actual ore discharge valve opening θactual (in percentage); actual ore discharge frequency or time interval tactual (in seconds); and actual ore pile status feedback, including the change in center of gravity position ΔCactual and the volume change rate Vrateactual.

[0063] Control error calculation method: The corresponding target control outputs are vtarget, θtarget, and ttarget; the control error vector E is represented as: E=[vactual−vtarget,θactual−θtarget,tactual−ttarget]; the state response error vector S is represented as: S=[ΔCactual−ΔCtarget,Vrateactual−Vratetarget]. These error vectors will be used in the next step of weight adjustment calculation.

[0064] To dynamically adjust the influence of each feature input in the control model on the final control output, an error sensitivity factor matrix is ​​introduced, and the model weight parameters are updated using an exponential moving average algorithm. Construction of error sensitivity factor: the error sensitivity factor λi is defined as the correlation degree between the i-th input feature and the control error E; λi is obtained through correlation analysis of the error vector and feature input variation in historical cycles, with a value range from 0 to 1, representing the weight adjustment priority.

[0065] Weight updating method: suppose the input vector of the current cooperative control model is U=[u1,u2,...,un], and the corresponding weights are W=[w1,w2,...,wn]; the exponential moving average updating formula is used for weight correction: ; wherein ei is the control error component caused by the corresponding feature input, and α is a smoothing factor (preferably set to 0.2); the updated weight vector is used for input calculation of the control model in the next cycle.

[0066] To ensure the dynamic stability and generalization capability of the control model and avoid dramatic adjustment of model weights caused by error fluctuations, the present invention provides a weight updating boundary: definition of constraint conditions: each weight parameter wi is limited within a set interval [wimin, wimax]; wimin and wimax are set empirically according to the importance of input features, for example, the interval for key features is set as [0.3, 0.9], and the interval for secondary features is set as [0.1, 0.5].

[0067] Clipping operation: if the updated weight winew exceeds the boundary, truncation processing is performed: if winew>wimax, winew is set to wimax; if winew<wimin, winew is set to wimin. This process ensures that the weight updating process is continuous and smooth, and prevents the model from jumping or overfitting under noise disturbance or abnormal errors.

[0068] The weight parameters after updating are reloaded into the cooperative control model for calculation of control output in the next cycle: When the control cycle Tc arrives, the updated weight vector Wnew and the standardized input vector U are used to recalculate the control output vector: , where Q is vector transposition; the output includes the target belt speed vtargetnew, the target opening of ore drawing valve θtargetnew and the ore drawing rhythm ttargetnew.

[0069] Control instructions are generated according to the above target values and sent to the actuators, forming a closed-loop control chain of "perception-control-feedback-adaptive adjustment"; continuous updating and correction are performed in multiple cycles, so that the control model can automatically optimize with environmental changes and differences in accumulation states, and improve the overall control effect and robustness.

[0070] This invention enables the collaborative control model to have dynamic learning and self-adjustment capabilities through a weighted adaptive strategy driven by control error. This allows it to adapt to complex working conditions such as uneven ore particle size, irregular stacking, and equipment response fluctuations, thereby improving the safety, accuracy, and stability of ore discharge operations.

[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for coordinated control of conveyor belt ore discharge based on visual recognition and three-dimensional laser fusion, characterized in that: include: Acquire continuous image sequences of the mining area and corresponding 3D laser point cloud data to construct a time-series registration dataset; The contours of ore deposits are extracted from image sequences, and the effective ore deposit areas are identified using a semantic segmentation network. Based on the identified effective ore accumulation area, and using the three-dimensional laser point cloud data, a three-dimensional ore pile surface model at the corresponding time is extracted to generate a set of fitted boundary points. The ore accumulation edge contour and boundary point set are spatially registered, and the ore accumulation geometric model is constructed by the minimum error fusion algorithm. In the ore accumulation geometry model, the centroid location, volume change rate, and edge gradient features of the ore accumulation are extracted, including: Based on the completed geometric model of ore accumulation, the model is divided into regular three-dimensional voxel units. Each voxel is assigned corresponding spatial coordinates and occupancy state. The overall centroid position of the ore accumulation is calculated based on the voxel distribution. The centroid position is the spatial average value of the center coordinates of each voxel after being weighted by the voxel volume. At consecutive time points, the number of voxels corresponding to the ore accumulation geometric model in two adjacent frames is calculated differentially, and combined with the sampling time interval, the volume change rate of the ore accumulation volume with time is obtained. Along the outer surface boundary of the ore accumulation geometric model, the change of normal vectors of adjacent surface units is extracted, and the edge gradient features characterizing the steepness of the ore accumulation edge are obtained by calculating the spatial gradient magnitude. Using the ore accumulation center of gravity location, volume change rate, and edge gradient characteristics as inputs, a coordinated control model for the ore discharge conveyor and feeding device is driven to adjust the conveyor speed and ore discharge valve opening, including: The extracted ore accumulation centroid position, volume change rate and edge gradient features are uniformly mapped into a standardized control input vector, and normalized according to the preset feature range. A collaborative control model based on the fusion of fuzzy rules and linear weights is constructed. The model adjusts the belt speed by the direction of the center of gravity offset, adjusts the opening of the ore discharge valve by the rate of volume change, and adjusts the ore discharge rhythm by the change of edge gradient. The feature input is dynamically updated within the control cycle to calculate the belt speed adjustment and the target opening value of the ore discharge valve, thus forming a control output command. Control commands are sent to the belt drive and feed actuator in real time to achieve closed-loop coordinated control of ore flow rate, accumulation pattern and ore discharge stability; Based on real-time control error and ore pile state feedback, the weight parameters in the cooperative control model are dynamically corrected to achieve adaptive cooperative control, including: Within each control cycle, the actual belt speed, ore discharge valve opening, and ore pile geometric changes are collected, and the error values ​​between these values ​​and the control output of the previous cycle are calculated to form a control error vector. Based on the control error vector and historical trends, an error sensitivity factor is constructed, and the weight parameters corresponding to each input feature in the collaborative control model are dynamically updated using an exponential moving average algorithm.

2. The belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion according to claim 1, characterized in that: The steps of extracting the ore deposit edge contours based on image sequences and identifying the effective ore deposit areas using a semantic segmentation network include: Edge enhancement preprocessing was performed on each frame of the constructed temporal registration dataset, and the initial contour of the ore accumulation edge was extracted using the multi-scale Sobel operator. A semantic segmentation neural network with U-Net structure as its core is constructed to perform pixel-level classification on preprocessed images and output a probability map containing mineral regions and non-mineral regions. The probability map is fused with the initial edge contour, and the segmentation result is optimized based on contour similarity and region connectivity to generate a mask map of the effective ore accumulation area. The mask image is used to extract the set of boundary points of the ore accumulation area in the image coordinate system.

3. The belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion according to claim 1, characterized in that: The step of generating the fitted boundary point set includes: Based on the boundary point set of the identified effective ore accumulation area in the image coordinate system, it is projected to the three-dimensional laser point cloud coordinate system through the external parameter mapping relationship, and the point cloud subset of the corresponding area is selected. Outlier removal and noise filtering are performed on the point cloud subset, and uniform sampling is performed using a voxel grid filtering method. The moving cube algorithm is used to construct a continuous 3D surface model of the ore pile from a subset of filtered point clouds, extract its surface boundary contour, and generate a set of fitted boundary points.

4. The belt conveyor ore discharge collaborative control method based on visual recognition and three-dimensional laser fusion according to claim 1, characterized in that: The steps for constructing the ore accumulation geometric model using the minimum error fusion algorithm include: The coordinates of the ore accumulation edge contour point set in the image coordinate system are normalized, and then transformed to the three-dimensional laser point cloud coordinate system based on the camera extrinsic matrix. An initial rigid registration algorithm is used to calculate the initial rotation matrix and translation vector between the edge contour point set and the three-dimensional boundary point set in order to establish the spatial mapping relationship between the two. Fine registration is performed based on the iterative nearest point algorithm to minimize the Euclidean distance error between corresponding point pairs and optimize spatial alignment accuracy. By jointly interpolating the registered edge contours and boundary point sets, a continuous closed geometric model of ore accumulation is constructed.

Images