Vanadium ore sorting intelligent control system based on machine vision

Abstract

The invention discloses a vanadium ore separation intelligent control system based on machine vision, relates to the technical field of ore separation intelligent control, and solves the problem of inaccurate separation caused by MFAC convergence, time delay coupling and a two-dimensional vision depth blind area of an existing ore separation system. Synchronizing multiple paths of process signals and images based on a hardware timestamp through an acquisition module; the fusion module constructs a multi-modal mine condition model and performs online filtering to extract a state vector and a trigger point; the observation module adopts online augmented Kalman decoupling ore grinding-rotational flow-grading coupling; the control module introduces sliding time domain optimization of network time delay compensation; the positioning module reconstructs a three-dimensional attitude through multi-view structured light phase decoding and baseline self-calibration; the execution module uses rolling time domain window trajectory scheduling; the calibration module adaptively updates parameters of each sub-module in real time; according to the method, the sorting control robustness and the injection positioning precision are remarkably improved, the wrong sorting rate and the maintenance cost are reduced, and the stable productivity under high flux is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control of ore dressing, and more particularly to a vanadium ore sorting intelligent control system based on machine vision. BACKGROUND

[0002] Vanadium ore is often associated with titanium and chromium minerals, and its surface optical properties are easily disturbed by the environment (such as dust humidity). Traditional sorting relies on manual experience or simple sensors, which is low in efficiency and unstable in precision. The intelligent control system based on machine vision combines image recognition and automatic control, which can realize real-time analysis of ore characteristics and precise regulation of sorting equipment, and becomes a key technology direction to improve the utilization rate of vanadium ore resources.

[0003] The existing technical solutions integrate visual recognition, big data analysis and lightweight neural networks in multiple aspects. In the aspect of two-dimensional visual recognition, patent CN119346466A proposes to use a ring-shaped LED light source and a high-speed CCD / CMOS camera to obtain real-time ore conveying images, extract the ore boundary and ore separator position through gray threshold, edge detection and morphological processing, and generate ore separator translation and rotation instructions based on the intersection of the boundary line and the adjustment line. In the aspect of big data analysis, patent CN118807969A constructs a three-level digital twin model of devices, units and whole processes, synchronizes the data of multi-source sensors such as grinding, cyclone and classification with the virtual model at high frequency, and uses the model-free adaptive control (MFAC) algorithm to close-loop adjust the ore supply pressure and concentration to improve the adaptability and online monitoring coverage of the ore dressing process. Patent CN118820980A stores historical ore sample and working condition data, uses a multi-level decision tree driven by information gain to automatically select sorting parameters, and applies the optimized values to subsequent sorting processes to realize the intelligentization and automation of parameter selection. In the aspect of lightweight deep learning target detection, patent CN115953568A constructs a detection model based on lightweight networks such as MobileNet and ShuffleNet, and performs online inference through an edge AI box to balance the sorting accuracy and computing resource occupation. In actual large-scale production, the above technologies still have some problems: First, due to the nonlinear coupling of the grinding, cyclone, and classification processes and their susceptibility to sudden changes in the particle size and concentration of the raw ore, the linearization assumption of MFAC based on the local incremental model is easily distorted under strong disturbances. The accumulation of algorithm estimation errors leads to oscillations in the control law. Furthermore, the data link between the field sensors and the intelligent terminal has a delay of approximately 50–200 milliseconds, causing a disconnect between the issuance and execution of control commands, which easily leads to overcompensation or undercompensation, severely affecting the particle size qualification rate and equipment lifespan. In addition, under high-throughput production, belt vibration and ore accumulation can easily cause occlusion of critical corner points, and the fragmentation of the single-view image contour causes numerical deviations in the calculation of the intersection of the adjustment line and the dividing line. At the same time, the system lacks depth information to reflect the pitch or height deviation of the separator. Under the condition of a belt speed of approximately 1.2 m / s, the jet deviation accumulates, resulting in a significant increase in the ore particle misclassification rate and frequent on-site shutdowns for maintenance. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention discloses an intelligent control system for vanadium ore sorting based on machine vision, aiming to solve the problems mentioned in the background technology.

[0005] To achieve the above-mentioned technical effects, the present invention adopts the following technical solution: A machine vision-based intelligent control system for vanadium ore sorting includes: a data acquisition module for periodically capturing images of the flow state and injection points of vanadium ore particles on a conveyor belt, and outputting the original sequence of particle size and concentration with displacement information; The fusion module is used to construct a multimodal mineral condition model based on the original particle size and concentration sequence and the injection trajectory data of the execution module. It extracts the particle distribution and velocity states through online heterogeneous filtering to obtain the ore state vector and the set of injection trigger points. The observation module is used to adaptively identify the coupling degree of the three intervals of grinding, swirling and classification based on the ore state vector and the set of injection trigger points using the online augmented Kalman observation method, obtain the dynamic mapping of the decoupled subsystem, and output the decoupled incremental mapping and coupling residual field. The control module is used to optimize the actuator incremental command by sliding time domain rolling based on the decoupled incremental mapping and residual field, and output the injection timing correction amount and nozzle dynamic displacement command. The positioning module is used to reconstruct the three-dimensional mesh on the surface of the ore separator through multi-view structured light phase decoding and baseline self-calibration algorithm, and map it to the ore particle ejection coordinate system, and output the attitude error vector. The execution module is used to calculate the injection trigger time and pose correction amount based on the output of the control module and the positioning module by using a rolling time-domain window scheduling algorithm to plan the spatiotemporal trajectory and execute it, while simultaneously sending the execution feedback back to the calibration module and the fusion module in real time. The calibration module is used to update the observation and prediction model parameters based on the set of injection trigger points, decoupled incremental mapping and attitude error vector using an adaptive identification algorithm, and feeds back the updated parameters to the fusion module, observation module and control module.

[0006] As a further technical solution of the present invention, the multimodal mineral condition model performs dual-channel feature extraction in parallel on GPU and multi-core CPU architecture: smoothed state estimation is obtained for process values ​​through extended Kalman filtering; texture and edge features are extracted from image frames using a multi-scale convolutional neural network pyramid; the two types of features are decomposed into a multimodal mineral condition tensor through high-order tensor decomposition, and then sparse discriminant projection is used for dimensionality reduction to output an ore state vector and a set of injection trigger point indexes containing granularity tomography, concentration gradient and flow velocity vectors.

[0007] As a further technical solution of the present invention, the online augmented Kalman observation method first constructs an augmented state vector in the observation module. ,in Composed of a multimodal ore state vector and a set of injection trigger points, representing time... The ore increment state vector, The incremental mapping parameters are represented by the three intervals of grinding, cyclone, and classification; then, the discrete nonlinear mapping function is used. With measurement model Define the state transition and observation equations, where To control the input increment vector, To account for process noise, the following procedure is then executed: At any moment In the prediction phase, a first-order Taylor expansion is used in [ Linearization at [point] yields the state transition Jacobian matrix. And combined with process noise covariance The updated augmented covariance is determined by the following formula: (1) In formula (1), To predict the covariance matrix, For a moment The posterior covariance matrix, The process noise covariance matrix; express The transpose of the matrix is ​​then used; subsequently, the prediction state vector is generated on a hardware-accelerated platform via efficient sparse matrix multiplication. With covariance ; During the update phase, the Jacobian matrix is ​​used. Construct the measurement matrix in real time and calculate the Kalman gain. The calculation formula is: (2) In formula (2), To observe the noise covariance matrix, matrix The transpose matrix is ​​used to represent the predicted covariance. Projected onto the observation space; by Comparison with actual measurements With predictive measurement The residual completes the augmented state correction, expressed as: (3) The updated state vector is split into decoupled incremental mapping parameters. With subsystem state And calculate the coupled residual field. .

[0008] As a further technical solution of the present invention, the process of generating the network delay probability distribution model and incorporating the delay correction sequence into the delay embedding state vector includes: Obtain Profinet end-to-end latency samples and construct an empirical histogram; The empirical histogram is resampled at equal intervals to obtain resampled histogram data; Based on the resampled histogram data, the EM algorithm is executed in parallel to estimate the parameters of the Gaussian mixture model, and the mixture weights and covariance matrix are obtained. During the online phase, the mixture weights and covariance matrix are updated using a recursive Bayesian method to generate a real-time Gaussian mixture distribution. Based on the real-time Gaussian mixture distribution, a delay correction sequence is generated and output through inverse transformation sampling.

[0009] As a further technical solution of the present invention, the working method of the rolling time-domain window scheduling algorithm includes: Obtain the optimal control increment sequence and attitude error vector, and fill a sliding buffer of length N in chronological order; The buffer data is resampled at equal intervals to generate a time node sequence; Construct continuous displacement and time-corrected trajectory based on the time node sequence using third-order B-spline interpolation; A linear programming piecewise solver is applied to continuous trajectory slices to calculate the pneumatic valve pulse width and servo interpolation command; When the sliding window moves to the right, the earliest node is replaced and a local QP re-optimization is triggered. The updated trigger time and pose correction instructions are output.

[0010] As a further technical solution of the present invention, the update feedback process of the adaptive identification algorithm includes: Receive jet trigger point set Decoupling incremental mapping and attitude error vector A parametric posterior probability model is constructed using a variational Bayesian expectation-maximization framework, and the joint likelihood function is calculated using the following formula: (4) In formula (4), Let be the probability density function. Let be the parameter vector to be identified, where For the observation module parameters, For prediction model parameters, The parameters are for the control module; the variational Bayesian expectation-maximization framework executes a cyclic iteration from u1 to u3: u1. Approximating the lower bound of evidence using Jessen's inequality. To obtain the expected sufficient statistic, the formula is: (5) In formula (5), Denotes the variational posterior distribution. Indicates the prior distribution of the parameters. The search range for KL divergence constraint parameters; u2, Decompose the parameter space into a linear subset NAND nonlinear subsets ,in: The update uses a recursive least squares method with a forgetting factor, and the input is a decoupled increment. Update the state transition matrix of the observation module; The update is based on the BFGS quasi-Newton method iteration, utilizing the attitude error vector. Jacobian determinant Correct the QP weight matrix Q of the control module; u3, if the Lyapunov function If the derivative is greater than or equal to zero, the parameter projection algorithm will be triggered. Constrained to a stable manifold; the Lyapunov function In It is a positive definite symmetric matrix. for The transpose of ; Finally, the updated Jacobian matrix of the observation module, the convolution kernel weights of the fusion module, and the coefficients of the QP objective function of the control module are output and fed back to the corresponding module parameter interface.

[0011] Based on the above technical solutions, the positive and beneficial effects of the present invention are as follows: This scheme extracts coupled residuals through online augmented Kalman observations and adaptively updates the decoupled incremental model. At the same time, it embeds the network delay probability distribution into the sliding time domain predictive control to ensure that the control law remains consistent under conditions of strong multivariate coupling and sudden disturbances. Regardless of the fluctuation of the raw ore particle size or concentration, it can achieve smooth incremental command updates, avoid overcompensation or undercompensation, and eliminate the oscillation risk caused by the linearization approximation distortion of MFAC from the source.

[0012] The three-dimensional mesh on the surface of the ore separator is reconstructed by combining multi-view structured light phase decoding and baseline self-calibration. The complete six-degree-of-freedom attitude error vector is output and accurately mapped to the injection trigger time and pose correction amount in the rolling time domain window using spatiotemporal trajectory planning. This overcomes the positioning deviation caused by two-dimensional visual occlusion and depth blind zone, and achieves high-precision matching between the injection point and the target ore particles, directly reducing misclassification and omission.

[0013] The system's built-in closed-loop calibration module updates the observer, control weights, and visual calibration parameters in real time based on injection feedback, coupling residuals, and attitude errors, forming an adaptive back-injection mechanism. This allows each module to continuously self-calibrate during operation without relying on manual intervention or periodic shutdowns for calibration, thereby significantly reducing maintenance frequency and improving equipment lifespan and production line uptime.

[0014] The synergistic effect of the above three aspects not only improves sorting purity and recovery rate, but also ensures that the system can maintain stable performance when belt speed and output are increased by eliminating control jitter, reducing missorting rework and maintenance-free calibration, thus achieving continuous and efficient operation in high-throughput environments. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is an architecture diagram of an intelligent control system for vanadium ore sorting based on machine vision, according to the present invention. Figure 2 This is a schematic diagram illustrating the working principle of the data acquisition module of this invention. Figure 3 This is a schematic diagram of the working principle framework of the control module of the present invention; Figure 4 This is a schematic diagram illustrating the working principle steps of the multi-eye structured light phase decoding and baseline self-calibration algorithm of the present invention; The diagram is labeled as follows: 101, Acquisition Module; 102, Fusion Module; 103, Observation Module; 104, Control Module; 105, Positioning Module; 106, Execution Module; 107, Calibration Module. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] To facilitate understanding of this embodiment, a detailed description of a machine vision-based intelligent control system for vanadium ore sorting disclosed in this application will be provided first. Please refer to [link to relevant documentation]. Figure 1 The diagram shown illustrates the framework of a machine vision-based intelligent control system for vanadium ore sorting. The system includes: The acquisition module 101 is used to periodically capture images of the flow state and injection points of vanadium ore particles on the conveyor belt and output the original sequence of particle size and concentration with displacement information. In this embodiment, the acquisition module 101 is mainly used to perform high-precision periodic time-series capture of the flow pattern and injection point images of vanadium ore particles on the conveyor belt, generating an original time-series sequence containing particle size and concentration information. For details, please refer to... Figure 2 The module includes equipment such as a mill pressure transmitter, a hydrocyclone conductivity sensor, a classifier accelerometer sensor, and multiple high-speed camera raw frames. It should be noted that the "mill pressure transmitter" in this application differs from existing pressure sensors; its output signal eliminates temperature drift through a built-in compensation network and achieves an accuracy of ±0.1%FS, preserving the spectral characteristics of pressure fluctuations under high load variations. The "hydrocyclone conductivity sensor" can detect the conductivity of the slurry in the grinding tail in real time, with a sampling rate of up to 2kHz. The "classifier accelerometer sensor," unlike existing accelerometers, introduces a magnetoresistive measurement mechanism to enhance the vibration bandwidth response to 5kHz. The multiple high-speed camera adopts a CMOS shutter array structure and can output four channels of 1080p@200fps raw frames in parallel. All the above acquisition devices embed an IEEE1588 synchronization chip to form a nanosecond-level global clock reference, and achieve deterministic start time alignment of all channels through an external hard-triggered signal link. Specific details can be determined according to actual conditions and are not limited thereto.

[0018] During the timing marking and data transfer process, the acquisition module 101 triggers multi-channel parallel sampling by sending EtherCAT messages through the PLC. The triggering condition is that the conveyor belt speed exceeds 0.5 m / s and the injection valve opening signal exceeds a preset threshold. The trigger signal achieves a granularity of 100 µs time delay guarantee between the substation and the master station through the EtherCAT protocol. The trigger period set by the PLC can be adjusted between 10 ms and 100 ms to adapt to different production capacity requirements. After the output of each sensor, the signal amplitude is first conditioned by a programmable gain amplifier, and then enters a sixth-order Butterworth anti-aliasing filter. Its 3dB cutoff frequency is determined by the actual screening particle size, and is generally taken as 2 kHz. The filtered signal is sampled by a high-speed ADC at a resolution of 16 bits. The ADC sampling clock is consistent with the IEEE 1588 clock, and each sample is assigned a value with a nanosecond-level time domain label. It should be noted that the "sixth-order anti-aliasing filter" in this application is different from the traditional low-order filter. It adopts a multi-feedback loop structure to reduce phase distortion and can better preserve the dynamic characteristics of the slurry. The frame stream data is driven by a hardware trigger circuit to drive the CMOS shutter array. The original frame is accompanied by inter-frame motion vector information. This vector is generated by a global displacement estimation algorithm between two adjacent frames. The sub-pixel accuracy is achieved by using the Lucas-Kanade pyramid optical flow method, which can reflect the tiny movement of mineral particles on the strip in real time.

[0019] After initial sampling, the module writes the obtained scalar stream and frame stream into a depth-first circular FIFO. This FIFO is composed of dual-port SRAM with a capacity of 100MB and supports PCIe Gen3 x8 DMA chain descriptors to achieve zero-copy data transfer. The DMA chain descriptor table records the base address, length, and time domain label of each data stream. If the number of scalar samples in the same time window exceeds the threshold N, the module adaptively increases the number of chain descriptors in the FIFO to avoid untimely data preparation. If the frame stream queue length reaches the set upper limit M, a frame loss protection strategy is triggered, discarding the earliest frame to ensure the real-time performance of subsequent channels. It should be noted that "zero-copy data transfer" in this application refers to direct data transfer between the DMA controller and the device memory mapping area without CPU intervention, thereby reducing bus occupancy and improving real-time transmission bandwidth.

[0020] During DMA writing, the module's internal hardware reordering unit performs frame-level alignment and timing reordering on all data streams entering the FIFO based on time domain labels. The reordering algorithm is based on the principle of local minimum latency priority throughput, completing the merging and order correction of burst packets from N streams in O(klogk) complexity. After reordering, image frames with displacement vectors and synchronization scalar signals flow into the data fusion interface in parallel. This interface implements a timestamp sorting algorithm based on a sliding window on the FPGA, using a variable window length Δt for asynchronous channel data alignment. The window length can be configured by external parameters to accommodate different granularity distribution scenarios. Subsequently, the fusion interface generates timing tuples containing granularity x_i, concentration c_i, and displacement vector v_i from the aligned data, formats them into a fixed message structure, and packages them.

[0021] Finally, the fused time-series data packets enter the industrial Ethernet via the TSN Qbv time window and QoS priority mapping logic. The time window size is set by the system-level scheduler according to the overall control cycle, with the macro-cycle selectable within the range of 1ms to 10ms to ensure deterministic transmission of control commands and observation data. QoS priority mapping assigns different queue priorities based on the data packet importance labels C0, C1, and C2, with the highest priority (A0) for injection point image frames and pressure / concentration signals, ensuring that the subsequent fusion module 102 can receive high-value data with the shortest possible delay. The priority of other monitoring data can be appropriately reduced. It should be noted that the "TSN Qbv time window" in this application differs from the traditional priority method of dividing periodic time slots using a maximum static window, supporting dynamic time slot adjustment to cope with sudden traffic surges.

[0022] During on-site deployment, the acquisition module 101 operates in real time under the collaboration of FPGA and PLC, and outputs link status, FIFO overflow count, and synchronization error RT_err through the diagnostic interface. Specific parameters can be determined according to actual conditions and are not limited thereto. All parameters, triggering conditions, and algorithms described in this embodiment can be adjusted according to the on-site ore particle size distribution, production capacity requirements, and network topology characteristics to meet the requirements of acquisition accuracy and real-time performance under different operating conditions.

[0023] The fusion module 102 is used to construct a multimodal mineral condition model based on the original granularity and concentration sequence and the injection trajectory data from the execution module 106. It extracts the ore particle distribution and velocity states through online heterogeneous filtering to obtain the ore state vector and the set of injection trigger points. The multimodal mineral condition model performs parallel dual-channel feature extraction on a GPU and multi-core CPU architecture: smoothed state estimates are obtained for process values ​​using extended Kalman filtering; texture and edge features are extracted from image frames using a multi-scale convolutional neural network pyramid. The two types of features are then decomposed into a multimodal mineral condition tensor through high-order tensor decomposition. Specifically, the fusion module 102 receives the original granularity-concentration sequence from the acquisition and execution module 106. With the jet trajectory vector sequence Then, first align the two on the timeline: by sampling rate. and trigger timestamp A unified time reference is established, and linear interpolation and sample rounding are performed to prepare time-consistent observation pairs for subsequent parallel feature extraction. , It should be noted that the term "parallel" in this application is different from serial sequential processing, referring to the simultaneous use of GPU cores and CPU thread pools to batch call feature algorithms on data from each channel.

[0024] Parallel feature extraction stage, process sequence First, input the extended Kalman filter (EKF) engine, using the state vector... Constructing the process noise covariance Q and observation noise covariance R: Parallel iterative generation of smoothed state estimation sequences on GPU tensor kernels { }, where F, G, and H are the state transition, input, and observation Jacobian matrices, respectively.

[0025] At the same time, a multi-scale convolutional neural network (CNN) pyramid architecture is used for the image frames: with a predefined set of convolutional kernel sizes. Extract feature maps at each layer The downsampling-upsampling residual connection mechanism is applied to aggregate features at various scales, and the texture feature spectrum Ut and edge intensity spectrum Vt are output.

[0026] In two characteristics After obtaining (Ut,Vt), a third-order spacetime tensor is constructed based on them. Where T is the time length, M is the EKF state dimension, and N is the CNN feature dimension. This indicates vector concatenation. It should be noted that the term "tensor decomposition" in this application differs from general matrix decomposition; it specifically refers to performing high-order singular value decomposition (HOSVD) on high-order tensors to obtain the principal factors of each modality.

[0027] The tensor decomposition stage employs HOSVD: based on the expansion matrices of Mode-1, Mode-2, and Mode-3. Calculate the singular value decomposition separately: Before extracting each pattern Principal component loads Reconstruct the core tensor in ... represents the product of patterns 1, 2, and 3; parameters ...based on the energy retention threshold, Confirmed. The core tensor G, namely the multimodal mineral conditions tensor, encodes the multidimensional coupling characteristics of grain size chromatography, concentration gradient, and flow velocity.

[0028] Next, sparse discriminant projection dimensionality reduction is used to output an ore state vector containing particle size stratification, concentration gradient, and velocity vector, as well as a set of injection trigger point indices. Specifically, the method involves first transforming the ore state vector into a tensor vector. Mapped to feature matrix Where K = r1r2r3, D = T. Based on this, discriminant analysis (LDA) and... - Norm regularization fusion, solving for the projection matrix W: in and These are the within-class and between-class scatter matrices, respectively. It is an eigenvalue diagonal matrix. To force a sparse solution, W is iteratively updated using the Alternating Directional Multiplier Method (ADMM) to finally obtain a dimensionality-reduced mapping. The particle size distribution, concentration gradient, and flow velocity vectors are extracted according to a threshold τ and combined to form the final ore state vector. This is paired with a corresponding set of injection trigger point indices. The vectorization and discriminative dimensionality reduction process is executed in parallel within a multi-core CPU thread pool, utilizing OpenMP to accelerate mapping computation. It should be noted that the "sparse discriminative projection" in this application differs from ordinary PCA dimensionality reduction; it explicitly enhances inter-class separability for subsequent decision-making. The "trigger point index set" is used to identify the geographic grid cells corresponding to the injection nozzles; its specific details can be determined based on actual conditions and are not limited thereto.

[0029] To ensure real-time performance, the system employs an asynchronous pipeline: EKF prediction updates and MS-CNN feature extraction are executed in parallel, while multi-core CPU computing resources are asynchronously invoked during the decomposition and dimensionality reduction stages. During operation, the module outputs processing latency and frame drop rate via heartbeat detection logic. If the processing latency or frame drop rate exceeds a preset threshold, a dynamic resource reallocation mechanism is triggered, adjusting GPU memory allocation or reducing model complexity to maintain system stability. The specific threshold can be determined based on the on-site hardware configuration and is not limited thereto. The parameters, triggering conditions, and algorithm implementations described above can be adjusted according to the on-site ore particle size distribution, injection system topology, and computing platform performance to meet the real-time multimodal ore sentiment perception requirements under different working conditions.

[0030] The observation module 103 is used to adaptively identify the coupling degree of the grinding, swirling, and classification intervals based on the ore state vector and the set of injection trigger points using an online augmented Kalman observation method. This yields a decoupled dynamic mapping of the subsystem and outputs the decoupled incremental mapping and the coupling residual field. It should be noted that the "online augmented Kalman observation" in this application differs from conventional EKF filtering; it uses both the parameters to be estimated and the system state as the augmented vector. The "coupling residual field" refers to the difference between the predicted output of the mapping model and the actual observation, used for subsequent compensation and correction. The augmentation dimension can be set according to the coupling strength and system scale, and there are no limitations on this.

[0031] When the system starts up, the module loads the discrete nonlinear mapping kernel f(·) and the measurement function h(·) through the multi-core CPU, and initializes the augmented state vector. The augmented covariance P0, with parameters including process noise covariance Q0 and observation noise covariance R0, can be set to Q0∈diag(10) based on the dynamic changes of the ore and the measurement accuracy, respectively. -6 …10 -4 ) and R0∈diag(10 -4 …10 -2 It should be noted that in this application, "process noise covariance" is different from observation noise, which reflects the uncertainty of subsystem modeling; "observation noise covariance" reflects sensor measurement error. Both can be determined based on experience or offline calibration, and there are no restrictions on this.

[0032] At each control time k, the triggering condition is that both the fusion module 102 and the execution module 106 have received data packets and the timestamp difference Δt_sync ≤ τ_sync_thresh (τ_sync_thresh can be set to 3 ms). The observation process executes the prediction and update phases sequentially. The prediction phase is based on the augmented state transition equation: in The Jacobian matrix is ​​computed in parallel within the CPU vectorization library using a first-order Taylor series linearization mechanism. For process noise, satisfy (0, Q_k). The prediction results X{k+1|k} and P_{k+1|k} are stored in the cache.

[0033] During the update phase, the observation model uses a measurement function. ,in (0,R_k), The Kalman gain is generated via sparse matrix multiplication on a hardware-accelerated platform. Calculations such as: Then perform state correction: Where the state vector Split into and The former represents the subsystem dynamics, and the latter represents the decoupling incremental mapping parameters of each subsystem.

[0034] Decoupling mapping fᵢ and coupling residual Extracted from: fᵢ(·)= Establish subsystem mapping functions for corresponding components. Asynchronous delivery is made via RPC interface. The RPC payload format is binary serialization Δfᵢ. Priority transmission is ensured through the TSN Qci queue, and serialization is performed by calling the Protobuf protocol. The specific RPC timeout can be set to 10 ms to avoid blocking.

[0035] To ensure online performance, the observation module 103 continuously collects data during operation. Norm and TraceTr( And monitor, when Tr( )>Tr_thresh or κ( When Q_k > κ_thresh, adaptive threshold adjustment is triggered: appropriately increasing Q_k or increasing the sample frequency improves the rolling observation accuracy. It should be noted that in this application, "covariance trace" and "condition number κ" are used to evaluate filter stability and numerical identifiability; the thresholds Tr_thresh and κ_thresh can be set according to on-site requirements and are not limited thereto. The online augmented Kalman observation process described in this embodiment is executed cyclically within each control cycle, enabling the observation module 103 to dynamically capture changes in the coupling degree across the three intervals and output decoupling parameters and residual fields in real time, supporting the adaptive optimization and updating of the control and calibration module 107.

[0036] The control module 104 is used to optimize the actuator incremental commands using a sliding time-domain rolling optimization based on the decoupled incremental mapping and residual field, and output the injection timing correction and nozzle dynamic displacement commands. It should be noted that in a machine vision-based vanadium ore sorting system, the control cycle is typically driven by a synchronous clock issued by the host computer or real-time operating system, with a typical cycle set of 20 ms. At each control moment, the control module 104 first obtains the decoupled three-segment incremental mapping parameter set and the coupled residual field vector from the observation module 103. To maintain the historical disturbance information and its time correlation, the module configures a fixed-depth delay embedding buffer in the FPGA's state register group or high-speed on-chip SRAM. The depth can be 3 to 5 control cycles, and the specific depth is determined based on experimental tuning results. The delay embedding mechanism involves concatenating the current mapping parameters and residual values ​​sequentially after the previous data to form an extended state vector. For example, if the previous mapping parameters were {0.03, –0.02, 0.01} and the residual field was {0.004, –0.005, 0.006}, and the current parameters are {0.02, –0.01, 0.03} and {0.005, –0.004, 0.007}, then after loading the new data, the buffer storage will become an ordered sequence of six values. This sequence is passed to the subsequent prediction and optimization units within the FPGA via the AXI-Stream bus. It should be noted that the term "delayed embedded state" in this application differs from the traditional method of only storing the current state; it refers to appending multiple historical sampled data to the state variables to maintain a memory of past perturbations or model biases.

[0037] Please see Figure 3In specific implementation, the control module 104 reads a recent network latency sample set from the execution layer or communication management unit. This sample typically consists of end-to-end latency measurements periodically reported by the Profinet real-time I / O controller. The number of samples can be set to 50–200, and the specific value can be adjusted according to the degree of bandwidth fluctuation on site. In this application, the term "end-to-end latency sample" differs from general communication timing statistics; it refers only to the time difference measurement from the transmission of a Profinet periodic frame to the feedback confirmation. After reading the latency sample, the module performs fast equidistant resampling through a parallel logic core in the FPGA, binning the sample histogram with equal widths to ensure consistent input dimensions for subsequent probabilistic models. Then, it calls the hardware-based EM (Expectation–Maximization) algorithm kernel to estimate the weights and covariance matrices of each component of the Gaussian mixture model in several parallel iterative loops. This EM algorithm kernel is implemented collaboratively by multiple DSP slices and BRAM, with the number of iterations typically set to 5–10 until the weight update converges. It should be noted that in this application, "Gaussian mixture model" refers to the linear weighted superposition of multi-peak density functions, rather than a single normal distribution. This approach can accurately characterize the multimodal characteristics of industrial network latency.

[0038] After model training is complete, the module updates the mixture weights and covariance at the beginning of each control cycle using a recursive Bayesian method. This process is performed in the embedded microcontroller of the FPGA to ensure the model's adaptive response to sudden load spikes. Subsequently, the control module 104 calls the inverse transform sampling algorithm to perform fast pseudo-random sampling of the current delay distribution, generating a delay correction sequence of the same length as the delay embedding depth. For example, if the depth is 3, three samples are output sequentially, such as {21 ms, 19 ms, 22 ms}. The sampling results are immediately incorporated into the aforementioned delay embedding state, combining the mapping parameters, residuals, and corresponding delay compensation values ​​of each historical cycle into a higher-dimensional state vector stream.

[0039] At this point, the control module 104 constructs a quadratic programming optimization problem with residual compensation and delay correction on the FPGA. The optimization variable is the sequence of executor incremental commands for the next N steps (e.g., N=5). The module sets upper and lower bounds for the incremental commands, such as a minimum of –0.02 MPa and a maximum of 0.02 MPa, and introduces appropriate weights for the residual compensation term to prevent the model from overcompensating the original decoupling error. Specifically, the module transmits all predicted states, residual vectors, and delay correction matrix parameters to the interior-point QP solver kernel via the AXI-Lite bus. The solver kernel consists of a customized KKT matrix generation unit and a multi-channel parallel back-substitution unit in the FPGA. First, it packages the Hessian matrix and gradient vector of the optimization objective into the BRAM, and then starts the iterative process, updating the dual and principal variables in each iteration until the residual magnitude is lower than the preset convergence threshold (e.g., 1e–6). It should be noted that the term "internal point method QP solver kernel" in this application is different from the general CPU library. It combines a custom pipeline in the hardware logic to greatly accelerate factorization and micro-iteration steps, thereby meeting the sub-microsecond solution latency.

[0040] After obtaining the optimal incremental command sequence, the module encapsulates the first step value and its corresponding delay compensation amount together using a FIFO structure and outputs it to the scheduling execution module 106 in Profinet RT packet format. This packet includes the time-corrected trigger time offset and pressure increment command, ensuring that the actual execution of the nozzle operation, after deducting network delay, is aligned with the prediction. The packet is mapped through a TSN Qbv time window and then passed directly from PCIe to the real-time Ethernet PHY, finally being sent to the pneumatic valve and servo actuator.

[0041] It should be noted that the term "rolling optimization" in this application differs from one-time global optimization. It refers to shifting the time window to the right in each control cycle, re-optimizing only the N steps of variables in the latest window, and reusing the calculation results from the previous cycle to reduce computational load and continuously track system dynamics. The specific window size and re-optimization frequency can be determined based on actual control accuracy requirements and FPGA computing resources; no limitations are imposed on these.

[0042] By organically combining delayed embedded states with Gaussian mixture Monte Carlo delay compensation, and implementing hardware-accelerated QP solution with residual compensation on an FPGA, the control module 104 can maintain stable predictive control performance in industrial mineral processing scenarios with strong multivariate coupling and significant communication delays. It automatically performs dual compensation for disturbances and delays, providing precise timing corrections and dynamic pose commands for injection scheduling. Experimental data on key performance indicators of the control module 104 during implementation are presented below to verify the real-time performance and accuracy of the proposed solution under different operating conditions.

[0043] Table 1 Control Module 104 Implementation Data Parameter Record Table Experiment number Sampling rate fps Delay embedding step number N Latency mean (ms) QP solving delay (ms) End-to-end latency (ms) Granularity deviation (pm) Concentration error (%) Convergence iteration number Frame loss rate (%) Exp-01 200 10 15.2 2.8 12.5 5.4 1.2 8 0.5 Exp-02 200 20 16.0 3.1 13.1 4.9 1.0 10 0.7 Exp-03 150 10 18.5 3.5 14.8 6.0 1.5 9 0.6 Exp-04 150 15 17.3 3.2 14.0 5.6 1.3 9 0.4 Exp-05 100 10 20.5 4.0 16.2 6.5 1.8 12 0.9 The above experiments used three image sampling rates: 200 fps, 150 fps, and 100 fps, comparing the impact of prediction steps (N=10, N=15, and N=20) on the latency embedding and QP solution process. The results show that when the sampling rate is 200 fps and N is 10, the system has the lowest end-to-end latency, with granularity bias and concentration error of 5.4 μm and 1.2%, respectively. The QP solution requires the fewest iterations (8), and the frame drop rate is less than 1%. With increasing prediction steps, the QP solution latency and end-to-end latency increase slightly, but the granularity and concentration errors further decrease, verifying the optimization effect of adjustable steps on accuracy. In the lower sampling rate (100 fps) experiment, although the QP solution latency and end-to-end latency increase, the overall accuracy remains within an acceptable range, indicating that the control module 104 has good real-time performance and robustness under different production capacity conditions.

[0044] The positioning module 105 is used to reconstruct the three-dimensional mesh on the surface of the ore separator through multi-view structured light phase decoding and baseline self-calibration algorithm, and map it to the ore particle ejection coordinate system, outputting an attitude error vector. It should be noted that the "multi-view structured light phase decoding" in this application differs from monocular structured light; it utilizes multiple cameras to simultaneously capture coded stripes and Gray code images to achieve phase redundancy. The "baseline self-calibration algorithm" differs from the preset calibration plate method; it adaptively estimates the baseline length and camera parameters based on field data to adapt to dynamic working conditions. The specific details can be determined according to the camera calibration accuracy requirements and field-of-view layout, and are not limited thereto.

[0045] Please see Figure 4 The working principle of multi-eye structured light phase decoding and baseline self-calibration algorithms includes: Acquire coded stripe and Gray code images simultaneously captured by multiple cameras; The coded stripe image is subjected to a three-frequency phase expansion method to generate a continuous phase map, and Gray code decoding is combined to eliminate phase ambiguity and output a deblurred phase map. Based on the deblurred phase mapping, the initial point cloud is extracted. Baseline self-calibration is performed on the initial point cloud and Gray code plane features. The camera intrinsic and extrinsic parameters are iteratively corrected using the tensor bundle optimization method to obtain the accurate intrinsic and extrinsic parameter matrices. The fuzzy phase map back projection is used to construct a three-dimensional mesh on the surface of the ore separator; A rigid transformation is used to map the 3D mesh on the surface of the sorter to the particle ejection coordinate system and align it with the theoretical geometric model, outputting a six-degree-of-freedom attitude error vector containing translational offset and rotational difference. The specific implementation is as follows: During module initialization, a multi-view camera is positioned along the rectangular border of the sorter. The camera coordinates (u_i, v_i) are triggered via a GPIO synchronization line, and the frame rate can be configured from 120 fps to 240 fps to balance sorting speed and reconstruction accuracy. The structured light projector projects a sequence I_c(x, y, t) using three-frequency coded stripes, and triggers a raster scan by an external trigger signal at the start of projection. The camera simultaneously acquires coded stripe patterns and Gray code images, where the coded stripe frequencies f_1, f_2, and f_3 are set to 8, 16, and 32 stripe periods, respectively, to support phase unfolding. The Gray code image sequence is generated according to the minimum Hamming distance principle, and the bit width can be set to n∈[8,12] bits to balance decoding stability and memory overhead.

[0046] In each decoding process, the image preprocessing unit applies a two-dimensional Gaussian filter to the stripe image to remove salt-and-pepper noise. The filter radius σ can be configured within the range of 1.0 to 2.5 pixels. Then, phase calculation is performed: φ_i(x,y)=atan2(Q_i(x,y),I_i(x,y)), where I_i and Q_i are the real and imaginary parts of the encoded stripes, respectively. Phase information is extracted using Hilbert transform. The continuous phase Ψ(x,y)=φ_1(x,y)−φ_2(x,y)·f_1 / f_2+... is calculated using a three-frequency phase expansion method, and combined with the Gray code decoding result G(x,y) to eliminate 2π ambiguity, achieving the deblurred phase mapping Ψ_unwrapped(x,y). It should be noted that the "phase expansion method" in this application differs from the traditional single-frequency phase method; the three-frequency scheme can maintain phase continuity in high-noise environments. The "deblurred phase mapping" is used to generate an unambiguous phase map, and the specific mask parameters can be adjusted according to the stripe frequency ratio, without limitation.

[0047] Based on Ψ_unwrapped(x,y), the initial point cloud P_0={(x_i,y_i,z_i)} is derived from the phase-depth mapping function z=Δφ·λ / 4π·B / T, where Δφ is the phase difference, λ is the structured light wavelength, B is the length of the projection and imaging baseline, and T is the fringe period. It should be noted that the "phase-depth mapping function" in this application differs from geometric triangulation; it directly maps the phase difference to a depth value to achieve sub-pixel-level depth accuracy. The initial point cloud is then associated with the Gray code planar feature F_g through feature matching, where F_g is obtained by Gray code decoding region boundary detection. The matching algorithm uses RANSAC to remove outliers to ensure feature reliability.

[0048] During baseline self-calibration, the correction algorithm first derives the reprojection error ε=Σ‖p_{ij}−π(K_i,[R_i|t_i]·P_j)‖^2 based on the initial camera intrinsic parameters K_i and extrinsic parameters [R_i|t_i]. Here, π(·) represents the camera projection model, p_{ij} represents the pixel coordinates, and P_j represents the point cloud coordinates. To reduce the error, the tensor bundle optimization method is used to iteratively update the parameters Θ_cam={K_i,R_i,t_i} under multi-camera observation constraints. The optimization objective is: min_Θ Σ_{i,j} w_{ij}·ε_{ij}+ρ‖Θ−Θ_0‖^2, where w_{ij} is the observation weight, ρ is the regularization coefficient, and Θ_0 is the initial guess value. Lie group parameterization ensures that the rotation matrix R_i is updated on the SO(3) manifold. It should be noted that the "Tensor Bundle Optimization Method" in this application differs from the conventional BA algorithm. It utilizes high-order tensors to fuse multi-view constraints to accelerate convergence. Specifically, the upper limit of the number of iterations can be determined based on the number of cameras and scene characteristics, and there is no limitation on this.

[0049] After the parameter Θ_cam converges, the corrected intrinsic and extrinsic parameters are substituted into the phase mapping back projection formula to generate an accurate three-dimensional mesh G_surface={(X_u,Y_u,Z_u)}, where (X_u,Y_u) are the projection coordinates of the mesh vertices in the spray coordinate system. The point cloud is smoothly covered at the boundary of the mesh cell by bilinear interpolation. After the mesh is generated, the rigid transformation T_world→spray∈SE(3) maps G_surface to the mineral particle spray coordinate system. Its form is G_mapped=T·G_surface, where T is calculated based on ICP registration and alignment with the theoretical geometric model G_model. The optimal T is when the registration error ε_icp=‖G_mapped−G_model‖_2 is minimized. It should be noted that in this application, "theoretical geometric model" refers to the design CAD model, whose vertex set G_model provides the alignment reference; "rigid transformation" is different from affine transformation, which maintains the geometric rigidity of the mesh to ensure physical consistency.

[0050] After mapping, the attitude error vector E_pose=[Δx,Δy,Δz,Δα,Δβ,Δγ]^T is calculated, where Δx,Δy,Δz are translational offsets, and Δα,Δβ,Δγ are Euler angle rotation errors. Error calculation is based on Euclidean distance and rotation error metrics: Δr=arccos((trace(R_error)−1) / 2). This vector, along with a timestamp, is distributed to the decision optimization module via the TSN Qbu time slot allocation strategy for injection point calibration. It should be noted that the "TSN Qbu time slot allocation" in this application differs from the Qbv static window, as it supports dynamic adjustment based on real-time network load to ensure critical message latency.

[0051] To ensure positioning accuracy, the module continuously monitors the mean reprojection error ε̄ and the ICP residual ε_icp_avg during operation for online diagnostics. When ε̄ > ε_thresh or ε_icp_avg > ε_icp_thresh, an adaptive recalibration process is triggered: the camera uses a predefined checkerboard grid for rapid calibration to reinitialize Θ_0 and update the TSN trigger time slot. It should be noted that the "adaptive recalibration process" in this application differs from manual periodic calibration; it achieves automatic correction through threshold triggering. The specific checkerboard grid size and recalibration frequency can be set according to changes in the field environment and are not limited thereto.

[0052] The parameters, algorithms, and triggering conditions described in the above embodiments can be adjusted according to the geometry of the separator, the camera field of view layout, and the dynamic working conditions on site to meet the requirements of high-throughput and high-precision online three-dimensional positioning.

[0053] The execution module 106 is used to calculate the injection trigger time and pose correction amount based on the output of the control module 104 and the positioning module 105 using a rolling time-domain window scheduling algorithm to plan the spatiotemporal trajectory and execute it. At the same time, it sends the execution feedback back to the calibration module 107 and the fusion module 102 in real time. In this application, the "rolling time-domain window scheduling algorithm" is different from the static scheduling method. It maintains the latest command with a sliding buffer of length N within a continuous control cycle. "Execution feedback" refers to the sampled return data of the injection valve status and nozzle position, which is used for closed-loop correction and fusion update. The specific data can be determined according to the field feedback frequency and is not limited thereto.

[0054] Upon module startup, the execution subsystem deploys a third-order B-spline interpolation kernel and a linear programming piecewise solver kernel on the FPGA logic unit and the multi-core real-time processor. The FPGA portion implements a sliding buffer structure, with a buffer length N that can be set from 10 to 20 to accommodate different dynamic response requirements; the real-time processor is responsible for buffer data management and trigger control. The system receives the Δu_i sequence and E_pose_i vector via an EtherCAT network, with the trigger condition being that the receiving timestamp interval Δt_recv ≤ T_s, ensuring uninterrupted continuous buffering.

[0055] Once the buffer is filled with N time units, the interpolation subprocess starts. Time nodes {t_k} are generated for Δu_i and E_pose_i respectively with an equal-time resampling interval Δt_s=T_s / (N−1). Then, the third-order B-spline interpolation algorithm is called on the real-time processor to perform multi-segment spline fitting on the displacement correction and angle correction sequences. Its mathematical form is S_k(τ)=a_k+ b_kτ+ c_kτ^2+ d_kτ^3, τ∈[t_k,t_{k+1}], and the coefficients {a_k,b_k,c_k,d_k} are obtained by solving the C^2 continuity condition and boundary derivative constraints. The boundary conditions are set by the current actuator feedback velocity and acceleration. It should be noted that the "third-order B-spline interpolation" in this application is different from linear interpolation. It provides a first-order continuous derivative within the interpolation interval to ensure the smoothness of the injection action. Specifically, the boundary derivative can be configured according to the nozzle inertia and hydrodynamic characteristics, and there are no restrictions on this.

[0056] After interpolation, the spline segment parameters are pushed to the FPGA linear programming piecewise solver. This solver establishes the following linear programming problem for each spline interval: min_{w_k,u_k}‖w_k−S_k(τ)‖_1 st A_k·w_k≤b_k, C_k·u_k≤d_k, where w_k is the pulse width vector, u_k is the servo interpolation step command, A_k and C_k are the valve flow rate and servo drive constraint matrices, respectively, and b_k and d_k are the corresponding boundary vectors. These constraints are obtained by mapping parameters from the technical manual provided by the injection valve manufacturer to the servo driver characteristics. The solver uses the simplex method to process the M intervals in parallel on the FPGA, ensuring a total scheduling delay L_sched≤T_s / 2 with minimal computational latency. It should be noted that the "linear programming piecewise solver" in this application differs from overall QP optimization; it performs local solutions for spline segments to reduce computation and improve real-time performance. The specific number of parallel threads can be determined based on FPGA resources and is not limited thereto.

[0057] After the local solution is completed, the module performs a sliding window right-shift operation: the earliest time node data is removed, the buffer index is incremented, and the data at the end of the window is replaced with the latest output of the control and positioning module 105; the real-time processor then triggers the QP re-optimization sub-process to refine the initial segment by incorporating the latest fluctuations and network latency corrections. The QP core receives w_1 and u_1 of the first segment and performs fine-tuning optimization with the support of the interior-point accelerator, outputting the updated trigger time t_1 and pose correction Δp_1. It should be noted that the "QP re-optimization" in this application is different from the initial QP solution; it focuses on compensating for small deviations in the first segment to achieve high-precision jet control; the specific number of QP solution iterations can be configured within the range of 5 to 15, and is not limited thereto.

[0058] During the command encapsulation stage, the FPGA control logic maps t_1 and Δp_1 to the injection control command packet according to a FIFO timing sequence. The command packet format includes {Valve_ID, PWM_w1, Servo_u1, Execute_Time}, where Valve_ID uniquely identifies the solenoid valve channel, PWM_w1 is the pulse width command, Servo_u1 is the servo step value, and Execute_Time is the trigger time. The command packet is sent to the actuator node through the TSN Qci priority queue, ensuring synchronous triggering accuracy ε_sync ≤ 100 μs. It should be noted that the "TSN Qci priority queue" in this application differs from ordinary QoS mapping. It defines command time slots in the time-sensitive network to prioritize the transmission of injection commands; the specific queue depth and time slot length can be set according to the network load and are not limited thereto.

[0059] After injection is completed, the feedback interface of the execution module 106 collects the valve status feedback signal and the servo position encoder output as execution feedback data. The feedback frequency can reach 1 kHz, and the data is transmitted back to the calibration module 107 and the fusion module 102 via EtherCAT fieldbus. The feedback data includes the actual trigger time t_act, the actual pose p_act, and the valve response time τ_resp, which are used to evaluate the execution error online and update the filtering and interpolation parameters. When the execution error exceeds the threshold ε_exec>ε_exec_thresh, the adaptive gain update logic is triggered to adjust the B-spline boundary conditions and valve constraint matrix for the next cycle. It should be noted that the "execution feedback" in this application is different from offline testing; it realizes online closed-loop error correction. The specific feedback threshold can be determined according to the injector performance and is not limited thereto.

[0060] The calibration module 107 is used to update the observation and prediction model parameters based on the set of injection trigger points, decoupled incremental mapping, and attitude error vector using an adaptive identification algorithm, and feeds the updated parameters back to the fusion module 102, observation module 103, and control module 104. In the calibration module 107, when the execution module 106 and positioning module 105 set the set of injection trigger points... Decoupling incremental mapping vector and attitude error vector Once the data is transmitted via the low-latency bus, the variational Bayesian posterior update process is triggered. The posterior parameter vector referred to here... Including observation subset Prediction subset With control subset These correspond to the Jacobian matrix elements of observation module 103, the weight coefficients of MPC optimization, and the mixed distribution weights of the network latency compensation model, respectively. Calibration module 107 first performs calibration on the embedded ARM processor based on the current... Constructing the joint likelihood function ,in Assuming the design matrix is ​​used The multivariate Gaussian distribution defined by the prior covariance matrix Σobs, p( The reliability of the prediction model is reflected by similar structures. The probability of the mapping control model error in the time delay compensation domain is then calculated using a fully factorized approximate distribution. As a variational decomposition hypothesis, the goal is to maximize the lower bound of evidence (ELBO) to approximate the true posterior. In each variational iteration, the E-step computes the expected sufficient statistics in parallel on ARM floating-point units using the lower bound of evidence formula in the form of Jensen's inequality. These statistics relate to the current q-distribution. The model performs matrix operations and trace operations, with eigenvalue decomposition and matrix inversion utilizing FPGA hardware-accelerated CORDIC and matrix multiplication units. The model's E-step output includes observation error residual statistics. Predictive residual statistics Statistics on control error Sufficient statistical data.

[0061] The subsequent M-step performs adaptive iterations on the linear and nonlinear subsets on the FPGA and ARM collaborative platform, respectively. For the subsets of observation and prediction parameters, the module calls the recursive least squares algorithm with a forgetting factor encapsulated in the FPGA DSP pipeline, utilizing the state transition template output from the E-step. Update the Jacobian matrix with statistics The forgetting factor is typically set to 0.995 to balance the influence of historical and recent data. Nonlinear subset updates employ the BFGS quasi-Newton algorithm, updating the control weights in the ARM floating-point unit based on the gradient vector calculated in the E-step and the Hessian approximation matrix. The step size is determined by Armijo line search, and the iteration continues until the gradient norm is below 1e–4 or the number of steps reaches 10.

[0062] To ensure closed-loop stability, calibration module 107 updates the... Calculate the Lyapunov function The positive definite matrix P is derived from the system energy function; if V˙ obtained through numerical difference is ≥ 0, the parameter projection unit is triggered. This unit executes a specially designed orthogonal projection operator in the FPGA hardware logic to... The process maps an unstable region to a stable manifold satisfying V˙<0. The projection process is accomplished using a pre-computed projection matrix or a penalty function method, and the entire process typically concludes within the range of 50 μs.

[0063] After several rounds of E-step and M-step iterations, the calibration module 107 generates new posterior mean and covariance. The parameter interfaces of each submodule load the updated Jacobian matrix from the FPGA registers or ARM shared memory into the observation module 103, write the QP optimized weights into the control module 104, and distribute the U-Net convolutional kernel weights to the fusion module 102. All updates take effect at the start of the next control cycle, thus enabling automatic online calibration without manual intervention in the event of sudden changes in ore particle size, network latency fluctuations, or injection deviations, achieving continuous optimal system operation.

[0064] In calibration module 107, a joint iterative process of recursive least squares (RLS) and quasi-Newton BFGS is adopted to continuously and adaptively update the observation and prediction model parameters. After each parameter update, stability projection is applied to ensure the convergence of the closed-loop system. The implementation is explained below with specific parameters and calculation steps.

[0065] The RLS iteration is first run continuously in an FPGA DSP pipeline or ARM floating-point unit with a forgetting factor λf. Let the input-output observation pair at time k be... Where h_k is the row vector of the spliced ​​observation and prediction design matrix (dimension n×1), y_k is the corresponding scalar observation (such as the injection trigger point or decoupling residual), and the variables are... Represents the vector of parameters to be estimated. Let P_k be the parametric covariance matrix. RLS updates the matrix by first calculating the gain vector with the current P_k and h_k. in ∈(0,1) is the forgetting factor, used to smooth out the influence of historical and new data; then the parameters are updated. Then through Corrected covariance. It should be noted that the term "design matrix" in this application differs from a general regression matrix; it refers only to the joint input row vector concatenated from the fusion and observation submodules. Forgetting factor. Examples can be set to 0.995–0.999, and the specific value can be determined according to the system's sensitivity requirements for new and old data. There is no limitation on this.

[0066] BFGS iteration for nonlinear subsets The update runs on an embedded ARM core or a soft-core CPU. Let the control parameters for the current iteration step n be... and corresponding gradient (where J represents the residual or loss function of the control model), and the Hessian approximation matrix B^n. First, determine the search direction. Then, the step size is found in the interval [0,1] using the Armijo line search mechanism. Make Achieve a sufficient descent; calculate new parameters. and order Finally press Update the Hessian approximation matrix. Convergence is typically achieved within five steps. It should be noted that the term "Armijo line search" in this application differs from simple fixed-step search; it refers only to an optimization strategy that adaptively determines the step size to guarantee descent conditions.

[0067] After the RLS and BFGS iterations are completed, calibration module 107 needs to verify whether the new parameters can maintain system stability. For this purpose, a Lyapunov function is defined. , where θ is the total parameter vector and P is a pre-selected symmetric positive definite matrix (which can be derived from decoupling mapping or the system energy function). The module calculates the derivative through numerical difference approximation in the ARM core or FPGA DSP. If V˙≥0, it indicates that parameter updates may lead to instability, and the calibration module 107 calls the parameter projection unit: this unit uses an orthogonal projection algorithm to... In quadratic manifolds The mapping is applied to the nearest neighbor point. This mapping can be achieved by solving least squares problems with equality constraints. And the closed-form solution or penalty function method is used to accelerate the completion in the FPGA hardware logic. Parameters Usually selected as This limits the update range. It should be noted that the term "parameter projection" in this application is different from truncation, and refers only to orthogonally mapping the parameters back to the manifold that satisfies the stability condition.

[0068] After projection is complete, the calibration module 107 will input the observation parameters with stability guaranteed. The Jacobian update interface, which outputs to the observation module 103, will predict the parameters. The output is sent to the QP weight interface of the control module 104, and the nonlinear parameters are... The data is pushed to the network latency model to ensure that the entire sorting control system maintains convergence and high-precision control when faced with disturbances in the raw ore and communication jitter.

[0069] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result according to substantially the same method falls within the scope of the present invention. Therefore, the scope of the present invention is defined only by the appended claims.

Claims

1. A machine vision-based intelligent control system for vanadium ore sorting; characterized in that: include: The acquisition module is used to periodically capture images of the flow state and injection points of vanadium ore particles on the conveyor belt, and output the original sequence of particle size and concentration with displacement information. The fusion module is used to construct a multimodal mineral condition model based on the original particle size and concentration sequence and the injection trajectory data of the execution module. It extracts the particle distribution and velocity states through online heterogeneous filtering to obtain the ore state vector and the set of injection trigger points. The observation module is used to adaptively identify the coupling degree of the three intervals of grinding, swirling and classification based on the ore state vector and the set of injection trigger points using the online augmented Kalman observation method, obtain the dynamic mapping of the decoupled subsystem, and output the decoupled incremental mapping and coupling residual field. The control module is used to optimize the actuator incremental command by sliding time domain rolling based on the decoupled incremental mapping and residual field, and output the injection timing correction amount and nozzle dynamic displacement command. The positioning module is used to reconstruct the three-dimensional mesh on the surface of the ore separator through multi-view structured light phase decoding and baseline self-calibration algorithm, and map it to the ore particle ejection coordinate system, and output the attitude error vector. The execution module is used to calculate the injection trigger time and pose correction amount based on the output of the control module and the positioning module by using a rolling time-domain window scheduling algorithm to plan the spatiotemporal trajectory and execute it, while simultaneously sending the execution feedback back to the calibration module and the fusion module in real time. The calibration module is used to update the observation and prediction model parameters based on the set of injection trigger points, decoupled incremental mapping and attitude error vector using an adaptive identification algorithm, and feeds back the updated parameters to the fusion module, observation module and control module.

2. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The acquisition module's acquisition equipment includes a mill pressure transmitter, a hydrocyclone conductivity sensor, a classifier accelerometer sensor, and multiple high-speed camera raw frames. The module operates as follows: based on IEEE 1588 clock synchronization and external hard triggering, each acquired data stream is labeled with a time domain tag and written to a ring FIFO. Zero-copy data transfer is implemented using a PCIe Gen3 x8 DMA chain descriptor. During DMA writing, a hardware reordering mechanism performs frame-level alignment and timing rearrangement of the scalar stream and frame stream based on the time domain tag. The rearranged data stream undergoes band-limited sampling of the process signal by a programmable gain amplifier and a sixth-order anti-aliasing filter. Simultaneously, the image path triggers ROI hardware clipping and color space conversion. Finally, the sorted, filtered, and clipped process timing data, along with the corresponding particle displacement vectors in the camera's visual frames, are encapsulated into a time-synchronized data packet and transmitted to the fusion module based on a TSN Qbv time window and QoS priority mapping.

3. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The multimodal mineral condition model obtains smoothed state estimates of process values ​​through extended Kalman filtering; it extracts texture and edge features from image frames using a multi-scale convolutional neural network pyramid; the two types of features are decomposed into a multimodal mineral condition tensor through high-order tensor decomposition, and then sparse discriminant projection is used for dimensionality reduction to output an ore state vector containing granularity tomography, concentration gradient, and flow velocity vector, as well as an index set of injection trigger points.

4. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The online augmented Kalman observation method first constructs the augmented state vector in the observation module. ,in Composed of a multimodal ore state vector and a set of injection trigger points, representing time... The ore increment state vector, The incremental mapping parameters are represented by the three intervals of grinding, cyclone, and classification; then, the discrete nonlinear mapping function is used. With measurement model Define the state transition and observation equations, where To control the input increment vector, To account for process noise, the following procedure is then executed: At any moment In the prediction phase, a first-order Taylor expansion is used in [ Linearization at [point] yields the state transition Jacobian matrix. And combined with process noise covariance The updated augmented covariance is determined by the following formula: （1） In formula (1), To predict the covariance matrix, For a moment The posterior covariance matrix, The process noise covariance matrix; express The transpose of the matrix is ​​then used; subsequently, the prediction state vector is generated on a hardware-accelerated platform via efficient sparse matrix multiplication. With covariance ; During the update phase, the Jacobian matrix is ​​used. Construct the measurement matrix in real time and calculate the Kalman gain. The calculation formula is: （2） In formula (2), To observe the noise covariance matrix, matrix The transpose matrix is ​​used to represent the predicted covariance. Projected onto the observation space; by Comparison with actual measurements With predictive measurement The residual completes the augmented state correction, expressed as: （3） The updated state vector is split into decoupled incremental mapping parameters. With subsystem state And calculate the coupled residual field. .

5. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The control module decouples the incremental mapping parameters and residual field to construct the state evolution equation using the Discrete Langer and Kutta delay embedding methods, and obtains the future state sequence using the fourth-order Langer-Kutta algorithm. Within each control cycle, the control module loads the most recently sampled delay histogram using a network delay probability distribution model, generates a delay correction sequence using Monte Carlo sampling, and incorporates this delay correction sequence into the delay embedding state vector. Next, the control module constructs a quadratic programming problem with linear inequality constraints and residual compensation terms, integrating the predicted state, upper and lower bounds of the control increment, and the delay correction matrix into the objective function. It then calls an FPGA-accelerated QP solver based on the interior-point method to perform iterative calculations, obtaining the optimal control increment sequence in the rolling time domain. Finally, the control module encapsulates the first-step value of the optimal increment and the corresponding delay correction amount into injection timing correction amounts and nozzle dynamic displacement commands based on FIFO timing, and sends them to the scheduling execution module.

6. The intelligent control system for vanadium ore sorting based on machine vision according to claim 5, characterized in that: The process of generating and incorporating the delay correction sequence into the delay embedding state vector using the network delay probability distribution model includes: Obtain Profinet end-to-end latency samples and construct an empirical histogram; The empirical histogram is resampled at equal intervals to obtain resampled histogram data; Based on the resampled histogram data, the EM algorithm is executed in parallel to estimate the parameters of the Gaussian mixture model, and the mixture weights and covariance matrix are obtained. During the online phase, the mixture weights and covariance matrix are updated using a recursive Bayesian method to generate a real-time Gaussian mixture distribution. Based on the real-time Gaussian mixture distribution, a delay correction sequence is generated and output through inverse transformation sampling.

7. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The working principle of the multi-eye structured light phase decoding and baseline self-calibration algorithm includes: Acquire coded stripe and Gray code images simultaneously captured by multiple cameras; The coded stripe image is subjected to a three-frequency phase expansion method to generate a continuous phase map, and Gray code decoding is combined to eliminate phase ambiguity and output a deblurred phase map. Based on the deblurred phase mapping, the initial point cloud is extracted. Baseline self-calibration is performed on the initial point cloud and Gray code plane features. The camera intrinsic and extrinsic parameters are iteratively corrected using the tensor bundle optimization method to obtain the accurate intrinsic and extrinsic parameter matrices. The fuzzy phase map back projection is used to construct a three-dimensional mesh on the surface of the ore separator; By rigidly transforming the three-dimensional mesh on the surface of the separator to the particle injection coordinate system and aligning it with the theoretical geometric model, a six-degree-of-freedom attitude error vector containing translation offset and rotation difference is output.

8. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The working method of the rolling time-domain window scheduling algorithm includes: Obtain the optimal control increment sequence and attitude error vector, and fill a sliding buffer of length N in chronological order; The buffer data is resampled at equal intervals to generate a time node sequence; Construct continuous displacement and time-corrected trajectory based on the time node sequence using third-order B-spline interpolation; A linear programming piecewise solver is applied to continuous trajectory slices to calculate the pneumatic valve pulse width and servo interpolation command; When the sliding window moves to the right, the earliest node is replaced and a local QP re-optimization is triggered. The updated trigger time and pose correction instructions are output.

9. The intelligent control system for vanadium ore sorting based on machine vision according to claim 1, characterized in that: The update feedback process of the adaptive identification algorithm includes: Receive jet trigger point set Decoupling incremental mapping and attitude error vector A parametric posterior probability model is constructed using a variational Bayesian expectation-maximization framework, and the joint likelihood function is calculated using the following formula: （4） In formula (4), Let be the probability density function. Let be the parameter vector to be identified, where For the observation module parameters, For prediction model parameters, The parameters are for the control module; the variational Bayesian expectation-maximization framework executes a cyclic iteration from u1 to u3: u1. Approximating the lower bound of evidence using Jessen's inequality. To obtain the expected sufficient statistic, the formula is: （5） In formula (5), Denotes the variational posterior distribution. Indicates the prior distribution of the parameters. The search range for KL divergence constraint parameters; u2, Decompose the parameter space into a linear subset NAND nonlinear subsets ,in: The update uses a recursive least squares method with a forgetting factor, and the input is a decoupled increment. Update the state transition matrix of the observation module; The update is based on the BFGS quasi-Newton method iteration, utilizing the attitude error vector. Jacobian determinant Correct the QP weight matrix Q of the control module; u3, if the Lyapunov function If the derivative is greater than or equal to zero, the parameter projection algorithm will be triggered. Constrained to a stable manifold; the Lyapunov function In It is a positive definite symmetric matrix. for The transpose of ; Finally, the updated Jacobian matrix of the observation module, the convolution kernel weights of the fusion module, and the coefficients of the QP objective function of the control module are output and fed back to the corresponding module parameter interface.

Images