Real-time detection method for particle motion trail of edge equipment dry magnetic separator

Abstract

The invention relates to the technical field of image processing, and discloses a real-time particle motion trail detection method for a dry magnetic separator of edge equipment. Comprising the following steps: constructing an edge depth cavity convolution block based on a depth separable convolution block and an edge detection block, constructing an up-sampling block based on the edge depth cavity convolution block, constructing a down-sampling block based on the edge depth cavity convolution block, constructing a feature extraction trunk according to depth separable convolution and the edge depth cavity convolution block, and extracting the feature extraction trunk according to the edge depth cavity convolution block. Constructing a first output channel by adopting the up-sampling block and the down-sampling block, and constructing a detection model based on the feature extraction trunk and the first output channel; and acquiring a picture of the to-be-detected dry magnetic separator, inputting the picture into the detection model, acquiring a predicted particle motion trail of the dry magnetic separator output by the detection model, and adjusting an ore separation plate of the dry magnetic separator based on the predicted particle motion trail of the dry magnetic separator. The problem that an existing dry magnetic separator motion trail detection model cannot effectively conduct trail detection is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology and discloses a method for detecting particle motion trajectory in a real-time edge device dry magnetic separator. Background Technology

[0002] Dry magnetic separation technology has significant application value in the field of mineral sorting, especially in water-free environments or scenarios requiring water-saving operations. With increasing resource scarcity and environmental protection requirements, dry magnetic separators, due to their water-free nature, are gradually becoming one of the preferred methods for mineral sorting. However, the performance of dry magnetic separators relies on precise control and monitoring, particularly the real-time identification and tracking of particle motion trajectories. How to efficiently and accurately monitor the operating status of dry magnetic separators and capture particle motion trajectories has become a key technical issue for improving the operating efficiency of magnetic separation equipment, reducing energy consumption, and ensuring production continuity.

[0003] Existing methods for detecting the motion trajectory of dry magnetic separators largely rely on traditional deep learning image processing algorithms and deep learning-based image segmentation models. Traditional image processing methods typically depend on feature extraction and matching algorithms, such as edge detection and optical flow methods. While these methods can provide effective detection results in certain specific scenarios, they are difficult to operate stably in complex backgrounds, dynamic motion, and high-frequency changing environments, and they also place high demands on the equipment, failing to meet the needs of efficient real-time monitoring.

[0004] In deep learning-based solutions, existing technologies such as "A Real-Time Dry Magnetic Separation Particle Sorting Method Based on UNet" (patent application number: 2023117233741) propose using the UNet structure for real-time sorting of magnetically separated particles. UNet, with its powerful feature extraction and image segmentation capabilities, can achieve relatively accurate particle identification and sorting. This method is particularly suitable for servers or industrial hosts with high computing power. However, due to the large number of parameters and high computational complexity, structures like UNet are difficult to deploy on edge devices with limited computing power, making them unsuitable for low-power, low-latency, and resource-constrained field applications. Furthermore, traditional convolutional neural network (CNN) models like UNet have limited ability to recognize dynamic motion trajectories, making it difficult to capture the rapid changes and motion characteristics of particles during the magnetic separation process. Summary of the Invention

[0005] This invention provides a method for detecting particle motion trajectory in a real-time edge device dry magnetic separator, which solves the problem that existing dry magnetic separator motion trajectory detection models cannot effectively detect trajectories.

[0006] To achieve the above objectives, the present invention employs the following technical solution: This invention provides a method for detecting particle motion trajectory in a real-time edge device dry magnetic separator, comprising the following steps: Step 1: Construct edge-dilated convolutional blocks based on a progressive fusion method combining depthwise separable convolutional blocks and edge detection blocks. Construct upsampling blocks with dual inputs and single outputs for feature enhancement based on weighted fusion of edge-dilated convolutional blocks. Construct downsampling blocks with dual inputs and single outputs for feature compression fusion based on edge-dilated convolutional blocks combined with multiple convolution processing. Construct a feature extraction backbone based on depthwise separable convolutional blocks and edge-dilated convolutional blocks. Construct the first output channel by combining multi-scale sampling and progressive fusion with upsampling blocks and downsampling blocks at different layers of the feature extraction backbone. Construct a detection model based on the feature extraction backbone and the first output channel. Step 2: Obtain the working video of the dry magnetic separator to be tested, cut it into images in chronological order, input it into the detection model, obtain the predicted particle movement trajectory of the dry magnetic separator output by the detection model, and adjust the ore separation plate of the dry magnetic separator based on the predicted particle movement trajectory of the dry magnetic separator. Adjusting the sorting plate of a dry magnetic separator based on the predicted particle motion trajectory involves: analyzing the motion images of particles under different magnetic fields, detecting the spatial distribution characteristics of the particle group, and then generating optimized control commands for the sorting area in real time to guide the dynamic adjustment of the sorting plate position. This makes the landing areas of strongly magnetic particles and weakly magnetic particles more clearly separated, thereby overcoming the industrial problem of blurred sorting boundaries and reduced beneficiation accuracy caused by particle trajectory fluctuations in traditional fixed sorting plates. This effectively improves the sorting accuracy and operational stability of the dry magnetic separator. The generation of optimized control commands based on the predicted particle motion trajectory is existing technology and will not be elaborated here. Through the above design, employing depthwise separable convolution and edge depthwise separable dilated convolution techniques, the computational load and model parameters are significantly reduced. This enables the model to run efficiently on resource-constrained edge devices, avoiding the problems of high computational complexity and large memory requirements of traditional deep learning models. It is particularly suitable for deployment in hardware environments such as embedded systems, industrial camera modules, and the RK3588 platform in industrial settings.

[0007] Furthermore, the edge depth-dipping convolutional block divides the input features into three parts, performs convolution processing on the first part and the second part of the features to different degrees, performs edge detection processing on the third part of the features to obtain edge features, and gradually fuses the edge features with the features of the first part and the second part to obtain output features.

[0008] Furthermore, the method of constructing edge-depth-hole convolutional blocks based on the gradual fusion of depth-separable convolutional blocks and edge detection blocks includes: constructing a three-channel parallel structure based on depth-separable convolutional blocks and edge detection blocks with different convolutional kernels; constructing a dual-channel parallel structure based on splicing layers and convolutional layers; and constructing a second output channel based on a first summing layer and a convolutional layer. Based on the three-channel parallel structure, the two-channel parallel structure, and the second output channel, a network structure is constructed by combining block layers, with the block layers, the three-channel parallel structure, the two-channel parallel structure, and the second output channel connected in sequence as edge depth-dilated convolutional blocks.

[0009] Furthermore, the three-channel parallel structure includes a first channel, a second channel, and a third channel; The first channel sequentially includes a depth-separable hole convolution with a kernel size of 5 and a hole stride of 3, and a 1*1 first convolutional layer; The second channel includes an edge detection block, a 1*1 second convolutional layer, and a 1*1 third convolutional layer. The second and third convolutional layers are in parallel and are both connected to the edge detection block. The third channel sequentially includes a depth-separable hole convolution with a kernel size of 7 and a hole stride of 3, and a 1*1 fourth convolutional layer. The first, second, third, and fourth convolutional layers are all connected to the dual-channel parallel structure.

[0010] Furthermore, the dual-channel parallel structure includes a fifth channel and a sixth channel, wherein the fifth channel is connected to the first convolutional layer and the second convolutional layer, and the sixth channel is connected to the third convolutional layer and the fourth convolutional layer; The fifth channel includes, in sequence, a first splicing layer and a 1*1 fifth convolutional layer; The sixth channel consists of a second splicing layer and a 1*1 sixth convolutional layer.

[0011] Furthermore, the second output channel sequentially includes a first additive layer and a 1*1 seventh convolutional layer; Both the fifth and sixth convolutional layers are connected to the first additive layer.

[0012] Furthermore, the construction of the feature extraction backbone based on depth-separable convolution and edge-diffused convolution blocks includes: constructing four backbone units, a first backbone unit, a second backbone unit, a third backbone unit, and a fourth backbone unit with the same network structure based on a network structure of depth-separable convolutional layers with a kernel size of 3 and a stride of 2 and edge-diffused convolutional blocks. The first to fourth backbone units are connected in sequence, and a depthwise separable convolutional layer with a kernel size of 3 and a stride of 1 is connected to the input of the first backbone unit to complete the construction of the feature extraction backbone.

[0013] Furthermore, the first output channel is constructed as follows: at the edge depth-dilated convolutional blocks of the feature extraction backbone from the first backbone unit to the fourth backbone unit, a depth-separable convolutional layer with a kernel size of 3 and a stride of 1 is connected. Upsampling blocks are set between each pair of depth-separable convolutional layers connected to the fourth backbone unit and the first backbone unit in order from bottom to top for feature enhancement. Downsampling blocks are set between each pair of upsampling blocks in order from top to bottom for feature compression and fusion. A fully connected layer is connected at the output end of the last downsampling block to complete the construction of the first output channel.

[0014] Furthermore, the upsampling block includes a seventh channel and an eighth channel; The seventh channel includes a 1*1 eighth convolutional layer, a transposed convolutional layer, a second additive layer, and an edge-depth-hole convolutional block connected in sequence. The eighth channel includes a 1*1 ninth convolutional layer, which is connected to the second additive layer.

[0015] Furthermore, the downsampling block includes a ninth channel and a tenth channel; The ninth channel includes a 1*1 tenth convolutional layer, a depth-separable convolutional layer, a third additive layer, and an edge-depth-hole convolutional block connected in sequence. The tenth channel includes a 1*1 eleventh convolutional layer, which is connected to the third additive layer.

[0016] Through the above design, multi-scale feature fusion and edge enhancement mechanisms can effectively capture the dynamic motion trajectory of particles in a dry magnetic separator. Using edge-depth separable dilated convolution technology, the model can more sensitively capture changes in particle edges in dynamic environments, thereby improving the accuracy of motion trajectory recognition, especially in complex scenarios such as particle overlap and motion blur, maintaining high detection performance.

[0017] Furthermore, the images input to the detection model are annotated using an image annotation tool; The annotation process includes: annotating the strong magnetic regions and weak magnetic regions in the image.

[0018] Furthermore, the detection model constructs a loss function based on classification loss, localization loss, and edge loss; The classification loss is constructed based on the error of the detection model in classifying the strong magnetic region and weak magnetic region of the dry magnetic separator in the image, and the classification loss is calculated by sampling the cross-entropy loss function; The localization loss is constructed based on the coordinate difference between the ground truth bounding box constructed by the detection model based on the particle motion trajectory of the dry magnetic separator in the image and the predicted bounding box constructed by the detection model based on the particle motion trajectory. The localization loss is calculated using the smoothed L1 loss function. The edge loss is constructed based on the accuracy of the detection model in optimizing the boundary of the predicted bounding box region, and the edge loss is calculated using the Dice coefficient loss function.

[0019] Furthermore, the loss function is calculated using the following formula: ; in, Represents the loss function; Indicates classification loss; The weighting coefficients represent the classification loss. Indicates positioning loss; The weighting coefficients representing the localization loss; Indicates marginal loss; The weighting coefficients represent the edge loss. The classification loss is calculated using the following formula: ; in, Indicates the first The category of each region, if it is a strong magnetic region, then The value is 1; if it is a weak magnetic region, then... =0; The first prediction made by the detection model represents the... The probability that a region belongs to a strong magnetic region; The positioning loss is calculated using the following formula: ; in, Represents the smoothing loss function; Indicates the center coordinates of the prediction box. Indicates the width and height of the prediction box; Represents the center coordinates of the true bounding box. Represents the width and height of the actual bounding box; The edge loss is calculated using the following formula: ; in, This represents the set of pixels that indicate a strong or weak magnetic region predicted by the detection model. A set of pixels representing a labeled strong or weak magnetic region; This represents the number of pixels that intersect the predicted region of the detection model with the labeled ground truth region; This represents the number of pixels in the predicted region as predicted by the detection model. This represents the number of pixels in the actual labeled area.

[0020] Furthermore, the detection model is converted from a PyTorch model to the ONNX format, and then from the ONNX format to the RKNN3588 format before being deployed to the hardware platform.

[0021] The above design achieves extremely low computational complexity and high inference speed, meeting the real-time processing requirements of high-frequency image streams in dry magnetic separators. Compared to traditional target detection methods, this invention's model offers a faster response speed, real-time monitoring of particle trajectories, and ensures delay-free detection in rapidly changing environments, adapting to the stringent real-time requirements of industrial settings. While maintaining high accuracy and real-time performance, it effectively reduces the consumption of computing resources and has low power consumption. This makes the model very suitable for long-term operation on edge devices, meeting the industrial field's requirements for low power consumption and stable operation over extended periods.

[0022] Beneficial effects: This invention provides a real-time edge device dry magnetic separator motion trajectory detection method. Combining depthwise separable convolution and dilated convolution, this invention can extract feature information of particle motion at multiple scales, which not only improves the detection accuracy of small-scale particle motion, but also enhances the perception ability of larger-scale motion, and improves the adaptability to complex motion trajectories of dry magnetic separators. It emphasizes robustness in complex environments. By enhancing the capture of edge information through edge-depth separable dilated convolution, this invention can maintain high detection accuracy and stability even under conditions of changing lighting, background interference, and blurred particle motion trajectories, adapting to the complex changes in industrial environments. It is not only suitable for edge devices with limited computing resources, but also easily adaptable to different hardware platforms (such as industrial cameras, embedded terminals, etc.). The efficient deployment of the model on multiple platforms ensures its broad application prospects and can provide reliable support for intelligent manufacturing and industrial automation in different scenarios. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the network structure of the detection model in an embodiment of the present invention; Figure 2 This is a schematic diagram of the network structure of the edge-depth-hole convolutional block according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the network structure of the upsampling block in an embodiment of the present invention; Figure 4 This is a schematic diagram of the network structure of the downsampling block in an embodiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0026] Please see Figure 1 This embodiment provides a method for detecting the motion trajectory of a real-time edge device dry magnetic separator, which includes the following steps: Step 1: Construct edge-dilated convolutional blocks based on a progressive fusion method combining depthwise separable convolutional blocks and edge detection blocks. Construct upsampling blocks with dual inputs and single outputs for feature enhancement based on weighted fusion of edge-dilated convolutional blocks. Construct downsampling blocks with dual inputs and single outputs for feature compression fusion based on edge-dilated convolutional blocks combined with multiple convolution processing. Construct a feature extraction backbone based on depthwise separable convolutional blocks and edge-dilated convolutional blocks. Construct the first output channel by combining multi-scale sampling and progressive fusion with upsampling blocks and downsampling blocks at different layers of the feature extraction backbone. Construct a detection model based on the feature extraction backbone and the first output channel. Regarding the construction of Edge Depthwise-Dilated Convolution (EdgeDDConv), to adapt to the computational limitations of edge devices, efficient depthwise separable convolution is used for feature extraction to reduce computational and storage requirements. For the specific construction process, please refer to [link to documentation]. Figure 2 This includes: constructing a three-channel parallel structure based on depth-separable convolutional blocks and edge detection blocks with different convolutional kernels; constructing a dual-channel parallel structure based on splicing layers and convolutional layers; and constructing a second output channel based on the first summing layer and convolutional layers. Based on the three-channel parallel structure, the two-channel parallel structure, and the second output channel, a network structure is constructed by combining block layers, with the block layers, the three-channel parallel structure, the two-channel parallel structure, and the second output channel connected in sequence as edge depth-dipping convolutional blocks. Specifically, the three-channel parallel structure includes a first channel, a second channel, and a third channel; The first channel consists of a depth-separable hole convolution with a kernel size of 5 and a hole stride of 3, and a 1*1 first convolutional layer. The second channel includes an edge detection block, a 1*1 second convolutional layer, and a 1*1 third convolutional layer. The second and third convolutional layers are in parallel and are both connected to the edge detection block. The third channel consists of a depth-separable convolutional layer with a kernel size of 7 and a hole stride of 3, and a 1*1 fourth convolutional layer. The first, second, third, and fourth convolutional layers are all connected to the dual-channel parallel structure.

[0027] More specifically, the dual-channel parallel structure includes a fifth channel and a sixth channel. The fifth channel is connected to the first and second convolutional layers, and the sixth channel is connected to the third and fourth convolutional layers. The fifth channel consists of the first splicing layer and a 1*1 fifth convolutional layer. The sixth channel consists of the second splicing layer and a 1*1 sixth convolutional layer.

[0028] The second output channel consists of the first additive layer and the 1*1 seventh convolutional layer in sequence; Both the fifth and sixth convolutional layers are connected to the first additive layer; When processing input features, the edge-dilated convolutional block divides the input features into three parts. The first and second parts undergo convolution processing at different degrees, while the third part is processed for edge detection to obtain edge features. These edge features are then progressively fused with the features from the first and second parts to obtain the output features. The first part consists of the image features input to the first channel, the second part to the third channel, and the third part to the second channel. In the first channel, local features are primarily extracted; in the second channel, complex features are primarily extracted; and in the third channel, edge features are primarily extracted. These features are then progressively fused with the second output channel through a dual-channel parallel structure. This process, by dividing the input image into blocks and processing each block differently, effectively extracts key features from the image, especially when processing large-scale data, thus efficiently utilizing computational resources. Please see Figure 3The Upscale Fusion Block (UFBlock) aims to enhance feature information at different scales by upsampling, convolving, and fusing input features. This module is particularly suitable for scenarios requiring multi-scale information fusion, and can efficiently merge input features of different resolutions to improve the model's ability to represent complex information. Its main function is to perform feature fusion at multiple scales, enabling low-resolution and high-resolution features to complement each other and improve the accuracy and information content of the overall feature representation. This operation involves performing 1×1 convolution, upsampling, and weighted fusion on the two input features, and finally outputting the fused features through an edge-dip convolutional block (EdgeDDConv). For the specific network structure of the upsampling block, the upsampling block includes the seventh channel and the eighth channel; The seventh channel consists of a 1*1 eighth convolutional layer, a transposed convolutional layer, a second additive layer, and an edge-depth-hole convolutional block. The eighth channel includes a 1*1 ninth convolutional layer, which is connected to the second additive layer; When processing two input features in the upsampling block, the first step is to match the feature dimensions of the two input features by transforming the dimensions of the eighth and ninth convolutional layers. Then, an upsampling operation is performed through the transposed convolutional layer, so that the low-resolution input features (feature map) have the same spatial dimension as the high-resolution input features, thus better realizing feature fusion. Finally, the expressive power of the output features is enhanced by the edge depth dilated convolutional block.

[0029] Please see Figure 4 The core objective of the DownScale Compression Block (DCBlock) is to reduce computational complexity by decreasing spatial resolution while preserving key information to ensure effective feature representation. This module is suitable for feature map compression operations, and effectively extracts edge information from features through EdgeDDConv (Edge Depth Diffusion Convolutional Block), enhancing the model's ability to perceive details such as motion trajectories. By performing multiple convolutions on the input feature map, feature fusion, and finally edge-depth separable dilated convolution operations, the representation of the feature map is optimized and adapted to the limited computing resources on edge devices. For the specific network structure of the downsampling block, the downsampling block includes the ninth channel and the tenth channel; The ninth channel consists of a 1*1 tenth convolutional layer, a depth-separable convolutional layer, a third additive layer, and an edge-depth-hole convolutional block. The tenth channel includes a 1*1 eleventh convolutional layer, which is connected to the third additive layer; When processing two input features in the downsampling block, the feature dimensions are first matched through the tenth and eleventh convolutional layers, and then through the depth convolution and pointwise convolution of the depth separable convolutional layer to reduce computational complexity. Local features are extracted from the input features after the tenth convolutional layer, and after fusion through the third addition layer, the details are captured through the edge depth-dilated convolutional block to enhance the expression of edge features.

[0030] Please see Figure 1 For constructing the feature extraction backbone, multiple backbone units are used to achieve more edge information at different scales, that is, the extraction of edge features at different scales, which reduces computational complexity and improves the sensitivity of particle motion trajectory. For the network structure for building the feature extraction backbone, it specifically includes: constructing four backbone units, namely the first backbone unit, the second backbone unit, the third backbone unit, and the fourth backbone unit, based on a network structure with a depthwise separable convolutional layer with a kernel size of 3 and a stride of 2 and an edge-diffused convolutional block. The first to fourth backbone units are connected in sequence, and a depthwise separable convolutional layer with a kernel size of 3 and a stride of 1 is connected to the input of the first backbone unit to construct the feature extraction backbone.

[0031] The construction of the first output channel based on the network structure of the feature extraction backbone includes: connecting depth-separable convolutional layers with kernel size of 3 and stride size of 1 to the edge depth-dipping convolutional blocks of the first to fourth backbone units of the feature extraction backbone; setting up sampling blocks between each pair of depth-separable convolutional layers connected to the fourth backbone unit and the first backbone unit in order from bottom to top for feature enhancement; setting down sampling blocks between each pair of up sampling blocks in order from top to bottom for feature compression and fusion; and connecting a fully connected layer at the output end of the last down sampling block to complete the construction of the first output channel. In the network structure of the feature extraction backbone and the first output channel, the input features are sampled and fused at different scales. The upsampling block preserves information at different scales and effectively fuses features at different scales, improving the model's real-time response capability on edge devices. The downsampling block compresses the features, reduces the amount of computation, and retains important motion trajectory information, ultimately realizing the prediction of particle motion trajectory in the magnetic dry separator.

[0032] Finally, a detection model is constructed based on the feature extraction backbone and the first output channel. The core design idea of ​​the detection model is to reduce the amount of computation and improve the detection accuracy of the dry magnetic separator's motion trajectory on the edge device through efficient feature extraction and fusion methods. The efficient feature extraction and fusion methods are implemented through the network structure of the feature extraction backbone, upsampling block, downsampling block and the first output channel. This reduces redundant computation and improves the computational efficiency, enabling the model to be efficiently deployed and run on the edge device to detect the particle motion trajectory of the dry magnetic separator in real time.

[0033] The images input to the detection model are annotated using an image annotation tool. The annotation process includes: annotating the strong magnetic regions and weak magnetic regions in the image; Strong magnetic areas: These are areas marked in the image that are strongly affected by magnetic forces. These areas are usually near the magnetic separator or parts affected by a strong magnetic field. Strong magnetic areas correspond to the effective working area of ​​a dry magnetic separator and are typically represented as high-contrast areas in the image.

[0034] Weak magnetic areas: These are areas less affected by magnetic forces. These areas are often far from the core components of the magnetic separator or areas with weak magnetic fields. In the image, weak magnetic areas may appear as areas with low contrast or relatively uniform areas.

[0035] The loss function of the detection model is constructed based on classification loss, localization loss, and edge loss; The classification loss is constructed based on the error of the detection model in classifying the strong and weak magnetic regions of the dry magnetic separator in the image. The classification loss is calculated using the cross-entropy loss function, which can effectively evaluate the difference between the predicted category distribution and the true label, and is expressed by the following formula: ; in, Indicates the first The category of each region, if it is a strong magnetic region, then The value is 1; if it is a weak magnetic region, then... =0; The first prediction made by the detection model represents the... The probability that a region belongs to a strong magnetic region; The localization loss is constructed based on the coordinate difference between the ground truth bounding boxes constructed by the detection model based on the particle motion trajectory of the dry magnetic separator in the image and the predicted bounding boxes constructed by the detection model based on the predicted particle motion trajectory. The localization loss is calculated using the smoothed L1 loss function. The smoothed L1 loss can effectively reduce the influence of outliers, making the loss more stable, and is expressed by the following formula: ; in, Represents the smoothing loss function; Indicates the center coordinates of the prediction box. Indicates the width and height of the prediction box; Represents the center coordinates of the true bounding box. Represents the width and height of the actual bounding box; The smoothed L1 loss provides relatively stable optimization results in regression tasks, especially for large errors, and has better robustness compared to the traditional L2 loss. It is defined as follows: ; in, This represents the error between the predicted value and the actual value; Edge loss is constructed based on the accuracy of the detection model's optimization of the predicted bounding box boundaries. It is calculated using the Dice coefficient loss function, which helps the model improve the accuracy of region boundaries. This is expressed by the following formula: ; in, This represents the set of pixels that indicate a strong or weak magnetic region predicted by the detection model. A set of pixels representing a labeled strong or weak magnetic region; This represents the number of pixels that intersect the predicted region of the detection model with the labeled ground truth region; This represents the number of pixels in the predicted region as predicted by the detection model. This represents the number of pixels in the actual labeled area.

[0036] The final loss function is expressed as follows: ; in, Represents the loss function; Indicates classification loss; The weighting coefficients represent the classification loss. Indicates positioning loss; The weighting coefficients representing the localization loss; Indicates marginal loss; The weighting coefficients represent the edge loss. In this embodiment, the weight coefficient of the classification loss is set to 1.0, the weight coefficient of the localization loss is set to 5.0 to ensure accurate localization, and the weight coefficient of the edge loss is set to 0.1 to assist in boundary refinement without affecting the main task.

[0037] The proposed loss function, by combining classification loss, localization loss, and edge loss, can simultaneously optimize the model's classification and localization accuracy, as well as boundary refinement capabilities in both strong and weak magnetic regions. Through reasonable weight adjustment, the contributions of each loss function can be balanced during training, thereby achieving efficient and accurate motion trajectory detection.

[0038] During the training of the detection model, the Adam optimizer was used to optimize the network weights. The initial learning rate was set to 0.001, and a learning rate decay strategy was adopted. After a certain number of iterations, the learning rate was reduced proportionally to achieve more stable convergence. The momentum decay coefficients of the optimizer were set to 0.9 and 0.999 to help accelerate convergence and improve training stability. During model training, the batch size was set to 32, and the training epochs were 50. Through the adjustment of these hyperparameters, the model can efficiently learn from the data and gradually optimize its performance. The training environment was based on the Windows 11 operating system and used an NVIDIA RTX 4090 graphics card for computation, which significantly improved computational efficiency. The software environment included Python 3.9, PyTorch 2.1, and CUDA 12.1, which fully utilized GPU acceleration to improve the speed and stability of model training.

[0039] Finally, during model deployment, the ONNX format model is converted to RKNN3588 format for deployment on specific hardware platforms. RKNN (Rockchip Neural Network) provides a deep learning inference framework that supports efficient neural network inference. Using the RKNN Toolkit, ONNX model files can be converted to a model format suitable for the RK3588 platform.

[0040] During the conversion process, the RKNN Toolkit compiles and optimizes the model to meet the inference requirements of the RK3588 hardware. The specific conversion process includes loading the ONNX model, performing necessary configurations, optimizing the model (such as selecting a quantization method), and finally compiling to generate the RKNN model file. The converted RKNN model file (usually in .rknn format) can be directly used for inference on the RK3588 platform.

[0041] Through the above steps, this study successfully achieved the conversion from a trained PyTorch model to a model adapted for the RK3588 platform. First, the trained PyTorch model was saved as a .pt file. Then, cross-platform conversion was achieved using the ONNX format. Finally, the ONNX model was converted to the RKNN format, which is supported by the RK3588 hardware, using the RKNN Toolkit. This conversion process ensures that the model can be efficiently deployed on different hardware platforms and utilizes the hardware acceleration of the RK3588 for fast inference.

[0042] Step 2: Obtain the working video of the dry magnetic separator to be tested, cut it into images in chronological order, input it into the detection model, obtain the predicted particle movement trajectory of the dry magnetic separator output by the detection model, and adjust the ore separation plate of the dry magnetic separator based on the predicted particle movement trajectory of the dry magnetic separator. Adjusting the sorting plate of a dry magnetic separator based on the predicted particle motion trajectory involves analyzing the motion images of particles under different magnetic fields, detecting the spatial distribution characteristics of the particle group, and then generating optimized control commands for the sorting area in real time to guide the dynamic adjustment of the sorting plate position. This makes the landing areas of strongly magnetic particles and weakly magnetic particles more clearly separated, thereby overcoming the industrial problem of blurred sorting boundaries and reduced mineral processing accuracy caused by particle trajectory fluctuations in traditional fixed sorting plates. This effectively improves the sorting accuracy and operational stability of the dry magnetic separator.

[0043] Finally, the motion trajectory detection method for a real-time edge device dry magnetic separator proposed in this invention was compared with several mainstream object detection models, including U-Net, Mask R-CNN, YOLOv5, and YOLOv8. These models have demonstrated excellent performance in different application scenarios, especially in semantic segmentation and object detection tasks. Therefore, by comparing with these models, we can more comprehensively evaluate the advantages and disadvantages of our proposed model in magnetic separator motion trajectory detection. Specific comparison results are shown in Table 1. Table 1: Comparison results of the model and corresponding method of this invention with mainstream target detection models:

[0044] In this study, all models were tested on the RK3588 hardware platform and optimized using INT8 quantization to improve inference efficiency and reduce computational resource consumption. Experimental results show that the detection model proposed in this study achieves 98.35% mAP@0.5, significantly outperforming other comparative models such as YOLOv8 (96.12%) and YOLOv5 (93.25%), demonstrating higher detection accuracy. Furthermore, the detection model of this invention achieves an inference speed of 75 FPS, demonstrating excellent real-time performance, far exceeding Mask R-CNN (2 FPS) and U-Net (3 FPS). These models are limited in their application in real-time detection tasks due to their slow inference speed. In contrast, although YOLOv5 and YOLOv8 have an advantage in inference speed, their accuracy is still inferior to the detection model of this invention. Through INT8 quantization, the computational efficiency and memory usage of all models are significantly optimized, enabling the detection model of this invention to be efficiently deployed on edge devices and achieve real-time object detection tasks. Therefore, combining high accuracy and fast inference speed, the detection model of this invention performs excellently on the RK3588 platform, and is especially suitable for application scenarios that require efficient real-time detection.

[0045] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for detecting particle motion trajectory in a real-time edge device dry magnetic separator, characterized in that, Includes the following steps: Step 1: Construct edge-dilated convolutional blocks based on a progressive fusion method combining depthwise separable convolutional blocks and edge detection blocks. Construct upsampling blocks with dual inputs and single outputs for feature enhancement based on weighted fusion of edge-dilated convolutional blocks. Construct downsampling blocks with dual inputs and single outputs for feature compression fusion based on edge-dilated convolutional blocks combined with multiple convolution processing. Construct a feature extraction backbone based on depthwise separable convolutional blocks and edge-dilated convolutional blocks. Construct the first output channel by combining multi-scale sampling and progressive fusion with upsampling blocks and downsampling blocks at different layers of the feature extraction backbone. Construct a detection model based on the feature extraction backbone and the first output channel. Step 2: Obtain the working video of the dry magnetic separator to be tested, cut it into images in chronological order, input it into the detection model, obtain the predicted particle movement trajectory of the dry magnetic separator from the output of the detection model, and adjust the ore separation plate of the dry magnetic separator based on the predicted particle movement trajectory of the dry magnetic separator.

2. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 1, characterized in that, The edge-depth dilated convolutional block divides the input features into three parts, performs convolution processing on the first and second parts of the features to different degrees, performs edge detection processing on the third part of the features to obtain edge features, and then gradually merges the edge features with the features of the first and second parts to obtain the output features.

3. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 2, characterized in that, The method of constructing edge-depth-hole convolutional blocks based on the gradual fusion of depth-separable convolutional blocks and edge detection blocks includes: constructing a three-channel parallel structure based on depth-separable convolutional blocks and edge detection blocks with different convolutional kernels; constructing a dual-channel parallel structure based on splicing layers and convolutional layers; and constructing a second output channel based on a first summing layer and a convolutional layer. Based on the three-channel parallel structure, the two-channel parallel structure, and the second output channel, a network structure is constructed by combining block layers, with the block layers, the three-channel parallel structure, the two-channel parallel structure, and the second output channel connected in sequence as edge depth-dilated convolutional blocks.

4. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 3, characterized in that, The three-channel parallel structure includes a first channel, a second channel, and a third channel; The first channel sequentially includes a depth-separable hole convolution with a kernel size of 5 and a hole stride of 3, and a 1*1 first convolutional layer; The second channel includes an edge detection block, a 1*1 second convolutional layer, and a 1*1 third convolutional layer. The second and third convolutional layers are in parallel and are both connected to the edge detection block. The third channel sequentially includes a depth-separable hole convolution with a kernel size of 7 and a hole stride of 3, and a 1*1 fourth convolutional layer. The first, second, third, and fourth convolutional layers are all connected to the dual-channel parallel structure.

5. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 4, characterized in that, The dual-channel parallel structure includes a fifth channel and a sixth channel. The fifth channel is connected to the first and second convolutional layers, and the sixth channel is connected to the third and fourth convolutional layers. The fifth channel includes, in sequence, a first splicing layer and a 1*1 fifth convolutional layer; The sixth channel consists of a second splicing layer and a 1*1 sixth convolutional layer.

6. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 5, characterized in that, The second output channel consists of a first additive layer and a 1*1 seventh convolutional layer. Both the fifth and sixth convolutional layers are connected to the first additive layer.

7. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to any one of claims 1-6, characterized in that, The construction of the feature extraction backbone based on depth-separable convolution and edge-dilated convolutional blocks includes: constructing four backbone units, namely the first backbone unit, the second backbone unit, the third backbone unit, and the fourth backbone unit, based on a network structure with a depth-separable convolutional layer with a kernel size of 3 and a stride size of 2 and an edge-dilated convolutional block. The first to fourth backbone units are connected in sequence, and a depthwise separable convolutional layer with a kernel size of 3 and a stride of 1 is connected to the input of the first backbone unit to complete the construction of the feature extraction backbone.

8. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 7, characterized in that, The first output channel is constructed as follows: at the edge depth-dilated convolutional blocks of the feature extraction backbone from the first backbone unit to the fourth backbone unit, a depth-separable convolutional layer with a kernel size of 3 and a stride of 1 is connected. Upsampling blocks are set between each pair of depth-separable convolutional layers connected to the fourth backbone unit and the first backbone unit in order from bottom to top for feature enhancement. Downsampling blocks are set between each pair of upsampling blocks in order from top to bottom for feature compression and fusion. A fully connected layer is connected at the output end of the last downsampling block to complete the construction of the first output channel.

9. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 8, characterized in that, The upsampling block includes a seventh channel and an eighth channel; The seventh channel includes a 1*1 eighth convolutional layer, a transposed convolutional layer, a second additive layer, and an edge-depth-hole convolutional block connected in sequence. The eighth channel includes a 1*1 ninth convolutional layer, which is connected to the second additive layer.

10. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 8, characterized in that, The downsampling block includes a ninth channel and a tenth channel; The ninth channel includes a 1*1 tenth convolutional layer, a depth-separable convolutional layer, a third additive layer, and an edge-depth-hole convolutional block connected in sequence. The tenth channel includes a 1*1 eleventh convolutional layer, which is connected to the third additive layer.

11. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to any one of claims 1-6, characterized in that, The images input to the detection model are annotated using an image annotation tool; The annotation process includes: annotating the strong magnetic regions and weak magnetic regions in the image.

12. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 11, characterized in that, The detection model constructs a loss function based on classification loss, localization loss, and edge loss; The classification loss is constructed based on the error of the detection model in classifying the strong magnetic region and weak magnetic region of the dry magnetic separator in the image, and the classification loss is calculated by sampling the cross-entropy loss function; The localization loss is constructed based on the coordinate difference between the ground truth bounding box constructed by the detection model based on the particle motion trajectory of the dry magnetic separator in the image and the predicted bounding box constructed by the detection model based on the particle motion trajectory. The localization loss is calculated using the smoothed L1 loss function. The edge loss is constructed based on the accuracy of the detection model in optimizing the boundary of the predicted bounding box region, and the edge loss is calculated using the Dice coefficient loss function.

13. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to claim 12, characterized in that, The loss function is calculated using the following formula: ； in, Represents the loss function; Indicates classification loss; The weighting coefficients represent the classification loss. Indicates positioning loss; The weighting coefficients representing the localization loss; Indicates marginal loss; The weighting coefficients represent the edge loss. The classification loss is calculated using the following formula: ； in, Indicates the first The category of each region, if it is a strong magnetic region, then The value is 1; if it is a weak magnetic region, then... =0; The first prediction made by the detection model represents the... The probability that a region belongs to a strong magnetic region; The positioning loss is calculated using the following formula: ； in, Represents the smoothing loss function; Indicates the center coordinates of the prediction box. Indicates the width and height of the prediction box; Represents the center coordinates of the true bounding box. Represents the width and height of the actual bounding box; The edge loss is calculated using the following formula: ； in, This represents the set of pixels that indicate a strong or weak magnetic region predicted by the detection model. A set of pixels representing a labeled strong or weak magnetic region; This represents the number of pixels that intersect the predicted region of the detection model with the labeled ground truth region; This represents the number of pixels in the predicted region as predicted by the detection model. This represents the number of pixels in the actual labeled area.

14. The method for detecting particle motion trajectory in a real-time edge device dry magnetic separator according to any one of claims 1-6, characterized in that, The detection model is converted from a PyTorch model to the ONNX format, and then from the ONNX format to the RKNN3588 format before being deployed to the hardware platform.

Images