A real-time edge device dry magnetic separator particle motion trajectory detection method

Abstract

The present application relates to the technical field of image processing, and discloses a kind of real-time edge device dry magnetic separator particle motion trajectory detection method.It includes: based on the edge depth hollow convolution block of depth separable convolution block and edge detection block is constructed, based on the edge depth hollow convolution block is constructed up sampling block, based on the edge depth hollow convolution block is constructed down sampling block, according to the edge depth hollow convolution block of depth separable convolution is constructed feature extraction stem, using up sampling block, down sampling block constructs first output channel, based on the feature extraction stem and first output channel constructs detection model;After obtaining the picture of the dry magnetic separator to be detected, input detection model, obtain the predicted particle motion trajectory of the dry magnetic separator output by detection model, based on the predicted particle motion trajectory of the dry magnetic separator, adjust the ore separating plate of the dry magnetic separator.The problem that some dry magnetic separator motion trajectory detection model cannot effectively carry out trajectory detection is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and discloses a real-time edge device dry magnetic separator particle motion trajectory detection method. BACKGROUND

[0002] Dry magnetic separation technology has important application value in the field of mineral separation, especially in water-free environments or water-saving operations. With the increasing demand for resource conservation and environmental protection, dry magnetic separators, with their water-free characteristics, have gradually become one of the preferred methods for mineral separation. However, the performance of dry magnetic separators depends on accurate control and monitoring, especially real-time identification and tracking of particle motion trajectories. How to efficiently and accurately monitor the operating state of the dry magnetic separator and capture the motion trajectory of the particles has become a key technical problem to improve the operating efficiency of the magnetic separation equipment, reduce energy consumption, and ensure production continuity.

[0003] Existing dry magnetic separator motion trajectory detection methods mostly rely on traditional deep learning image processing algorithms and deep learning-based image segmentation models. Traditional image processing methods usually rely on feature extraction and matching algorithms, such as edge detection and optical flow methods. Although these methods can provide effective detection results in certain specific scenarios, they are difficult to work stably in complex backgrounds, dynamic motion, and high-frequency changing environments, and require high-performance equipment, which cannot meet the demand for efficient and real-time monitoring.

[0004] In the deep learning-based scheme, existing technologies such as "A dry magnetic separator particle real-time separation method based on UNet" (Patent Application No. 2023117233741) propose using UNet structure for real-time separation of magnetic particles. UNet can achieve accurate particle identification and separation through its powerful feature extraction and image segmentation capabilities. This method is particularly suitable for servers or industrial hosts with high computing power. However, UNet and other structures have a large number of parameters and high computational complexity, making them difficult to deploy on edge devices with limited computing power, which makes them unsuitable for low-power, low-latency, and resource-constrained field applications. In addition, traditional convolutional neural network (CNN) models such as UNet have limited ability to identify dynamic motion trajectories, making it difficult to capture the rapid changes and motion characteristics of particles during the magnetic separation process. SUMMARY

[0005] The present application provides a real-time edge device dry magnetic separator particle motion trajectory detection method to address the problem that existing dry magnetic separator motion trajectory detection models cannot effectively detect trajectories.

[0006] To achieve the above-mentioned purpose, the present application realizes the technical scheme as follows:

[0007] The application provides a particle motion trajectory detection method for a real-time edge device dry magnetic separator, comprising the following steps:

[0008] Step 1: constructing an edge deep hollow convolution block based on a deep separable convolution block and an edge detection block in a step-by-step fusion mode, constructing a double-input single-output up-sampling block for feature enhancement based on the edge deep hollow convolution block and a weighted fusion, constructing a double-input single-output down-sampling block for feature compression fusion based on the edge deep hollow convolution block and a multiple convolution processing, constructing a feature extraction backbone based on a deep separable convolution and the edge deep hollow convolution block, constructing a first output channel on different layers of the feature extraction backbone by combining a multi-scale sampling with the step-by-step fusion of the up-sampling block and the down-sampling block, and constructing a detection model based on the feature extraction backbone and the first output channel;

[0009] The edge deep hollow convolution block divides the input features into three parts, performs different degree of convolution processing on the first part of features and the second part of features, performs edge detection processing on the third part of features, obtains edge features, and performs step-by-step fusion processing on the edge features and the first part and the second part of features to obtain output features.

[0010] The edge deep hollow convolution block based on the deep separable convolution block and the edge detection block in the step-by-step fusion mode comprises the following steps: constructing a three-channel parallel structure based on a deep separable convolution block with different convolution kernels and an edge detection block, constructing a double-channel parallel structure based on a splicing layer and a convolution layer, and constructing a second output channel based on a first addition layer and a convolution layer.

[0011] The network structure sequentially connected by the block layer, the three-channel parallel structure, the double-channel parallel structure and the second output channel is constructed as the edge deep hollow convolution block based on the three-channel parallel structure, the double-channel parallel structure and the second output channel in combination with the block layer.

[0012] Step 2: obtaining a working video of the dry magnetic separator to be detected, cutting the working video into pictures in time sequence and inputting the pictures into the detection model, obtaining a predicted particle motion trajectory of the dry magnetic separator output by the detection model, and adjusting a separating plate of the dry magnetic separator based on the predicted particle motion trajectory of the dry magnetic separator.

[0013] The method for adjusting the separating plate of the dry magnetic separator based on the predicted particle motion trajectory of the dry magnetic separator comprises: analyzing the motion images of the particles under the action of different magnetic fields, detecting the spatial distribution characteristics of the particle groups, and then generating an optimized control instruction of the separation area in real time to guide the dynamic adjustment of the position of the separating plate, so that the falling point areas of the strong magnetic particles and the weak magnetic particles are more clearly separated, thereby overcoming the industrial problems of blurred separation boundary and decreased beneficiation precision of the traditional fixed separating plate under the fluctuation of the particle trajectory, effectively improving the separation precision and operation stability of the dry magnetic separator; regarding the generation of the optimized control instruction based on the predicted particle motion trajectory, the prior art will not be described here;

[0014] Through the above design, the depthwise separable convolution and edge depthwise separable dilated convolution techniques are adopted to greatly reduce the calculation amount and model parameters. This enables the model to efficiently run on resource-constrained edge devices, avoiding the problems of high computational complexity and large memory requirement of traditional deep learning models, and is particularly suitable for deployment in hardware environments such as embedded systems, industrial camera modules, RK3588 platforms, etc. in industrial fields.

[0015] Further, the three-channel parallel structure includes a first channel, a second channel, and a third channel.

[0016] The first channel includes, in sequence, a depthwise separable dilated convolution with a convolution kernel of 5 and a dilated step of 3, and a first convolution layer of 1*1;

[0017] The second channel includes an edge detection block, a second convolution layer of 1*1, and a third convolution layer of 1*1, the second convolution layer and the third convolution layer are in parallel structure, and both are connected to the edge detection block;

[0018] The third channel includes, in sequence, a depthwise separable dilated convolution with a convolution kernel of 7 and a dilated step of 3, and a fourth convolution layer of 1*1, the first convolution layer, the second convolution layer, the third convolution layer, and the fourth convolution layer are connected to the double-channel parallel structure.

[0019] Further, the double-channel parallel structure includes a fifth channel and a sixth channel, the fifth channel is connected to the first convolution layer and the second convolution layer, and the sixth channel is connected to the third convolution layer and the fourth convolution layer;

[0020] The fifth channel includes, in sequence, a first splicing layer, a fifth convolution layer of 1*1;

[0021] The sixth channel includes, in sequence, a second splicing layer, a sixth convolution layer of 1*1.

[0022] Further, the second output channel comprises a first addition layer and a 1*1 seventh convolution layer in sequence;

[0023] The fifth convolution layer and the sixth convolution layer are connected with the first addition layer.

[0024] Further, the feature extraction backbone is constructed according to the depth separable convolution and the edge depth hole convolution block, comprising: based on the network structure of the depth separable convolution layer with a convolution kernel of 3 and a step of 2 and the edge depth hole convolution block, four first backbone units, second backbone units, third backbone units and fourth backbone units with the same network structure are constructed;

[0025] The first backbone unit to the fourth backbone unit are connected in sequence, and a depth separable convolution layer with a convolution kernel of 3 and a step of 1 is connected at the input end of the first backbone unit, so as to complete the construction of the feature extraction backbone.

[0026] Further, the first output channel is constructed by connecting the depth separable convolution layer with a convolution kernel of 3 and a step of 1 at the edge depth hole convolution block of the first backbone unit to the fourth backbone unit of the feature extraction backbone, setting the up-sampling block between the two to enhance the feature from bottom to top, setting the down-sampling block between the two to compress and fuse the feature from top to bottom, and connecting the fully connected layer at the output end of the last down-sampling block to complete the construction of the first output channel.

[0027] Further, the up-sampling block comprises a seventh channel and an eighth channel;

[0028] The seventh channel comprises a 1*1 eighth convolution layer, a transposed convolution layer, a second addition layer and an edge depth hole convolution block connected in sequence;

[0029] The eighth channel comprises a 1*1 ninth convolution layer, and the ninth convolution layer is connected with the second addition layer.

[0030] Further, the down-sampling block comprises a ninth channel and a tenth channel;

[0031] The ninth channel comprises a 1*1 tenth convolution layer, a depth separable convolution layer, a third addition layer and an edge depth hole convolution block connected in sequence;

[0032] The tenth channel comprises a 1*1 eleventh convolution layer, and the eleventh convolution layer is connected with the third addition layer.

[0033] Through the above design, through the multi-scale feature fusion and edge enhancement mechanism, the dynamic motion trajectory of the particles in the dry magnetic separator can be effectively captured. Through the edge depth separable hollow convolution technology, the model can more sensitively capture the changes of the particle edge in the dynamic environment, thereby improving the recognition accuracy of the motion trajectory, especially in the complex scene of particle overlap and motion blur, maintaining high detection performance.

[0034] Further, the picture of the input detection model is labeled by a picture labeling tool.

[0035] The labeling process includes labeling the strong magnetic region and the weak magnetic region in the picture.

[0036] Further, the detection model constructs a loss function based on a classification loss, a positioning loss, and an edge loss.

[0037] The classification loss is constructed based on the error of the detection model in classifying the strong magnetic region and the weak magnetic region of the dry magnetic separator in the picture, and the classification loss is calculated using a cross-entropy loss function.

[0038] The positioning loss is constructed based on the coordinate difference between the real box constructed by the detection model for the particle motion trajectory in the picture and the predicted box constructed by the detection model for the particle motion trajectory, and the positioning loss is calculated using a smooth L1 loss function.

[0039] The edge loss is constructed based on the accuracy of the optimization of the boundary of the predicted box region by the detection model, and the edge loss is calculated using a Dice coefficient loss function.

[0040] Further, the loss function is calculated by the following formula:

[0041] ;

[0042] Wherein, represents the loss function; represents the classification loss; represents the weight coefficient of the classification loss; represents the positioning loss; represents the weight coefficient of the positioning loss; represents the edge loss; represents the weight coefficient of the edge loss;

[0043] The classification loss is calculated by the following formula:

[0044] ;

[0045] Wherein, represents the class of the i-th region, if it is a strong magnetic region, then represents the class of the i-th region, if it is a strong magnetic region, then is 1, if it is a weak magnetic area, then is 0; denotes the probability that the detection model predicts the i-th region to belong to a strong magnetic area;

[0046] The positioning loss is calculated by the following formula:

[0047] ;

[0048] wherein, denotes a smoothing loss function; denotes the center coordinates of the prediction box, denotes the width and height of the prediction box; denotes the center coordinates of the real box, denotes the width and height of the real box;

[0049] The edge loss is calculated by the following formula:

[0050] ;

[0051] wherein, denotes the pixel set of the strong magnetic area or the weak magnetic area predicted by the detection model; denotes the pixel set of the labeled strong magnetic area or weak magnetic area; denotes the number of intersection pixels of the predicted region of the detection model and the labeled real region; denotes the number of pixels of the predicted region predicted by the detection model; denotes the number of pixels of the labeled real region.

[0052] Further, the format of the detection model is converted from PyTorch model to ONNX format, and then converted from ONNX format to RKNN3588 format and deployed to hardware platform.

[0053] Through the above design, it has extremely low computational complexity and high inference speed, which can meet the real-time processing needs of high-frequency image flow in the dry magnetic separator process. Compared with the traditional target detection method, the model of the application can provide faster response speed, real-time monitoring of the motion trajectory of the particles, ensure no delay detection in the fast changing environment, and adapt to the strict requirements of real-time in industrial field;

[0054] While maintaining high accuracy and real-time performance, the consumption of computing resources is effectively reduced, and the power consumption is relatively low. This makes the model very suitable for long-term operation on edge devices, meeting the needs of low power consumption and long-term stable operation in industrial field.

[0055] Beneficial effects:

[0056] ​The application provides a real-time edge device dry magnetic separator motion trajectory detection method, which combines deep separable convolution and hole convolution, can extract feature information of particle motion on multiple scales, improves detection accuracy of small-range particle motion, enhances perception ability of large-range motion, and improves adaptability to complex motion trajectory of the dry magnetic separator;

[0057] Robustness in complex environments is emphasized. Through edge deep separable hole convolution, the application can capture edge information, and still maintain high detection accuracy and stability under conditions such as light changes, background interference, and blurred particle motion trajectories, adapting to complex changes in industrial sites;

[0058] Not only suitable for edge devices with limited computing resources, but also can be easily adapted to different hardware platforms (such as industrial cameras, embedded terminals, etc.). Efficient deployment of the model on multiple platforms ensures its wide application prospects, and can provide reliable support for intelligent manufacturing and industrial automation in different scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 The network structure diagram of the detection model of the embodiment of the application is shown;

[0060] Figure 2 The network structure diagram of the edge deep hole convolution block of the embodiment of the application is shown;

[0061] Figure 3 The network structure diagram of the up-sampling block of the embodiment of the application is shown;

[0062] Figure 4 The network structure diagram of the down-sampling block of the embodiment of the application is shown. DETAILED DESCRIPTION

[0063] The technical solutions of the application will be described below in a clear and complete manner. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0064] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as those commonly understood by a person of ordinary skill in the art to which the present application belongs. The terms "first", "second", and similar terms are used herein merely to distinguish one element from another, and are not intended to imply any order, quantity, or importance. Similarly, the terms "one" or "a" or "an" are not limited to one, but rather mean at least one. The terms "connected" or "coupled" or similar terms are not limited to a direct connection or coupling, but rather include an indirect connection or coupling, whether or not it is physical, mechanical, electrical, or otherwise. The terms "upper", "lower", "left", "right", and the like are used merely to indicate relative positions, and when the absolute positions of the described objects are changed, the relative positions are also changed accordingly.

[0065] Please refer to Figure 1 The embodiment provides a kind of real-time edge device dry magnetic separator motion trajectory detection method, including the following steps:

[0066] Step 1: based on the gradual fusion mode of combining depthwise separable convolution block and edge detection block, edge depthwise dilated convolution block is constructed, based on edge depthwise dilated convolution block combination weighting fusion, the up-sampling block of double input single output for feature enhancement is constructed, based on edge depthwise dilated convolution block combination multiple convolution processing, the down-sampling block of double input single output for feature compression fusion is constructed, according to depthwise separable convolution and edge depthwise dilated convolution block, feature extraction backbone is constructed, on the different layers of feature extraction backbone, multi-scale sampling is combined with gradual fusion up-sampling block and down-sampling block to construct first output channel, based on feature extraction backbone and first output channel, detection model is constructed;

[0067] Regarding the construction of edge depthwise dilated convolution block (Edge DDConv), in order to adapt to the calculation limit of edge device, efficient depthwise separable convolution is used for feature extraction by reducing the amount of calculation and storage requirement, for specific construction process, please refer to Figure 2 , including: based on the depthwise separable convolution block of different convolution kernel and edge detection block, three-channel parallel structure is constructed, based on splicing layer and convolution layer, double-channel parallel structure is constructed, based on first addition layer and convolution layer, second output channel is constructed;

[0068] Based on three-channel parallel structure, double-channel parallel structure, second output channel combination block layer, network structure connected in order with block layer, three-channel parallel structure, double-channel parallel structure and second output channel is constructed as edge depthwise dilated convolution block;

[0069] Specifically, three-channel parallel structure includes first channel, second channel and third channel.

[0070] The first channel comprises, in sequence, a depth separable dilated convolution with a convolution kernel of 5 and a hollow step of 3 and a first convolution layer of 1*1;

[0071] The second channel comprises an edge detection block, a second convolution layer of 1*1 and a third convolution layer of 1*1, the second convolution layer and the third convolution layer are in parallel structure and are connected with the edge detection block;

[0072] The third channel comprises, in sequence, a depth separable dilated convolution with a convolution kernel of 7 and a hollow step of 3 and a fourth convolution layer of 1*1, the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer are connected with a double-channel parallel structure.

[0073] More specifically, the double-channel parallel structure comprises a fifth channel and a sixth channel, the fifth channel is connected with the first convolution layer and the second convolution layer, and the sixth channel is connected with the third convolution layer and the fourth convolution layer;

[0074] The fifth channel comprises, in sequence, a first splicing layer and a fifth convolution layer of 1*1;

[0075] The sixth channel comprises, in sequence, a second splicing layer and a sixth convolution layer of 1*1.

[0076] The second output channel comprises, in sequence, a first addition layer and a seventh convolution layer of 1*1;

[0077] The fifth convolution layer and the sixth convolution layer are connected with the first addition layer;

[0078] When the edge depth dilated convolution block processes the input features, the edge depth dilated convolution block processes the input features into three parts, performs different degree of convolution processing on the first part and the second part of the features, performs edge detection processing on the third part of the features, obtains edge features, and gradually fuses the edge features with the first part and the second part of the features to obtain output features, wherein the first part is the image features input into the first channel, the second part is the image features input into the third channel, and the third part is the image features input into the second channel, in the first channel, local features are mainly extracted, in the second channel, complex features are mainly extracted, and in the third channel, edge features are mainly extracted, and then different features are gradually spliced and fused through the double-channel parallel structure and the second output channel. In this process, by dividing the input image into blocks and processing each block differently, key features in the image can be effectively extracted, especially when processing large-scale data, computing resources can be efficiently utilized;

[0079] Please refer to Figure 3The Upscale Fusion Block (UFBlock) aims to enhance feature information at different scales by upsampling, convolution, and fusion of input features. This module is particularly suitable for scenarios requiring multi-scale information fusion and can efficiently merge input features of different resolutions to improve the model's expression ability for complex information.

[0080] The main function is to perform feature fusion at multiple scales, allowing low-resolution and high-resolution features to complement each other and improve the accuracy and information content of overall feature expression. This operation performs 1x1 convolution, upsampling, and weighted fusion on the two input features, and finally outputs the final fused features through the EdgeDDConv block.

[0081] For the specific network structure of the Upscale Fusion Block, the Upscale Fusion Block includes a seventh channel and an eighth channel.

[0082] The seventh channel includes an eighth convolutional layer, a transpose convolutional layer, a second addition layer, and an EdgeDDConv block in order.

[0083] The eighth channel includes a ninth convolutional layer, which is connected to the second addition layer.

[0084] When processing two input features, the Upscale Fusion Block first converts the dimensions of the two input features through the eighth convolutional layer and the ninth convolutional layer to match the feature dimensions, then performs upsampling through the transpose convolutional layer to make the low-resolution input features (feature maps) have the same spatial dimensions as the high-resolution input features, better realizing feature fusion, and finally enhancing the expression ability of the output features through the EdgeDDConv block.

[0085] Please refer to Figure 4 The core goal of the DownScale Compression Block (DCBlock) is to reduce computational complexity by reducing spatial resolution while preserving key information to ensure effective feature expression. This module is suitable for feature map compression and effectively extracts edge information from features through the EdgeDDConv block, enhancing the model's ability to perceive details such as motion trajectories.

[0086] Through multiple convolution processing, feature fusion, and finally edge depth separable dilated convolution operations on input feature maps, the representation of feature maps is optimized, and the limited computing resources on edge devices are adapted.

[0087] For the specific network structure of the DownScale Compression Block, the DownScale Compression Block includes a ninth channel and a tenth channel.

[0088] The ninth channel comprises, in sequence, a tenth convolutional layer of 1*1, a depth separable convolutional layer, a third addition layer, and an edge depth hole convolutional block;

[0089] The tenth channel comprises an eleventh convolutional layer of 1*1, which is connected with the third addition layer;

[0090] When the two input features are processed by the downsampling block, the feature dimensions of the input features processed by the tenth convolutional layer and the eleventh convolutional layer are matched first, then the depth convolution and point-by-point convolution of the depth separable convolutional layer are performed to reduce the calculation complexity, local features of the input features processed by the tenth convolutional layer are extracted, and after fusion by the third addition layer, the details of the input features are captured and processed by the edge depth hole convolutional block to enhance the expression of the edge features.

[0091] See Figure 1 For the feature extraction backbone, more edge information of different scales, i.e., the extraction of edge features of different scales, is achieved by a plurality of backbone units, which reduces the calculation complexity and improves the sensitivity of the particle motion trajectory;

[0092] For the network structure of the feature extraction backbone, the network structure comprises four first, second, third and fourth backbone units, each of which comprises a depth separable convolutional layer with a convolution kernel of 3 and a step of 2 and an edge depth hole convolutional block.

[0093] The first, second, third and fourth backbone units are connected in sequence, and a depth separable convolutional layer with a convolution kernel of 3 and a step of 1 is connected to the input end of the first backbone unit to construct the feature extraction backbone.

[0094] The construction of the first output channel based on the network structure of the feature extraction backbone comprises connecting a depth separable convolutional layer with a convolution kernel of 3 and a step of 1 to the edge depth hole convolutional blocks of the first to fourth backbone units of the feature extraction backbone, setting an upsampling block between each two of the edge depth hole convolutional blocks from bottom to top in sequence from the depth separable convolutional layer connected to the fourth backbone unit to the depth separable convolutional layer connected to the first backbone unit for feature enhancement, setting a downsampling block between each two of the upsampling blocks from top to bottom for feature compression and fusion, and connecting a fully connected layer to the output end of the last downsampling block to complete the construction of the first output channel.

[0095] In the network structure of the feature extraction backbone and the first output channel, the input features are subjected to multi-scale sampling and step-by-step fusion of different scales, wherein the upsampling block saves information of different scales and effectively fuses features of different scales to improve the real-time response capability of the model on edge devices, and the downsampling block compresses the features to reduce the calculation amount while retaining important motion trajectory information, thereby realizing the prediction of the particle motion trajectory in the magnetic dry magnetic separator.

[0096] Finally, a detection model is constructed based on the feature extraction backbone and the first output channel. The core design idea of ​​the detection model is to reduce the amount of computation and improve the detection accuracy of the dry magnetic separator's motion trajectory on the edge device through efficient feature extraction and fusion methods. The efficient feature extraction and fusion methods are implemented through the network structure of the feature extraction backbone, upsampling block, downsampling block and the first output channel. This reduces redundant computation and improves the computational efficiency, enabling the model to be efficiently deployed and run on the edge device to detect the particle motion trajectory of the dry magnetic separator in real time.

[0097] The images input to the detection model are annotated using an image annotation tool.

[0098] The annotation process includes: annotating the strong magnetic regions and weak magnetic regions in the image;

[0099] Strong magnetic areas: These are areas marked in the image that are strongly affected by magnetic forces. These areas are usually near the magnetic separator or parts affected by a strong magnetic field. Strong magnetic areas correspond to the effective working area of ​​a dry magnetic separator and are typically represented as high-contrast areas in the image.

[0100] Weak magnetic areas: These are areas less affected by magnetic forces. These areas are often far from the core components of the magnetic separator or areas with weak magnetic fields. In the image, weak magnetic areas may appear as areas with low contrast or relatively uniform areas.

[0101] The loss function of the detection model is constructed based on classification loss, localization loss, and edge loss;

[0102] The classification loss is constructed based on the error of the detection model in classifying the strong and weak magnetic regions of the dry magnetic separator in the image. The classification loss is calculated using the cross-entropy loss function, which can effectively evaluate the difference between the predicted category distribution and the true label, and is expressed by the following formula:

[0103] ;

[0104] in, Indicates the first The category of each region, if it is a strong magnetic region, then The value is 1; if it is a weak magnetic region, then... =0; The first prediction made by the detection model represents the... The probability that a region belongs to a strong magnetic region;

[0105] The positioning loss is constructed based on the coordinate difference between the real box constructed by the detection model for the dry magnetic separator particle motion trajectory in the picture and the prediction box constructed by the detection model for the particle motion trajectory, the positioning loss is calculated by using a smooth L1 loss function, the smooth L1 loss can effectively reduce the influence of abnormal values, so that the loss is more stable, and is expressed by the following formula:

[0106] ;

[0107] Wherein, represents a smooth loss function; represents the center coordinates of the prediction box, represents the width and height of the prediction box; represents the center coordinates of the real box, represents the width and height of the real box;

[0108] And the smooth L1 loss can provide a relatively stable optimization effect in the regression task, especially for large error conditions, compared with the traditional L2 loss, it has better robustness, and is defined as:

[0109] ;

[0110] Wherein, represents the error between the predicted value and the real value;

[0111] The edge loss is constructed based on the accuracy of the optimization of the edge of the prediction box region by the detection model, the edge loss is calculated by using a Dice coefficient loss function, the Dice coefficient loss can help the model improve the boundary accuracy of the region, and is expressed by the following formula:

[0112] ;

[0113] Wherein, represents the pixel set of the strong magnetic region or the weak magnetic region predicted by the detection model; represents the pixel set of the strong magnetic region or the weak magnetic region labeled; represents the intersection pixel number of the prediction region of the detection model and the labeled real region; represents the pixel number of the prediction region predicted by the detection model; represents the pixel number of the labeled real region.

[0114] Then the expression of the final loss function is as follows:

[0115] ;

[0116] Wherein, represents the loss function; represents the classification loss; a weight coefficient representing the classification loss; a weight coefficient representing the localization loss; a weight coefficient representing the localization loss; a weight coefficient representing the edge loss; a weight coefficient representing the edge loss;

[0117] In this embodiment, the weight coefficient of the classification loss is set to 1.0, the weight coefficient of the localization loss is set to 5.0 to ensure accurate positioning, and the weight coefficient of the edge loss is set to 0.1 to assist in boundary refinement without affecting the main task.

[0118] The proposed loss function can optimize the classification, positioning accuracy and boundary refinement ability of the model in strong magnetic and weak magnetic regions by combining classification loss, localization loss and edge loss. Through reasonable weight adjustment, the contributions of each loss can be balanced during training, thereby achieving efficient and accurate motion trajectory detection tasks.

[0119] During the training process of the detection model, the Adam optimizer is used to optimize the network weights, the initial learning rate is set to 0.001, and the learning rate decay strategy is adopted. After a certain number of iterations, the learning rate will be reduced by a certain percentage to achieve more stable convergence. The momentum decay coefficients of the optimizer are set to 0.9 and 0.999 respectively to help accelerate convergence and improve training stability. During model training, the batch size is set to 32 and the training period (epoch) is 50. Through the adjustment of these hyperparameters, the model can efficiently learn from data and gradually optimize performance. The training environment is based on Windows 11 operating system, using NVIDIA RTX 4090 graphics card for calculation, which significantly improves the calculation efficiency. The software environment includes Python 3.9, PyTorch 2.1 and CUDA 12.1, which utilizes GPU acceleration to improve the speed and stability of model training.

[0120] Finally, when deploying the detection model, in order to deploy to a specific hardware platform, the ONNX format model is converted to RKNN3588 format. RKNN (Rockchip Neural Network) provides a deep learning inference framework that supports efficient neural network inference. Using RKNN Toolkit, ONNX model files can be converted to model formats suitable for RK3588 platforms.

[0121] During the conversion process, the RKNN Toolkit compiles and optimizes the model to meet the inference requirements of RK3588 hardware. The specific conversion process includes loading the ONNX model, making necessary configurations, optimizing the model (such as selecting quantization methods), and finally compiling to generate an RKNN model file. The converted RKNN model file (usually in.rknn format) can be directly used for inference applications on the RK3588 platform.

[0122] Through the above steps, this study successfully realized the conversion of the trained PyTorch model to the model adapted to the RK3588 platform. First, the trained PyTorch model is saved as a.pt file, then cross-platform conversion is realized through the ONNX format, and finally the ONNX model is converted to the RKNN format supported by RK3588 hardware through RKNN Toolkit. Such conversion process ensures that the model can be efficiently deployed on different hardware platforms and utilize the hardware acceleration of RK3588 for fast inference.

[0123] Step 2: Obtain the working video of the dry magnetic separator to be detected, cut it into pictures in chronological order, and input it into the detection model to obtain the predicted particle motion trajectory of the dry magnetic separator. Based on the predicted particle motion trajectory of the dry magnetic separator, adjust the ore distribution plate of the dry magnetic separator.

[0124] Adjusting the ore distribution plate of the dry magnetic separator based on the predicted particle motion trajectory of the dry magnetic separator includes: analyzing the motion images of particles under the action of different magnetic fields, detecting the spatial distribution characteristics of particle groups, and then generating real-time optimization control instructions for the separation area to guide the dynamic adjustment of the ore distribution plate position, so that the falling point areas of strong magnetic particles and weak magnetic particles are more clearly separated, thereby overcoming the industrial problems of blurred separation boundary and decreased beneficiation precision caused by the fluctuation of particle trajectory under the traditional fixed ore distribution plate, effectively improving the separation precision and running stability of the dry magnetic separator.

[0125] Finally, the real-time edge device dry magnetic separator motion trajectory detection method of the present application is compared with a variety of mainstream target detection models, including U-Net, Mask R-CNN, YOLOv5 and YOLOv8. These models have proven their excellent performance in different application scenarios, especially in semantic segmentation and target detection tasks. Therefore, by comparing with these models, the advantages and disadvantages of the model proposed in this paper in magnetic separator motion trajectory detection can be more comprehensively evaluated. For specific comparison results, please refer to Table 1:

[0126] Table 1: Comparison results of the model and corresponding method of the present application with mainstream target detection models:

[0127]

[0128] In this study, all models were tested on the RK3588 hardware platform, and INT8 quantization optimization was used to improve inference efficiency and reduce computational resource consumption. The experimental results show that the detection model proposed in this study achieves an mAP@0.5 of 98.35%, significantly outperforming other comparative models such as YOLOv8 (96.12%) and YOLOv5 (93.25%), demonstrating higher detection accuracy. In addition, the inference speed of the detection model of the present invention is 75 FPS, which performs excellently in real-time performance, far exceeding Mask R-CNN (2 FPS) and U-Net (3 FPS), which are limited in real-time detection tasks due to their slow inference speed. In contrast, YOLOv5 and YOLOv8, although having an advantage in inference speed, still cannot match the detection model of the present invention in terms of accuracy. Through INT8 quantization, the computational efficiency and memory usage of all models have been significantly optimized, enabling the detection model of the present invention to be efficiently deployed on edge devices and to achieve real-time target detection tasks. Therefore, combining high accuracy and fast inference speed, the detection model of the present invention performs excellently on the RK3588 platform, especially suitable for application scenarios that require efficient and real-time detection.

[0129] The foregoing describes in detail preferred embodiments of the present invention. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present invention. Therefore, any technical solution that can be obtained by logical analysis, reasoning or limited experiments based on the existing technology according to the concept of the present invention shall be within the protection scope determined by the claims.

Claims

1. A method for detecting particle trajectory of a real-time edge device dry magnetic separator, characterized in that, The method comprises the following steps: Step 1: constructing an edge deep hole convolution block based on a deep separable convolution block combined with an edge detection block in a step-by-step fusion manner, constructing a double-input single-output up-sampling block for feature enhancement based on the edge deep hole convolution block combined with weighted fusion, constructing a double-input single-output down-sampling block for feature compression fusion based on the edge deep hole convolution block combined with multiple convolution processing, constructing a feature extraction backbone according to the deep separable convolution and the edge deep hole convolution block, constructing a first output channel on different layers of the feature extraction backbone by combining multi-scale sampling with the up-sampling block and the down-sampling block in a step-by-step fusion manner, and constructing a detection model based on the feature extraction backbone and the first output channel; The edge deep hole convolution block divides the input features into three parts, performs different degrees of convolution processing on the first and second parts of the features, performs edge detection processing on the third part of the features, obtains edge features, and performs step-by-step fusion processing on the edge features and the first and second parts of the features to obtain output features; The edge deep hole convolution block constructed based on the deep separable convolution block combined with the edge detection block in a step-by-step fusion manner comprises: constructing a three-channel parallel structure based on the deep separable convolution block with different convolution kernels and the edge detection block, constructing a double-channel parallel structure based on a splicing layer and a convolution layer, and constructing a second output channel based on a first addition layer and a convolution layer; The network structure sequentially connected by the block layer, the three-channel parallel structure, the double-channel parallel structure, and the second output channel is constructed as the edge deep hole convolution block based on the three-channel parallel structure, the double-channel parallel structure, and the second output channel combined with the block layer; Step 2: obtaining a working video of the dry magnetic separator, cutting the working video into pictures in chronological order, inputting the pictures into the detection model, obtaining a predicted particle motion trajectory of the dry magnetic separator output by the detection model, and adjusting the ore distribution plate of the dry magnetic separator based on the predicted particle motion trajectory of the dry magnetic separator.

2. The method of claim 1, wherein the method is performed by the real-time edge device dry magnetic separator particle trajectory detection system of claim 1. The three-channel parallel structure comprises a first channel, a second channel, and a third channel; The first channel comprises, in sequence, a deep separable hole convolution with a convolution kernel of 5 and a hole step of 3, and a first convolution layer of 1*1; The second channel comprises an edge detection block, a second convolution layer of 1*1, and a third convolution layer of 1*1, the second convolution layer and the third convolution layer are in parallel structure, and both are connected with the edge detection block; The third channel comprises, in sequence, a deep separable hole convolution with a convolution kernel of 7 and a hole step of 3, and a fourth convolution layer of 1*1, and the first convolution layer, the second convolution layer, the third convolution layer, and the fourth convolution layer are connected with the double-channel parallel structure.

3. The method of claim 2, wherein the method further comprises: The double-channel parallel structure comprises a fifth channel and a sixth channel, the fifth channel is connected with the first convolution layer and the second convolution layer, and the sixth channel is connected with the third convolution layer and the fourth convolution layer; The fifth channel comprises, in sequence, a first splicing layer and a fifth convolution layer of 1*1; The sixth channel comprises, in sequence, a second splicing layer and a sixth convolution layer of 1*1.

4. The method of claim 3, wherein the method further comprises: The second output channel comprises, in sequence, a first addition layer and a seventh convolution layer of 1*1. The fifth convolutional layer and the sixth convolutional layer are connected with the first addition layer.

5. The method of claim 1-4, wherein, The feature extraction backbone is constructed according to the depth separable convolution and the edge depth hole convolution block, and includes: four first backbone units, second backbone units, third backbone units and fourth backbone units with the same network structure, which are constructed based on the depth separable convolution layer with a convolution kernel of 3 and a step of 2 and the edge depth hole convolution block. The first backbone unit to the fourth backbone unit are connected in sequence, and a depth separable convolution layer with a convolution kernel of 3 and a step of 1 is connected to the input end of the first backbone unit to complete the construction of the feature extraction backbone.

6. The method of claim 5, wherein the method further comprises: The first output channel is constructed by connecting the depth separable convolution layer with a convolution kernel of 3 and a step of 1 to the edge depth hole convolution block of the first backbone unit to the fourth backbone unit of the feature extraction backbone, setting an up-sampling block between each two of the depth separable convolution layers from the depth separable convolution layer connected to the fourth backbone unit to the depth separable convolution layer connected to the first backbone unit to perform feature enhancement, setting a down-sampling block between each two of the up-sampling blocks in the order from top to bottom to perform feature compression and fusion, and connecting a fully connected layer to the output end of the last down-sampling block to complete the construction of the first output channel.

7. The method of claim 6, wherein the method further comprises: The up-sampling block includes a seventh channel and an eighth channel. The seventh channel includes an eighth convolutional layer with a convolution kernel of 1*1, a transposed convolutional layer, a second addition layer and an edge depth hole convolution block connected in sequence. The eighth channel includes a ninth convolutional layer with a convolution kernel of 1*1, which is connected with the second addition layer.

8. The method of claim 6, wherein the method further comprises: The down-sampling block includes a ninth channel and a tenth channel. The ninth channel includes a tenth convolutional layer with a convolution kernel of 1*1, a depth separable convolutional layer, a third addition layer and an edge depth hole convolution block connected in sequence. The tenth channel includes an eleventh convolutional layer with a convolution kernel of 1*1, which is connected with the third addition layer.

9. The method of claim 1-4, wherein, The picture input into the detection model is processed by a picture labeling tool. The labeling processing includes labeling the strong magnetic region and the weak magnetic region in the picture.

10. The method of claim 9, wherein the method further comprises: The detection model constructs a loss function based on a classification loss, a positioning loss and an edge loss. The classification loss is constructed based on the error of the classification of the strong magnetic region and the weak magnetic region of the dry magnetic separator in the picture by the detection model, and the classification loss is calculated by using a cross-entropy loss function. The positioning loss is constructed based on the coordinate difference between the real box constructed by the detection model for the particle motion trajectory in the picture and the prediction box constructed by the detection model for the particle motion trajectory, and the positioning loss is calculated by using a smooth L1 loss function. The edge loss is constructed based on the accuracy of the optimization of the boundary of the prediction box region by the detection model, and the edge loss is calculated by using a Dice coefficient loss function.

11. The method of claim 10, wherein the method further comprises: The loss function is calculated by the following formula: ； wherein, represents a loss function; represents a classification loss; represents a weight coefficient of the classification loss; represents a positioning loss; represents a weight coefficient of the positioning loss; represents an edge loss; represents a weight coefficient of the edge loss; The classification loss is calculated by the following formula: ； in, Indicates the first The category of each region, if it is a strong magnetic region, then The value is 1; if it is a weak magnetic region, then... =0; The first prediction made by the detection model represents the... The probability that a region belongs to a strong magnetic region; The positioning loss is calculated by the following formula: ； wherein, represents a smooth loss function; represents the center coordinate of the prediction box, represents the width and height of the prediction box; represents the center coordinate of the real box, represents the width and height of the real box; The edge loss is calculated by the following formula: ； wherein, represents a set of pixels of a strong magnetic region or a weak magnetic region predicted by the detection model; represents a set of pixels of a strong magnetic region or a weak magnetic region labeled; represents the number of intersection pixels of the predicted region of the detection model and the labeled real region; represents the number of pixels of the predicted region predicted by the detection model; represents the number of pixels of the labeled real region.

12. The method of claim 1-4, wherein, The format of the detection model is converted from a PyTorch model to an ONNX format, and then converted from the ONNX format to an RKNN3588 format before being deployed to a hardware platform.

Images