Abrasive Particle Tracking Method, System, Product, Medium and Computer Device

Abstract

The present invention belongs to the technical field of image processing. A method, system, product, medium and computer device for abrasive particle tracking are provided. The current frame image is preprocessed to highlight the abrasive particle region, the abrasive particle contour of the preprocessed current frame image is extracted to generate an abrasive particle detection result. According to the abrasive particle detection result, the next frame position of each abrasive particle is predicted based on the unscented Kalman filter to determine the prediction trajectory; based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, the current frame position of the abrasive particle is matched with the prediction trajectory; when the matching is successful, the prediction trajectory with successful matching is updated, and when the matching is not successful, a new prediction trajectory is created and the matching continues. The present invention combines a multi-feature matching strategy, a non-linear motion trajectory prediction measurement based on the unscented Kalman filter, and a parallelized patch-aware attention mechanism, improving the detection and tracking accuracy of abrasive particle targets.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method, system, product, medium, and computer device for abrasive particle tracking. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of online oil monitoring technology, real-time monitoring of abrasive particles in the lubricating oil of mechanical equipment has become an important means for preventive maintenance and fault warning. The online oil monitoring system continuously collects oil samples during the operation of the equipment, and uses image processing and pattern recognition algorithms to quantitatively analyze and classify the tiny abrasive particles in the oil, so as to timely reflect the wear state of the equipment and reduce the risk of downtime. Compared with the traditional offline detection method, online monitoring has the advantages of fast response and high automation.

[0004] In the abrasive particle detection task, in addition to paying attention to its type, size, and morphological characteristics, stable tracking of abrasive particles is also a key issue. Since the abrasive particles will tumble as the lubricating oil flows, the projected shape and angle of the abrasive particles in the microscopic image will constantly change. For example, some elliptical or irregularly shaped abrasive particles may exhibit different aspect ratios and areas at different angles, and even present a nearly circular shape at a specific angle. This phenomenon may cause the detection algorithm to misidentify the same abrasive particle as multiple different abrasive particles in different frames, thus affecting the accurate judgment of its morphological characteristics (such as aspect ratio, area, perimeter, etc.). In addition, when multiple abrasive particles with similar shapes flow within the same field of view, the ID exchange problem is likely to occur, that is, the algorithm wrongly assigns the trajectory of one abrasive particle to another abrasive particle with a similar shape, reducing the tracking stability. Therefore, designing a method for cross-frame matching and motion prediction of abrasive particles to ensure the consistency of its ID in the entire video sequence is the key to improving the accuracy of abrasive particle morphological analysis.

[0005] In the field of computer vision, object detection technology has been widely applied to object recognition and tracking tasks. Among them, the YOLO (You Only Look Once) series of object detection models, with their end-to-end single-stage detection architecture, achieve efficient object recognition and localization. The YOLO model divides the entire image into an S×S grid and predicts the bounding box, confidence, and class probability of the object within each grid, thus completing the detection task in one forward pass. With the continuous evolution of versions, the detection accuracy and multi-scale object detection ability of the YOLO model have been continuously improved. For example, YOLOv3 has been optimized in multi-scale feature fusion, YOLOv5 has improved the detection speed through a lightweight network structure, and YOLOv11 has further improved the feature extraction network and adopted a more reasonable loss function (such as GIOU loss), making its performance more superior in dense object detection and small object detection.

[0006] However, in the application of online oil monitoring, abrasive particle detection based on traditional YOLO models still faces the following challenges:

[0007] (1) The problem of small object detection. In high-resolution microscopic images, the size of abrasive particles is usually small and densely distributed. Traditional single-scale feature extraction methods are difficult to effectively distinguish adjacent tiny objects, reducing the detection accuracy.

[0008] (2) The matching problem. Traditional object matching strategies only rely on single morphological features such as area and perimeter for object association. However, the tumbling and rotation of abrasive particles may cause drastic changes in their morphological features in consecutive frames, easily resulting in false matching or ID swapping.

[0009] (3) The motion model problem. Existing abrasive particle tracking methods mostly adopt a simple Kalman filter uniform motion model. However, due to the non-uniformity of the oil flow rate, the motion trajectory of abrasive particles is often non-linear motion with acceleration, deceleration, or external force influence, resulting in large errors in prediction methods based on the linear motion assumption in practical applications. Summary of the Invention

[0010] To solve the deficiencies of the existing technology, the present invention provides an abrasive particle tracking method, system, product, medium, and computer device, which combines a multi-feature matching strategy, non-linear motion trajectory prediction measurement based on unscented Kalman filter, and a parallelized patch-aware attention mechanism, improving the detection and tracking accuracy of abrasive particle targets.

[0011] To achieve the above objectives, the present invention adopts the following technical solutions:

[0012] In the first aspect, the present invention provides an abrasive particle tracking method.

[0013] An abrasive particle tracking method includes the following processes:

[0014] Preprocess the current frame image to highlight the abrasive particle region;

[0015] Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result;

[0016] According to the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine a prediction trajectory;

[0017] Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory;

[0018] When the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching.

[0019] In a second aspect, the present invention provides an abrasive particle tracking system.

[0020] An abrasive particle tracking system, comprising:

[0021] An image preprocessing unit, configured to: preprocess the current frame image to highlight the abrasive particle region;

[0022] An abrasive particle detection unit, configured to: extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result;

[0023] A trajectory prediction unit, configured to: according to the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine a prediction trajectory;

[0024] A trajectory matching unit, configured to: based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory;

[0025] A trajectory updating unit, configured to: when the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching.

[0026] In a third aspect, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium;

[0027] A processor, adapted to execute a computer program;

[0028] A computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by the processor, the abrasive particle tracking method as described in the first aspect of the present invention is implemented.

[0029] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, and the computer program is adapted to be loaded and executed by a processor to perform the abrasive particle tracking method as described in the first aspect of the present invention.

[0030] Fifthly, the present invention provides a computer program product including a computer program, and when the computer program is executed by a processor, the abrasive particle tracking method as described in the first aspect of the present invention is implemented.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] 1. The present invention introduces a parallel patch-aware attention (PPA) module to optimize small object detection. In the feature extraction network of YOLOv11, the PPA module is embedded. This module extracts multi-scale features through the cooperation of local branches, global branches, and serial convolution branches. This design enhances the detection ability for tiny abrasive particles, ensures the retention of key information during multiple downsampling processes, and improves the detection accuracy.

[0033] 2. The present invention proposes an optimization matching strategy for the morphological feature extraction method based on OpenCV. By directly using OpenCV to extract morphological features such as the area, perimeter, roundness, and aspect ratio of abrasive particles for abrasive particle matching, it not only solves the problems of inaccurate matching caused by the morphological changes of abrasive particles when only relying on the area and perimeter of abrasive particles for matching, but also significantly improves the real-time performance while ensuring the matching accuracy compared with the method of training an abrasive particle re-identification model using ResNet50, ensuring accurate and fast matching of the same abrasive particle between different frames.

[0034] 3. The present invention introduces the unscented Kalman filter (UKF) to improve motion prediction. By using the UKF to model the non-linear motion of abrasive particles and predicting the position of the next frame of abrasive particles using the state transition equation, the accuracy of motion estimation is enhanced, and the reliability of abrasive particle tracking is improved.

[0035] The advantages of the additional aspects of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0037] Figure 1 It is a schematic overall flow chart of the abrasive particle tracking method provided in Embodiment 1 of the present invention;

[0038] Figure 2 Schematic diagram of the parallel patch-aware attention module (PPA) provided in Embodiment 1 of the present invention;

[0039] Figure 3 Schematic diagram of the loss function for model training provided in Embodiment 1 of the present invention; among them, (A) is the training set bounding box regression loss, (B) is the training set classification loss, (C) is the training set distributed regression loss, (D) is the validation set bounding box regression loss, (E) is the validation set classification loss, and (F) is the validation set distributed regression loss;

[0040] Figure 4 Schematic diagram of the accuracy of model training provided in Embodiment 1 of the present invention, where (A) is the schematic diagram of precision, (B) is the schematic diagram of recall rate, (C) is the schematic diagram of mAP50, and (D) is the schematic diagram of mAP50-95;

[0041] Figure 5 Relationship curve between the accuracy and recall rate of the validation set data provided in Embodiment 1 of the present invention;

[0042] Figure 6 Schematic diagram of the abrasive particle target tracking result provided in Embodiment 1 of the present invention;

[0043] Figure 7 Schematic diagram of an abrasive particle tracking system provided in Embodiment 2 of the present invention;

[0044] Figure 8 Schematic diagram of a computer device provided in Embodiment 3 of the present invention. Detailed implementation manners

[0045] The present invention will be further described below in conjunction with the drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are all exemplary and are intended to provide further explanations of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0047] Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0048] Embodiment 1:

[0049] This implementation method proposes a particle tracking method, which combines a multi-feature matching strategy, a non-linear motion model based on the Unscented Kalman Filter (UKF), and a Parallelized Patch-Aware Attention (PPA) mechanism on the basis of the YOLOv11 model to improve the detection and tracking accuracy of particle targets. As Figure 1 shown, first, a particle dataset is constructed and the YOLOv11 model is trained. Feature extraction, feature fusion, coordinate prediction, loss calculation, and model training are performed through the YOLOv11 model. For the video of particles to be detected, the trained YOLOv11 model is used to determine the detection results (coordinates and confidence levels), and the detection results are input into the tracking model. The tracking model is used for motion prediction, data association and matching, and trajectory update to obtain the tracking results, thereby realizing the real-time tracking of particles.

[0050] More specifically, it includes the following processes:

[0051] S1: Data preprocessing and dataset construction.

[0052] S11: Extract each frame of the input video and preprocess each frame of the image (such as denoising, contrast enhancement, and size normalization, unified to 2592×1944 pixels);

[0053] S12: Use the LabelImg tool to construct a particle image dataset, annotate the particle bounding box and class information, etc., and construct a dataset in the YOLO format. The specific format is [class_id, x, y, w, h], which respectively represent the object class, the horizontal and vertical coordinates (x, y) of the center point of the bounding box, the size (w, h) of the bounding box, and the upper left corner of the image is the coordinate origin.

[0054] S13: Divide the constructed dataset into three categories: training set, validation set, and test set according to 8:1:1.

[0055] S2: YOLOv11 model training.

[0056] S21: Improve the YOLOv11 model for the particle detection task.

[0057] (1) Insert a Parallelized Patch-Aware Attention PPA module in the middle layer of the YOLOv11 network (such as Figure 3As shown in the figure, by simultaneously capturing local details and global background information, PPA can significantly enhance the feature expression ability of small targets, enabling the network to more accurately focus on subtle target regions, thereby enhancing the accuracy and robustness of small target detection. More specifically, the middle layer of the original YOLOv11 network consists of four consecutive convolutions. By adding two PPA modules (each including serial tensor unfolding, weighting, feature selection, and folding / reshape), it enters the attention module, and the attention module includes the processes of fused output, calculating attention, weighted fusion, and outputting enhanced features.

[0058] In this implementation, the input feature map F (with size ) is divided into non-overlapping Patches, and the size of each Patch is , defined as . Subsequently, the subsequent processing is performed on each small block, and the processed results are stitched and fused.

[0059] (2) Construct local branches, global branches, and serial convolutional branches.

[0060] The local branch uses standard convolutions to extract local detail features , the global branch extracts global context features through global average pooling or self-attention mechanisms , and the serial branch uses deep convolutions to further extract high-level semantic features .

[0061] (3) Calculate the attention matrix using the outputs of the local and global branches . First, obtain the query matrix Q and the key matrix K through learnable matrix transformations:

[0062] (1);

[0063] Among them, and are learnable parameter matrices. Then, calculate the attention matrix ;

[0064] (2);

[0065] Among them, is the scaling factor (usually taken as ).

[0066] (5) After applying the attention matrix to the local branch features, it is cascaded or weighted and summed with the high-level semantic features extracted by the serial branch to form the final multi-scale fusion features:

[0067] (3);

[0068] Among them, represents element-wise multiplication, represents a fusion operation (for example, it can be splicing followed by convolution or direct weighted summation), and this fused feature will be used as the input of the subsequent detection layer, significantly improving the detection ability for tiny abrasive particles.

[0069] S22: During the training process, a comprehensive loss function is adopted, including bounding box regression loss, classification loss, and matching and motion prediction error. Among them, the bounding box regression loss adopts the GIOU loss, and its definition is:

[0070] (4);

[0071] Among them, IOU is the intersection over union, that is, the ratio of the intersection area to the union area of two bounding boxes, which is an index in object detection to measure the overlapping degree between the predicted box and the ground truth box. GIOU is the generalized intersection over union, which introduces a penalty term for the minimum closed region of the two boxes on the basis of IOU. When the intersection area of the two boxes is 0, it avoids the value of GIOU being 0.

[0072] Such as Figure 3 and 4 shown, the bounding box regression loss, classification loss, and distribution regression loss all continuously decrease with the increase of the training epoch, indicating that the bounding box regression accuracy and classification accuracy of the model on the training set are gradually improving; at the same time, the bounding box regression loss, classification loss, and distribution regression loss on the validation set also basically show a decreasing trend, and there is no significant deviation from the training loss curve, indicating that the performance of the model on the validation set has a similar convergence law to the training set.

[0073] At the same time, the training and validation losses both converge to a relatively low level in the later stage, and the curves are stable, without significant rebound phenomena, indicating that the model has not shown serious overfitting. The curves of precision and recall can clearly reflect that the ability of the object detection model to determine positive and negative samples continuously improves with the progress of training; while mAP50 (mean Average Precision at IoU = 50%, that is, the average precision of the model on all classes when the IoU threshold is 50%) and mAP50 - 95 (mean Average Precision at IoU = 50% - 95%, that is, the average precision of the model on all classes in the range where the IoU threshold ranges from 50% to 95% (step size 0.05)) respectively reflect the average precision performance of the model when the GIOU threshold is 0.5 and in the interval from 0.5 to 0.95. It can be observed that all indicators steadily increase during the training process and tend to converge in the later stage, indicating that the overall detection performance of the model has reached a certain stable level.

[0074] S23: Use the Adam optimizer and set the initial learning rate to , set the number of training rounds to 100 rounds, and adjust hyperparameters such as the learning rate and weight decay according to the performance on the validation set until the model converges.

[0075] As Figure 5 shown, it is the P-R curve of the validation set data of the present invention, which shows the relationship between the model precision (Precision) and recall rate (Recall), indicating that the present model still maintains a high precision rate at a high recall rate.

[0076] S3: Build a Deepsort object tracking model, train the abrasive feature extraction network, and use the detection boxes of each frame of YOLOv11 as the input to complete the construction of the YOLOv11-Deepsort abrasive tracking model.

[0077] S31: Feature extraction.

[0078] Directly use the OpenCV method to extract morphological features such as the area, perimeter, roundness, and aspect ratio of the abrasive grains for abrasive grain matching. Compared with the method of training the abrasive grain re-identification model with ResNet50, it has higher real-time performance while ensuring the matching accuracy, ensuring that the same abrasive grain can be accurately and quickly matched between different frames.

[0079] Use the trained YOLOv11 model to extract the target boxes of each frame of the video;

[0080] Convert the image of each frame into a grayscale image, and then use filtering, binarization (such as Otsu threshold or adaptive threshold), and morphological operations (such as opening operation and closing operation) to remove noise;

[0081] Use the OpenCV method to extract the contours of all abrasive grains in the binary image, and extract multi-dimensional morphological feature vectors for the abrasive grain contours detected in each frame :

[0082] (5);

[0083] Among them, represents the area of the target abrasive grain; represents the perimeter of the target abrasive grain; represents the roundness of the target abrasive grain, defined as ; represents the aspect ratio of the target abrasive grain, defined as .

[0084] Generate detection results. Each abrasive grain not only has position information (such as the bounding box [x, y, w, h]), but also has a morphological feature vector , pack this information into a detection dictionary Z and pass it to the subsequent Deepsort (Deep Simple Online and Realtime Tracking) module:

[0085] (6);

[0086] S32: Motion prediction.

[0087] Use the Unscented Kalman Filter (UKF) to predict the next-frame position of each abrasive grain, providing a prior for data association to make the motion estimation more accurate. The UKF state transition equation is:

[0088] (7);

[0089] where, is the state transition function, is the state of the current frame , is the control input, is the state of the current frame , is the state of the next frame.

[0090] (1) Construct a motion prediction module based on UKF for the possible non-linear motion characteristics of abrasive grains in the video;

[0091] (2) Define the target state vector as:

[0092] (8);

[0093] where, is the target center position, is the target speed;

[0094] (3) Establish a state transition model:

[0095] UKF is used to handle non-linear state transitions. Its core process is as follows:

[0096] (9);

[0097] The specific form is:

[0098] (10);

[0099] (11);

[0100] (12);

[0101] (13);

[0102] Among them, represents the time interval between each frame in the video, represents the current frame of the target in the direction of speed, and are acceleration components, is the process noise, represents the current frame of the target in the x direction of speed.

[0103] (4) Establish an observation model :

[0104] (14);

[0105] Usually set , is the observation noise, is the state vector of the target box at the k frame; and are the center coordinates of the target box at the k frame.

[0106] (5) Use UKF for state prediction and update, calculate the predicted mean and covariance using non - linear transformation and sampling points to ensure accurate estimation of the abrasive motion state under the actual complex flow field.

[0107] S33: Data association and matching:

[0108] Combine motion and appearance information, perform optimal matching through the Hungarian algorithm, optimize the cross - frame association of targets, and achieve multi - target tracking. In the data association process of Deepsort, it is necessary to construct a matching cost matrix by combining the predicted state and the current detection result.

[0109] For each predicted trajectory (the state obtained from UKF prediction) and the current frame detection result (the morphological feature vector Z), define the comprehensive distance:

[0110] (15);

[0111] Among them: is the distance of position information, represents the - th existing target, that is, the target that has been tracked and predicted by UKF in the previous few frames, represents the - th target detected in the current frame. If the center coordinates of the predicted box in the next frame are , and the center coordinates of the detection box in the current frame are , then the Euclidean distance is adopted:

[0112] (16);

[0113] is the morphological feature distance,

[0114] For the morphological feature part, the normalized Euclidean distance is adopted:

[0115] (17);

[0116] Among them, is the predicted feature, is the detection feature, is the standard deviation of this feature, used for normalization, and the coefficients and control the relative weights of motion and morphological information.

[0117] Using the matching cost matrix , the Hungarian algorithm is adopted to solve the optimal matching scheme (the Hungarian algorithm finds the one-to-one correspondence between each trajectory and the detection target in the matching cost matrix, aims to minimize the total cost, and finally outputs an optimal allocation scheme that minimizes the total matching cost), ensuring the minimum overall matching cost. If the matching cost of a certain detection result after being matched by the Hungarian algorithm with the trajectory is lower than the set threshold, then this matching is determined to be successful; the matching result determines whether the current detection result is associated with the predicted trajectory. For the detection results that are not successfully matched, new trajectories are created (create a new matrix containing the appearance features and motion conditions of the un-detected successful target abrasive grains); if a certain existing trajectory has not been matched with any detection result in multiple consecutive frames, it indicates that the target represented by this trajectory may have disappeared. To maintain the accuracy of the tracking system and prevent false tracking, this trajectory is deleted.

[0118] S34: UKF trajectory update.

[0119] Update the successfully matched trajectories (because the features such as the appearance and speed of the abrasive grains will change, so it is necessary to update their change sequences), create new trajectories, and delete the trajectories that have not been matched for a long time. Through the measurement matrix (usually a linear mapping that maps the state vector to the measurement space) and the predicted covariance calculate:

[0120] (18);

[0121] Among them, is the measurement noise covariance matrix.

[0122] Let the detection measurement be z, and update the status:

[0123] (19);

[0124] Where, is the position of the target object in the predicted next frame, is the position of the target object in the updated next frame.

[0125] And update the covariance:

[0126] (20);

[0127] Where, is the predicted covariance, is the covariance of the k th frame.

[0128] In this way, each tracked target can obtain a more accurate state estimate according to the new detection results. As Figure 6 shown, it is a schematic diagram of the tracking results of abrasive particles in an actual video tracking scenario. The figure mainly includes: a detection box with an id of 179 and a confidence level of 0.66, a detection box with an id of 158 and a confidence level of 0.62, a detection box with an id of 288 and a confidence level of 0.27, a detection box with an id of 306 and a confidence level of 0.35, a detection box with an id of 87 and a confidence level of 0.47, a detection box with an id of 218 and a confidence level of 0.49, a detection box with an id of 152 and a confidence level of 0.64, a detection box with an id of 216 and a confidence level of 0.54, a detection box with an id of 214 and a confidence level of 0.61, and a detection box with an id of 282 and a confidence level of 0.58.

[0129] In this implementation method, the above model is used to track abrasive particles in the video. After obtaining the position information of each abrasive particle in each frame, a unique identifier is assigned to each abrasive particle. The center of the detection box is used to replace the abrasive particle, and the trajectory of the abrasive particle with the same ID over time is drawn. A CSV file is created as the trajectory record file for subsequent analysis. At the same time, in this implementation method, the shape features of the same abrasive particle in different postures are also extracted, and the area, perimeter, roundness, aspect ratio, etc. of each frame are calculated to track the morphological changes of the abrasive particle with the same ID in the time dimension and analyze the change trend of the abrasive particle morphology. By adopting the above technical solutions, the present invention realizes high-precision and real-time tracking of abrasive particle targets in the context of on-line oil monitoring. The improved modules cooperate with each other, showing obvious advantages in target matching, motion prediction, and small target detection, providing effective technical support for actual industrial monitoring.

[0130] Example 2:

[0131] AsFigure 7 As shown, this implementation provides a particle tracking system, including:

[0132] An image preprocessing unit, configured to: preprocess the current frame image to highlight the particle region;

[0133] A particle detection unit, configured to: extract the particle contour of the preprocessed current frame image and generate a particle detection result;

[0134] A trajectory prediction unit, configured to: based on the particle detection result, predict the position of each particle in the next frame based on the unscented Kalman filter and determine the predicted trajectory;

[0135] A trajectory matching unit, configured to: based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the particle and the position of the predicted trajectory, perform the matching between the current frame position of the particle and the predicted trajectory;

[0136] A trajectory updating unit, configured to: when the matching is successful, update the predicted trajectory with successful matching; when the matching is unsuccessful, create a new predicted trajectory and continue the matching.

[0137] For the specific working methods of the above-mentioned units, see the introduction in Embodiment 1 and will not be elaborated here.

[0138] It can be understood that the above-mentioned units can be separately or all combined into one or several other units to form, or some of them can be further split into multiple smaller units with functional divisions to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of this application. The above units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of this application, the system can also include other units. In practical applications, these functions can also be assisted by other units and can be achieved through the cooperation of multiple units.

[0139] According to another embodiment of this application, the system described in this embodiment can be constructed by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method described in Embodiment 1 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), and the method of Embodiment 1 of this application can be implemented. The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the above-mentioned computing device through the computer-readable recording medium, and run therein.

[0140] Example 3:

[0141] As Figure 8 shown, this implementation provides an electronic device, which includes a processor 1001, a communication interface 1002, and a computer-readable storage medium 1003. Among them, the processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 can be connected through a bus or other means.

[0142] Among them, the communication interface 1002 is used to receive and send data. The computer-readable storage medium 1003 can be stored in the memory of the electronic device. The computer-readable storage medium 1003 is used to store a computer program. The computer program includes program instructions. The processor 1001 is used to execute the program instructions stored in the computer-readable storage medium 1003.

[0143] The processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device. It is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function.

[0144] The processor 1001 is configured to execute the following process:

[0145] Preprocess the current frame image to highlight the abrasive particle area;

[0146] Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result;

[0147] According to the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory;

[0148] Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory;

[0149] When the matching is successful, update the prediction trajectory with successful matching. When the matching is not successful, create a new prediction trajectory and continue the matching.

[0150] For the specific working method, see the introduction in Example 1 and will not be elaborated here.

[0151] Example 4:

[0152] This implementation provides a computer-readable storage medium (Memory). A computer-readable storage medium is a memory device in an electronic device, used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the electronic device, and of course can also include the extended storage medium supported by the electronic device. The computer-readable storage medium provides a storage space, and this storage space stores the processing system of the electronic device.

[0153] Moreover, in this storage space, one or more instructions suitable for being loaded and executed by the processor are also stored. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one computer-readable storage medium located far from the aforementioned processor.

[0154] In one embodiment, one or more instructions are stored in the computer-readable storage medium; the processor loads and executes one or more instructions stored in the computer-readable storage medium to implement the following process:

[0155] Preprocess the current frame image to highlight the abrasive particle area;

[0156] Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result;

[0157] According to the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory;

[0158] Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory;

[0159] When the matching is successful, update the prediction trajectory with successful matching; when the matching is unsuccessful, create a new prediction trajectory and continue the matching.

[0160] For the specific working method, please refer to the introduction in Embodiment 1, which will not be elaborated here.

[0161] Embodiment 5:

[0162] This implementation provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and these computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, enabling the electronic device to perform the following process:

[0163] Preprocess the current frame image to highlight the abrasive particle region;

[0164] Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result;

[0165] According to the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory;

[0166] Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory;

[0167] When the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching.

[0168] For the specific working method, refer to the introduction in Embodiment 1 and will not be elaborated here.

[0169] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that the computer can access, or a data processing device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)), etc.

[0171] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An abrasive particle tracking method, characterized in that, It includes the following processes: Preprocess the current frame image to highlight the abrasive particle area; Extract the abrasive particle contours of the preprocessed current frame image to generate abrasive particle detection results; According to the abrasive particle detection results, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory; Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, match the current frame position of the abrasive particle with the prediction trajectory; When the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching; Use the improved YOLOv11 model to extract the abrasive particle target box of the preprocessed current frame image, and combine the abrasive particle target box to extract the contours of the abrasive particles by using the OpenCV method; the improved YOLOv11 model includes: a local branch, a global branch, and a serial convolutional branch; The local branch uses standard convolution to extract local detail features, the global branch extracts global context features through global average pooling or self-attention mechanism, and the serial branch uses deep convolution to further extract high-level semantic features; Calculate the query matrix and the key matrix by using the local detail features and the global context features, and calculate the attention matrix according to the query matrix and the key matrix; After element-wise multiplying the attention matrix with the local detail features, cascade or weighted sum with the high-level semantic features to obtain multi-scale fusion features, and obtain the abrasive particle target box according to the multi-scale fusion features.

2. The abrasive particle tracking method according to claim 1, wherein Generating the abrasive particle detection results includes: Each abrasive grain includes: position information and morphological feature vector , where and are the abscissa and ordinate of the center point of the bounding box respectively, and are the width and height of the bounding box respectively, A represents the target area, P represents the target perimeter, C represents the target roundness, and AR represents the aspect ratio of the target.

3. The abrasive particle tracking method according to claim 1, wherein The weighted sum of the distance of the current frame position and the position information of the predicted trajectory from the distance of the morphological features , including: ； Among them, is the position information distance, is the morphological feature distance, and are weight coefficients.

4. The abrasive particle tracking method according to claim 3, wherein ， ； Among them, is the center point coordinate of the predicted bounding box for the next frame, is the center coordinate of the detected bounding box for the current frame, is the predicted feature, is the detected feature, is the standard deviation of the feature, k represents the k th frame.

5. The abrasive particle tracking method according to claim 1, wherein Determine the matching cost matrix based on the calculated weighted sum, and use the matching cost matrix to solve the optimal matching scheme by using the Hungarian algorithm.

6. An abrasive particle tracking system, characterized in that, It includes: An image preprocessing unit configured to: preprocess the current frame image to highlight the abrasive particle area; An abrasive particle detection unit configured to: extract the abrasive particle contours of the preprocessed current frame image to generate abrasive particle detection results; A trajectory prediction unit configured to: according to the abrasive particle detection results, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory; A trajectory matching unit configured to: based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, match the current frame position of the abrasive particle with the prediction trajectory; A trajectory update unit configured to: when the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching; Use the improved YOLOv11 model to extract the abrasive particle target box of the preprocessed current frame image, and combine the abrasive particle target box to extract the contours of the abrasive particles by using the OpenCV method; the improved YOLOv11 model includes: a local branch, a global branch, and a serial convolutional branch; The local branch uses standard convolutions to extract local detailed features, the global branch extracts global context features through global average pooling or self-attention mechanism, and the serial branch uses deep convolutions to further extract high-level semantic features; Calculate the query matrix and the key matrix using the local detailed features and the global context features, and calculate the attention matrix according to the query matrix and the key matrix; After performing element-wise multiplication of the attention matrix and the local detailed features, concatenate or perform weighted summation with the high-level semantic features to obtain multi-scale fusion features, and obtain the abrasive target box according to the multi-scale fusion features.

7. A computer device, characterized in that, Comprising: A processor and a computer-readable storage medium; The processor is adapted to execute a computer program; The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the abrasive tracking method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the abrasive tracking method according to any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by the processor, the abrasive tracking method according to any one of claims 1 to 5 is implemented.

Images