Abrasive particle tracking method and system, product, medium and computer equipment

Abstract

The invention belongs to the technical field of image processing. The invention provides an abrasive particle tracking method and system, a product, a medium and computer equipment, and the method comprises the steps: preprocessing a current frame image to highlight an abrasive particle region, extracting an abrasive particle contour of the preprocessed current frame image, generating an abrasive particle detection result, and predicting the next frame position of each abrasive particle based on unscented Kalman filtering according to the abrasive particle detection result. Determining a prediction track; based on the current frame position of the abrasive particles and the weighted sum of the position information distance and the morphological feature distance of the predicted trajectory, matching the current frame position of the abrasive particles with the predicted trajectory; and when the matching is successful, updating the successfully matched prediction trajectory, and when the matching is not successful, creating a new prediction trajectory and continuing the matching. According to the invention, by combining a multi-feature matching strategy, non-linear motion trajectory prediction measurement based on unscented Kalman filtering and a parallelized patch perception attention mechanism, the detection and tracking precision of the abrasive particle target is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method, a system, a product, a medium and a computer device for abrasive particle tracking. Background Art

[0002] The statements in this part merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of on-line oil monitoring technology, real-time monitoring of abrasive particles in the lubricating oil of mechanical equipment has become an important means for preventive maintenance and fault warning. The on-line oil monitoring system continuously collects oil samples during the operation of the equipment, and uses image processing and pattern recognition algorithms to quantitatively analyze and classify the tiny abrasive particles in the oil, so as to timely reflect the wear state of the equipment and reduce the risk of downtime. Compared with the traditional off-line detection method, on-line monitoring has the advantages of fast response and high automation.

[0004] In the abrasive particle detection task, in addition to paying attention to its type, size and morphological characteristics, stable tracking of abrasive particles is also a key issue. Since the abrasive particles will tumble as the lubricating oil flows, the projected shape and angle of the abrasive particles in the microscopic image will constantly change. For example, some elliptical or irregularly shaped abrasive particles may exhibit different aspect ratios and areas at different angles, and even present a nearly circular shape at a specific angle. This phenomenon may cause the detection algorithm to misidentify the same abrasive particle as multiple different abrasive particles in different frames, thus affecting the accurate judgment of its morphological characteristics (such as aspect ratio, area, perimeter, etc.). In addition, when multiple abrasive particles with similar shapes flow within the same field of view, the ID exchange problem is likely to occur, that is, the algorithm wrongly assigns the trajectory of one abrasive particle to another abrasive particle with a similar shape, reducing the tracking stability. Therefore, designing a method for cross-frame matching and motion prediction of abrasive particles to ensure the consistency of its ID in the entire video sequence is the key to improving the accuracy of abrasive particle morphological analysis.

[0005] In the field of computer vision, object detection technology has been widely applied to object recognition and tracking tasks. Among them, the YOLO (You Only Look Once) series of object detection models, with their end-to-end single-stage detection architecture, have achieved efficient object recognition and localization. The YOLO model divides the entire image into an S×S grid and predicts the bounding box, confidence, and class probability of the object within each grid, thus completing the detection task in one forward propagation. With the continuous evolution of versions, the detection accuracy and multi-scale object detection ability of the YOLO model have been continuously improved. For example, YOLOv3 has been optimized in multi-scale feature fusion, YOLOv5 has improved the detection speed through a lightweight network structure, and YOLOv11 has further improved the feature extraction network and adopted a more reasonable loss function (such as GIOU loss), making its performance in dense object detection and small object detection more superior.

[0006] However, in the application of online oil fluid monitoring, abrasive particle detection based on traditional YOLO models still faces the following challenges: (1) The problem of small object detection. In high-resolution microscopic images, the size of abrasive particles is usually small and they are densely distributed. Traditional single-scale feature extraction methods are difficult to effectively distinguish adjacent tiny objects, reducing the detection accuracy. (2) The matching problem. Traditional object matching strategies only rely on single morphological features such as area and perimeter for object association. However, the tumbling and rotation of abrasive particles may cause drastic changes in their morphological features in consecutive frames, easily resulting in incorrect matching or ID swapping. (3) The motion model problem. Existing abrasive particle tracking methods mostly adopt a simple Kalman filter uniform motion model. However, due to the non-uniformity of the oil fluid flow rate, the motion trajectory of abrasive particles is often non-linear motion with acceleration, deceleration, or affected by external forces, resulting in large errors in prediction methods based on the linear motion assumption in practical applications. Summary of the Invention

[0007] To solve the deficiencies of the existing technology, the present invention provides an abrasive particle tracking method, system, product, medium, and computer device, which combines a multi-feature matching strategy, non-linear motion trajectory prediction measurement based on unscented Kalman filter, and a parallelized patch-aware attention mechanism, improving the detection and tracking accuracy of abrasive particle targets.

[0008] To achieve the above objectives, the present invention adopts the following technical solutions: In the first aspect, the present invention provides an abrasive particle tracking method.

[0009] An abrasive particle tracking method includes the following processes: Preprocess the current frame image to highlight the abrasive particle area; Extract the abrasive contour of the preprocessed current frame image to generate the abrasive detection result; Based on the abrasive detection result, predict the position of each abrasive in the next frame based on the unscented Kalman filter to determine the prediction trajectory; Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive and the prediction trajectory; When the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching.

[0010] In a second aspect, the present invention provides an abrasive tracking system.

[0011] An abrasive tracking system includes: An image preprocessing unit configured to preprocess the current frame image to highlight the abrasive area; An abrasive detection unit configured to extract the abrasive contour of the preprocessed current frame image to generate the abrasive detection result; A trajectory prediction unit configured to predict the position of each abrasive in the next frame based on the unscented Kalman filter according to the abrasive detection result to determine the prediction trajectory; A trajectory matching unit configured to perform the matching between the current frame position of the abrasive and the prediction trajectory based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive and the position of the prediction trajectory; A trajectory updating unit configured to update the prediction trajectory with successful matching when the matching is successful, and create a new prediction trajectory and continue the matching when the matching is unsuccessful.

[0012] In a third aspect, the present invention provides a computer device, including: a processor and a computer-readable storage medium; The processor is adapted to execute a computer program; The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the abrasive tracking method as described in the first aspect of the present invention.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, and the computer program is adapted to be loaded and executed by a processor to implement the abrasive tracking method as described in the first aspect of the present invention.

[0014] In a fifth aspect, the present invention provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the abrasive tracking method as described in the first aspect of the present invention.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The present invention introduces a parallel patch-aware attention (PPA) module to optimize small target detection. In the feature extraction network of YOLOv11, the PPA module is embedded. This module extracts multi-scale features through the collaborative action of local branches, global branches, and serial convolution branches. This design enhances the detection ability of tiny abrasive particles, ensures the retention of key information during multiple downsampling processes, and improves the detection accuracy.

[0016] 2. The present invention proposes an optimization matching strategy for the morphological feature extraction method based on OpenCV. OpenCV is used to directly extract morphological features such as the area, perimeter, roundness, and aspect ratio of abrasive particles for abrasive particle matching. This not only solves the problem of inaccurate matching caused by the morphological changes of abrasive particles when only relying on the area and perimeter of abrasive particles for matching, but also, compared with the method of training an abrasive particle re-identification model using ResNet50, this strategy significantly improves the real-time performance while ensuring the matching accuracy, and ensures accurate and rapid matching of the same abrasive particle between different frames.

[0017] 3. The present invention introduces the unscented Kalman filter (UKF) to improve motion prediction. The UKF is used to model the non-linear motion of abrasive particles, and the state transition equation is used to predict the position of the abrasive particle in the next frame, enhancing the accuracy of motion estimation and improving the reliability of abrasive particle tracking.

[0018] The advantages of the additional aspects of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0020] Figure 1 It is a schematic diagram of the overall process of the abrasive particle tracking method provided in Embodiment 1 of the present invention; Figure 2 It is a schematic diagram of the structure of the parallel patch-aware attention module (PPA) provided in Embodiment 1 of the present invention; Figure 3 It is a schematic diagram of the loss function of model training provided in Embodiment 1 of the present invention; wherein, (A) is the training set bounding box regression loss, (B) is the training set classification loss, (C) is the training set distribution regression loss, (D) is the validation set bounding box regression loss, (E) is the validation set classification loss, and (F) is the validation set distribution regression loss; Figure 4Schematic diagram of the accuracy of model training provided in Embodiment 1 of the present invention. Among them, (A) is the schematic diagram of precision, (B) is the schematic diagram of recall rate, (C) is the schematic diagram of mAP50, and (D) is the schematic diagram of mAP50-95; Figure 5 Relationship curve of precision and recall rate of the validation set data provided in Embodiment 1 of the present invention; Figure 6 Schematic diagram of the abrasive particle target tracking result provided in Embodiment 1 of the present invention; Figure 7 Schematic diagram of an abrasive particle tracking system provided in Embodiment 2 of the present invention; Figure 8 Schematic diagram of a computer device provided in Embodiment 3 of the present invention. Detailed implementation manners

[0021] The present invention will be further described below in conjunction with the drawings and embodiments.

[0022] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0023] Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0024] Embodiment 1: This implementation proposes an abrasive particle tracking method. Based on the YOLOv11 model, this method combines a multi-feature matching strategy, a non-linear motion model based on the Unscented Kalman Filter (UKF), and a Parallelized Patch-Aware Attention (PPA) mechanism to improve the detection and tracking accuracy of abrasive particle targets. As Figure 1 shown, first, an abrasive particle dataset is constructed, and the YOLOv11 model is trained. Feature extraction, feature fusion, coordinate prediction, loss calculation, and model training are performed through the YOLOv11 model; for the abrasive particle video to be detected, the trained YOLOv11 model is used to determine the detection results (coordinates and confidence), and the detection results are input into the tracking model. The tracking model is used for motion prediction, data association and matching, and trajectory update to obtain the tracking results, thereby realizing the real-time tracking of abrasive particles.

[0025] More specifically, it includes the following processes: S1: Data preprocessing and dataset construction.

[0026] S11: Extract each frame from the input video and preprocess each frame image (such as denoising, contrast enhancement, and size normalization to a unified size of 2592×1944 pixels). S12: Use the LabelImg tool to construct an abrasive particle image dataset, annotate the abrasive particle bounding box and category information, etc., and construct a dataset in the YOLO format. The specific format is [class_id, x, y, w, h], which respectively represent the object category, the horizontal and vertical coordinates (x, y) of the center point of the bounding box, the size of the bounding box (w, h), and the upper left corner of the image is the coordinate origin.

[0027] S13: Divide the constructed dataset into three categories: training set, validation set, and test set according to the ratio of 8:1:1.

[0028] S2: Train the YOLOv11 model.

[0029] S21: Improve the YOLOv11 model for the abrasive particle detection task.

[0030] (1) Insert a parallel patch-aware attention (PPA) module (as shown in Figure 3 ) into the middle layer of the YOLOv11 network. The PPA can significantly improve the feature expression ability of small targets by simultaneously capturing local details and global background information, enabling the network to more accurately focus on fine target regions, thereby enhancing the accuracy and robustness of small target detection. More specifically, the middle layer of the original YOLOv11 network consists of four consecutive convolutions. By adding two PPA modules (each including serial tensor unfolding, weighting, feature selection, and folding / reshape), it enters the attention module. The attention module includes the processes of fused output, calculating attention, weighted fusion, and outputting enhanced features.

[0031] In this implementation, non-overlapping Patch partitioning is performed on the input feature map F (with size ). Each Patch size is , defined as . Subsequent processing is performed on each small block, and the processed results are stitched and fused.

[0032] (2) Construct a local branch, a global branch, and a serial convolution branch.

[0033] The local branch uses standard convolutions to extract local detail features , the global branch extracts global context features through global average pooling or self-attention mechanisms , and the serial branch uses deep convolutions to further extract high-level semantic features .

[0034] (3) Calculate the attention matrix using the outputs of the local and global branches , first obtain the query matrix Q and the key matrix K through learnable matrix transformation: (1); Among them, and are learnable parameter matrices, and then calculate the attention matrix ; (2); Among them, is the scaling factor (usually taken as ).

[0035] (5) After applying the attention matrix to the local branch features, concatenate or perform weighted summation with the high-level semantic features extracted by the serial branch to form the final multi-scale fusion feature: (3); Among them, represents element-wise multiplication, represents the fusion operation (for example, it can be concatenated and then convolved or directly perform weighted summation), and this fusion feature will be used as the input of the subsequent detection layer, significantly improving the detection ability for tiny abrasive particles.

[0036] S22: During the training process, an integrated loss function is adopted, including the bounding box regression loss, the classification loss, and the matching and motion prediction error. Among them, the bounding box regression loss uses the GIOU loss, and its definition is: (4); Among them, IOU is the intersection over union, that is, the ratio of the intersection area of two bounding boxes to the union area, which is an index in object detection to measure the overlap degree between the predicted box and the ground truth box. GIOU is the generalized intersection over union, which introduces a penalty term for the minimum closed region of the two boxes on the basis of IOU, avoiding the value of GIOU being 0 when the intersection area of the two boxes is 0.

[0037] As Figure 3 and 4 show, the bounding box regression loss, the classification loss, and the distribution regression loss all decrease continuously with the increase of the training epoch, indicating that the bounding box regression accuracy and classification accuracy of the model on the training set are gradually improving; at the same time, the bounding box regression loss, the classification loss, and the distribution regression loss on the validation set also basically show a decreasing trend, and there is no significant deviation from the training loss curve, indicating that the performance of the model on the validation set has a similar convergence law to the training set.

[0038] Meanwhile, both the training and validation losses converge to a relatively low level in the later stage, and the curves are stable without significant rebounds, indicating that the model has not suffered from severe overfitting. The curves of precision and recall can clearly reflect that the ability of the object detection model to determine positive and negative samples continuously improves with the progress of training; while mAP50 (mean Average Precision at IoU = 50%, that is, the average precision of the model on all classes when the IoU threshold is 50%) and mAP50-95 (mean Average Precision at IoU = 50%-95%, that is, the average precision of the model on all classes in the range where the IoU threshold increases from 50% to 95% (step size 0.05)) respectively reflect the average precision performance of the model when the GIOU threshold is 0.5 and in the range from 0.5 to 0.95. It can be observed that all metrics steadily increase during the training process and tend to converge in the later stage, indicating that the overall detection performance of the model has reached a certain stable level.

[0039] S23: The Adam optimizer is adopted, and the initial learning rate is set to , the number of training epochs is set to 100, and hyperparameters such as the learning rate and weight decay are adjusted according to the performance on the validation set until the model converges.

[0040] As Figure 5 shown, it is the P-R curve of the validation set data of the present invention, which shows the relationship between the precision and recall of the model, indicating that the present model still maintains a high precision at a high recall rate.

[0041] S3: Build a Deepsort object tracking model, train the abrasive particle feature extraction network, and take the detection boxes of each frame of YOLOv11 as the input to complete the construction of the YOLOv11-Deepsort abrasive particle tracking model.

[0042] S31: Feature extraction.

[0043] Directly adopt the OpenCV method to extract morphological features such as the area, perimeter, roundness, and aspect ratio of abrasive particles for abrasive particle matching. Compared with the method of training an abrasive particle re-identification model with ResNet50, it has higher real-time performance while ensuring the matching accuracy, ensuring accurate and fast matching of the same abrasive particle between different frames.

[0044] Use the trained YOLOv11 model to extract the target boxes of each frame of the video; Convert the image of each frame into a grayscale image, and then use filtering, binarization (such as Otsu threshold or adaptive threshold), and morphological operations (such as opening operation and closing operation) to remove noise; The contours of all abrasive grains in the binary image are extracted by the OpenCV method, and for the abrasive grain contours detected in each frame, multi-dimensional morphological feature vectors are extracted. : (5); Among them, represents the area of the target abrasive grain; represents the perimeter of the target abrasive grain; represents the roundness of the target abrasive grain, defined as ; represents the aspect ratio of the target abrasive grain, defined as .

[0045] The detection results are generated. Each abrasive grain not only has position information (such as the bounding box [x, y, w, h]), but also has a morphological feature vector , and these information are packed into a detection dictionary Z and passed to the subsequent Deepsort (Deep Simple Online and Realtime Tracking, multi-object tracking based on deep learning) module: (6); S32: Motion prediction.

[0046] The unscented Kalman filter UKF is used to predict the position of each abrasive grain in the next frame, providing a prior for data association to make the motion estimation more accurate. The UKF state transition equation is: (7); Among them, is the state transition function, is the state of the current frame , is the control input, is the noise of the current frame , is the state of the next frame.

[0047] (1) A motion prediction module based on UKF is constructed for the possible non-linear motion characteristics of abrasive grains in the video; (2) The target state vector is defined as: (8); Among them, is the target center position, is the target speed; (3) Establish a state transition model: UKF is used to handle non-linear state transitions, and its core process is as follows: (9); The specific form is as follows: (10); (11); (12); (13); Wherein: represents the time interval between each frame in the video, represents the current frame of the target in the direction of the speed, and is the acceleration component, is the process noise, represents the current frame of the target in the x direction of the speed.

[0048] (4) Establish an observation model : (14); Usually set , is the observation noise, is the state vector of the target box at the k frame; and are the center coordinates of the target box at the k frame.

[0049] (5) Use UKF for state prediction and update, calculate the predicted mean and covariance using non - linear transformation and sampling points to ensure accurate estimation of the abrasive motion state under actual complex flow fields.

[0050] S33: Data association and matching: Combine motion and appearance information, perform optimal matching through the Hungarian algorithm, optimize the cross - frame association of targets, and achieve multi - target tracking. In the data association process of Deepsort, it is necessary to construct a matching cost matrix by combining the predicted state and the current detection result.

[0051] For each predicted trajectory (state obtained from UKF prediction) and the current frame detection result (morphological feature vector Z), define the comprehensive distance: (15); Wherein: is the distance of position information, represents the th existing target, that is, the target that has been tracked and predicted by UKF in the previous few frames, Indicates the th target detected in the current frame. If the center coordinates of the predicted bounding box in the next frame are , and the center coordinates of the detected bounding box in the current frame are , then the Euclidean distance is adopted: (16); is the morphological feature distance. For the morphological feature part, the normalized Euclidean distance is adopted: (17); Among them, is the predicted feature, is the detected feature, is the standard deviation of this feature, used for normalization, and the coefficients and control the relative weights of the motion and morphological information.

[0052] Using the matching cost matrix , the Hungarian algorithm is adopted to solve the optimal matching scheme (the Hungarian algorithm finds the one-to-one correspondence between each trajectory and the detected target in the matching cost matrix, aiming to minimize the total cost, and finally outputs an optimal allocation scheme that minimizes the total matching cost), ensuring the minimum overall matching cost. If the matching cost of a certain detection result after being matched by the Hungarian algorithm with a trajectory is lower than the set threshold, then this match is judged to be successful; the matching result determines whether the current detection result is associated with the predicted trajectory. For the detection results that are not successfully matched, new trajectories are created (create a new matrix containing the appearance features and motion conditions of the undetected successful targets); if a certain existing trajectory has not been matched with any detection result for multiple consecutive frames, it indicates that the target represented by this trajectory may have disappeared. To maintain the accuracy of the tracking system and prevent false tracking, this trajectory is deleted.

[0053] S34: UKF trajectory update.

[0054] Update the successfully matched trajectories (since the features such as the appearance and speed of the abrasive grains will change, so it is necessary to update their change sequences), create new trajectories, and delete the trajectories that have not been matched for a long time. Through the measurement matrix (usually a linear mapping that maps the state vector to the measurement space) and the predicted covariance calculate: (18); Among them, is the measurement noise covariance matrix.

[0055] Let the detection measurement be z, and update the state: (19); Wherein, is the position of the target object in the predicted next frame, is the position of the target object in the updated next frame.

[0056] And update the covariance: (20); Wherein, is the predicted covariance, is the covariance of the k th frame.

[0057] In this way, each tracked target can obtain a more accurate state estimate according to the new detection results. As Figure 6 shown, it is a schematic diagram of the tracking result of abrasive particles in an actual video tracking scenario. The figure mainly includes: a detection box with an id of 179 and a confidence level of 0.66, a detection box with an id of 158 and a confidence level of 0.62, a detection box with an id of 288 and a confidence level of 0.27, a detection box with an id of 306 and a confidence level of 0.35, a detection box with an id of 87 and a confidence level of 0.47, a detection box with an id of 218 and a confidence level of 0.49, a detection box with an id of 152 and a confidence level of 0.64, a detection box with an id of 216 and a confidence level of 0.54, a detection box with an id of 214 and a confidence level of 0.61, and a detection box with an id of 282 and a confidence level of 0.58.

[0058] In this implementation, the above model is used to track abrasive particles in the video. After obtaining the position information of each abrasive particle in each frame, a unique identifier is assigned to each abrasive particle. The center of the detection box is used to replace the abrasive particle, and the trajectory of the abrasive particle with the same ID over time is drawn. A CSV file is created as the trajectory record file for subsequent analysis. At the same time, in this implementation, the shape features of the same abrasive particle in different poses are also extracted, and the area, perimeter, roundness, aspect ratio, etc. of each frame are calculated to track the morphological changes of the abrasive particle with the same ID in the time dimension and analyze the change trend of the abrasive particle morphology. By adopting the above technical solutions, the present invention realizes high-precision and real-time tracking of abrasive particle targets in the context of on-line oil monitoring. The improved modules cooperate with each other and show obvious advantages in target matching, motion prediction, and small target detection, providing effective technical support for actual industrial monitoring.

[0059] Embodiment 2: As Figure 7 shown, this implementation provides an abrasive particle tracking system, including: An image preprocessing unit configured to preprocess the current frame image to highlight the abrasive particle area; The abrasive particle detection unit is configured to extract the abrasive particle contour of the preprocessed current frame image and generate an abrasive particle detection result; The trajectory prediction unit is configured to, according to the abrasive particle detection result, predict the position of each abrasive particle in the next frame based on the unscented Kalman filter and determine the predicted trajectory; The trajectory matching unit is configured to match the current frame position of the abrasive particle with the predicted trajectory based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position information of the predicted trajectory; The trajectory update unit is configured to update the predicted trajectory with which the matching is successful when the matching is successful, and create a new predicted trajectory and continue the matching when the matching is not successful.

[0060] For the specific working methods of the above units, see the introduction in Embodiment 1 and will not be elaborated here.

[0061] It can be understood that the above units can be respectively or all combined into one or several other units to form, or some of them can be further split into multiple smaller units with functional division to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the system may also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0062] According to another embodiment of the present application, the system described in this embodiment can be constructed and the method of Embodiment 1 of the present application can be realized by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method described in Embodiment 1 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM). The computer program can be recorded on a computer-readable recording medium, loaded into the above computing device through the computer-readable recording medium, and run therein.

[0063] Embodiment 3: As Figure 8As shown in the figure, this implementation provides an electronic device, which includes a processor 1001, a communication interface 1002, and a computer-readable storage medium 1003. Among them, the processor 1001, the communication interface 1002, and the computer-readable storage medium 1003 can be connected through a bus or other means.

[0064] Among them, the communication interface 1002 is used to receive and send data. The computer-readable storage medium 1003 can be stored in the memory of the electronic device. The computer-readable storage medium 1003 is used to store computer programs. The computer programs include program instructions. The processor 1001 is used to execute the program instructions stored in the computer-readable storage medium 1003.

[0065] The processor 1001 (or CPU (Central Processing Unit, central processing unit)) is the computing core and control core of the electronic device, which is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function.

[0066] The processor 1001 is configured to execute the following process: Preprocess the current frame image to highlight the abrasive particle area; Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result; According to the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory; Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory; When the matching is successful, update the prediction trajectory with successful matching. When the matching is unsuccessful, create a new prediction trajectory and continue the matching.

[0067] For the specific working method, please refer to the introduction in Embodiment 1, which will not be elaborated here.

[0068] Embodiment 4: This implementation provides a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in the electronic device, which is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the electronic device and, of course, the extended storage medium supported by the electronic device. The computer-readable storage medium provides a storage space, and this storage space stores the processing system of the electronic device.

[0069] Also, one or more instructions suitable for being loaded and executed by a processor are stored in this storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one computer-readable storage medium located far from the aforementioned processor.

[0070] In one embodiment, one or more instructions are stored in the computer-readable storage medium; the processor loads and executes one or more instructions stored in the computer-readable storage medium to implement the following process: Preprocess the current frame image to highlight the abrasive particle area; Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result; Based on the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory; Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory; When the matching is successful, update the prediction trajectory with successful matching; when the matching is not successful, create a new prediction trajectory and continue the matching.

[0071] For the specific working method, see the introduction in Embodiment 1 and will not be elaborated here.

[0072] Embodiment 5: This implementation provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the electronic device to perform the following process: Preprocess the current frame image to highlight the abrasive particle area; Extract the abrasive particle contour of the preprocessed current frame image to generate an abrasive particle detection result; Based on the abrasive particle detection result, predict the next frame position of each abrasive particle based on the unscented Kalman filter to determine the prediction trajectory; Based on the weighted sum of the position information distance and the morphological feature distance between the current frame position of the abrasive particle and the position of the prediction trajectory, perform the matching between the current frame position of the abrasive particle and the prediction trajectory; When the matching is successful, update the prediction trajectory with successful matching; when the matching is not successful, create a new prediction trajectory and continue the matching.

[0073] For the specific working method, see the introduction in Embodiment 1 and will not be elaborated here.

[0074] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this application can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0075] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data processing device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0076] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, various changes and modifications can be made to the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A wear particle tracking method, characterized in that: The process includes: Preprocessing the current frame image to highlight the wear particle area; Extracting the wear particle contour of the preprocessed current frame image and generating the wear particle detection result; According to the wear particle detection result, predict the next frame position of each wear particle based on the unscented Kalman filter to determine the predicted trajectory; Matching the current frame position of the abrasive particle with the predicted trajectory based on the weighted sum of the position information distance and the morphological feature distance of the current frame position of the abrasive particle and the predicted trajectory; When the match is successful, the predicted trajectory of the successful match is updated. When the match is unsuccessful, a new predicted trajectory is created and the matching continues.

2. The wear tracking method according to claim 1, wherein: The improved YOLOv11 model is used to extract the wear particle target frame of the preprocessed current frame image, and the wear particle contour is extracted by the OpenCV method in combination with the wear particle target frame; the improved YOLOv11 model includes: a local branch, a global branch and a serial convolution branch; The local branch uses standard convolution to extract local detail features, the global branch extracts global context features through global average pooling or self-attention mechanism, and the serial branch uses deep convolution to further extract high-level semantic features; Calculate a query matrix and a key matrix using local detail features and global context features, and calculate an attention matrix based on the query matrix and the key matrix; After element-wise multiplication of the attention matrix and the local detail feature, the matrix is ​​cascaded or weighted summed with the high-level semantic feature to obtain a multi-scale fusion feature, and the wear particle target frame is obtained according to the multi-scale fusion feature.

3. The wear tracking method according to claim 1 or 2, characterized in that: Generates wear particle inspection results including: Each abrasive particle, including: location information and the morphological feature vector ,in, and are the horizontal and vertical coordinates of the center point of the bounding box, and are the width and height of the bounding box, A represents the target area, P represents the target perimeter, C represents the target roundness, and AR represents the aspect ratio of the target.

4. The wear tracking method according to claim 1 or 2, wherein: The weighted sum of the position information distance and morphological feature distance of the current frame position and the predicted trajectory ,include: ； in, is the location information distance, is the morphological feature distance, and is the weight coefficient.

5. The wear tracking method according to claim 4, wherein: ， ； in, is the coordinate of the center point of the prediction box of the next frame, is the center coordinate of the detection box of the current frame, For predicting features, To detect features, is the standard deviation of the feature, k Representative k frame.

6. The wear tracking method according to claim 1 or 2, wherein: A matching cost matrix is ​​determined based on the calculated weighted sum, and the matching cost matrix is ​​used to solve the optimal matching solution using the Hungarian algorithm.

7. A wear particle tracking system, characterized in that: include: The image preprocessing unit is configured to: preprocess the current frame image to highlight the wear particle area; The wear particle detection unit is configured to: extract the wear particle contour of the preprocessed current frame image and generate a wear particle detection result; A trajectory prediction unit is configured to: predict the next frame position of each wear particle based on the wear particle detection result and unscented Kalman filter to determine a predicted trajectory; The trajectory matching unit is configured to: match the current frame position of the abrasive particle with the predicted trajectory based on the current frame position of the abrasive particle and a weighted sum of the position information distance and the morphological feature distance of the predicted trajectory; The trajectory updating unit is configured to: when the matching is successful, update the predicted trajectory of the successful matching; when the matching is unsuccessful, create a new predicted trajectory and continue matching.

8. A computer device, characterized in that: include: A processor and a computer readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the wear tracking method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the wear tracking method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the wear tracking method according to any one of claims 1 to 6 is implemented.

Images