Motion abrasive particle detection and tracking method based on fusion Gaussian mixture model and watershed algorithm

Abstract

The invention discloses a moving abrasive particle detection and tracking method based on fusion of a Gaussian mixture model (GMM) and a watershed algorithm, which is characterized in that on the basis of acquiring abrasive particle moving targets based on the Gaussian mixture model (GMM), each extracted target is segmented by using the watershed algorithm based on distance transformation, so that the adhered abrasive particle targets can be accurately detected, and the detection accuracy is improved. Algorithm verification is carried out on abrasive particle videos in different stages, the detection accuracy of the abrasive particle adhesion target can reach 98.86%, the number, size and other characteristics of abrasive particles can be accurately extracted, and abrasion faults can be analyzed easily. In addition, on the basis of accurate detection of the adhered abrasive particle target, the SORT algorithm is further applied to track the multi-target abrasive particles in real time, and the result shows that the tracking effect of the adhered abrasive particle target is good.

Description

Technical Field

[0001] This invention relates to the field of oil abrasive monitoring technology, and in particular to a method for detecting and tracking moving abrasive particles based on a fusion Gaussian mixture model and a watershed algorithm. Background Technology

[0002] In modern industry, machinery and equipment are becoming increasingly complex and intelligent. To ensure their safe operation, the technical requirements for equipment condition monitoring are becoming increasingly stringent. Statistics show that over 50% of serious mechanical equipment failures are caused by lubrication failure and excessive friction. Oil wear particles are the direct product of surface wear on friction pairs of mechanical parts, containing rich wear information and serving as an effective basis for reflecting the wear condition of mechanical equipment. [2] The size and concentration of abrasive particles can reflect the degree of wear on mechanical equipment. The material of the abrasive particles can identify the wear location, thus determining the location of mechanical equipment failure. The color and morphological characteristics of the abrasive particles are key factors in inferring the type and cause of wear. Therefore, oil abrasive particle monitoring is an important means of achieving equipment maintenance and fault early warning.

[0003] Compared to oil abrasive monitoring technologies based on inductance and electrostatics, abrasive particle images provide a more intuitive view of abrasive particle features. Intelligent image processing techniques can extract numerous features from abrasive particle images, such as particle size, concentration, color, and morphology, enabling the simultaneous determination of the degree, location, and cause of equipment wear. However, abrasive particle adhesion is a common problem in abrasive particle images. For example, in ferrographic images, abrasive particles exhibit a chain-like distribution due to magnetic attraction, with significant overlap, making it impossible to study the morphological characteristics and type of individual particles.

[0004] To address the issues of abrasive grain overlap and deposition chains, existing research proposes a fusion segmentation algorithm that first utilizes a watershed algorithm for initial region segmentation, followed by clustering to merge over-segmented regions. This fusion combines watershed and ant colony clustering, enabling automatic segmentation of abrasive grain deposition chains and abnormally large abrasive grains. Other studies have proposed using color features (Lab) and texture features (LBP) as quantification standards based on watershed segmentation, further merging similar regions according to certain fusion criteria. This achieves a fusion watershed algorithm and a region similarity merging segmentation algorithm for abrasive grain segmentation. However, due to the special imaging conditions and abrasive grain overlap, ferrographic images are extremely complex, and the accuracy of traditional watershed and clustering-based segmentation algorithms is generally low. To address the overlap problem in static abrasive grain images, some research has introduced microfluidic technology to build a moving abrasive grain image acquisition system. This simplifies the acquisition process, acquiring dynamic images during abrasive grain movement, fundamentally avoiding abrasive grain accumulation. Furthermore, background subtraction is used to extract foreground moving abrasive grain targets, achieving abrasive grain target detection. However, when there are a large number of abrasive particles in the microchannel, some of the abrasive particles in motion will still stick together, which will cause the problem of false detection of the sticky abrasive particle targets, thus affecting the extraction of features such as the number and morphology of abrasive particles. Summary of the Invention

[0005] Therefore, based on the above background, this invention proposes a method for detecting and tracking moving abrasive particles based on a fusion of Gaussian mixture model and watershed algorithm to address the false detection of moving and sticking abrasive particles. This invention obtains the moving abrasive particle targets based on Gaussian mixture model (GMM), and then uses the watershed algorithm based on distance transform to segment the extracted targets, enabling accurate detection of sticking abrasive particle targets. Furthermore, the SORT algorithm is used to track multiple target abrasive particles in real time, ultimately enabling more accurate detection and tracking of moving abrasive particle targets. This provides a new approach for further development of online accurate monitoring technology for oil abrasive particles.

[0006] The technical solution provided by this invention is as follows:

[0007] A method for detecting and tracking moving abrasive particles based on a fusion of Gaussian mixture model and watershed algorithm, characterized by comprising the following steps:

[0008] S1: Motion abrasive target detection based on Gaussian mixture model

[0009] S1.1: Background model estimation: Perform background model estimation for each frame of the abrasive video;

[0010] The series of pixel values ​​at each coordinate point in each frame of an image that change over time can be viewed as a random process {I1,...,I...} tFurthermore, it is assumed that the random process satisfies the superposition of K Gaussian distributions. Therefore, for a pixel I at a certain coordinate point in the image at time t... t The probability density function can be expressed as:

[0011]

[0012] Where K is the number of Gaussian distributions, N i It is the i-th Gaussian distribution at time t, ω i,t ,μ i,t and σ i,t 2 N are respectively i The weights, mean, and variance;

[0013] S1.2: For all pixels I in each frame of the image t Match (x,y) with the background model;

[0014] S1.3: Model parameter update: Update the background model using the current image;

[0015] If pixel I t If (x,y) matches the i-th Gaussian distribution in the background model, then the weights, mean, and method of that Gaussian distribution are updated as follows:

[0016] ω i,t =(1-α)ω i,t-1 +α (3)

[0017] μ i,t =(1-ρ)ω i,t-1 +ρI t (x,y) (4)

[0018] σ i,t 2 =(1-ρ)σ i,t-1 2 +ρ(I t -μ i,t-1 ) 2 (5)

[0019] Where α is the background template update rate, α∈[0,1], and its magnitude affects the sensitivity of the model to the occurrence of moving pixels, ρ=α / ω i,t-1 ;

[0020] If the pixel does not match the background model, then use I t The Gaussian distribution with the mean (x,y) is replaced with the Gaussian distribution with the lowest weight;

[0021] S1.4: Foreground abrasive target extraction

[0022] An adaptive background model is obtained from S1.1 to S1.3. The current image and the background model are then matched to obtain the abrasive particle target detection result, which is a binary image.

[0023] S2: Segmentation of Adhesive Abrasive Particles Based on Watershed Algorithm

[0024] S2.1: Perform distance transformation on the binary image of the abrasive target detection result in step S1: calculate the Euclidean distance from all non-zero pixels in the binary image to the nearest zero pixel, and use this distance as the gray value of the pixel to obtain a distance-transformed image that can intuitively show the distance distribution from each pixel in the original binary image to the abrasive boundary.

[0025] S2.2: Set an appropriate threshold for the distance transform image and extract the marked regions of each abrasive grain target;

[0026] S2.3: The watershed algorithm is used to automatically segment the adhering abrasive particles using the marked area obtained in step S2.2;

[0027] S2.4: Use the position and contour of each segmented region to draw the target bounding box of each abrasive grain, and obtain the center position and size of the target detection box;

[0028] S3: Tracking of Moving Abrasive Targets

[0029] S3.1: Kalman Filter Prediction

[0030] In this step, the Kalman filter prediction is divided into two stages: prediction and update.

[0031] S3.1.1 First, define the state variable X of the abrasive target in the k-th frame image. k Sum of observed variables Z k for:

[0032] X k =[x,y,s,r,dx,dy,ds,dr] T (6)

[0033] Z k =[x,y,s,r] T (7)

[0034] Where (x,y) are the center coordinates of the abrasive target box, s and r are the area and aspect ratio of the bounding box, respectively, and dx,dy,ds and dr are the derivatives of the state components with time k.

[0035] S3.1.2: Prediction Phase

[0036] First, verify the estimated value using the state prediction equation.

[0037]

[0038] Where A is the state transition matrix, W k-1 This is process noise;

[0039] S3.1.2: Update Phase

[0040] The optimal estimate is obtained from the state update equation.

[0041]

[0042] Where H is the state observation matrix, Z k Let X be the observed value. K is the Kalman gain, which makes the true value X equal to the observed value. k and optimal estimate The error between them is minimal;

[0043] S3.2: Data Association

[0044] Preferably, the number of Gaussian distributions in step S1.1 is 3-5.

[0045] Preferably, pixel matching in step 1.2 is |I t (x,y)-μ i,t |<2.5σ i,t 2 (2) If the pixel matches the i-th Gaussian distribution, it means that the pixel matches the background model and is therefore determined to be a background pixel; otherwise, the pixel is determined to be a particle target pixel.

[0046] Preferably, in step S3.2, the IOU cost matrix of the predicted bounding box and the detection bounding box is input into the Hungarian algorithm to obtain the optimal matching result of the predicted bounding box and the detection bounding box. The formula for calculating IOU is:

[0047]

[0048] Among them, S A and S B These are the areas of the predicted bounding box and the detected bounding box, respectively.

[0049] The beneficial effects of adopting the above technical solution are as follows:

[0050] This invention, based on Gaussian Mixture Model (GMM) to obtain moving abrasive targets, further utilizes a watershed algorithm based on distance transform to segment the extracted targets, enabling accurate detection of adherent abrasive targets. Algorithm verification through abrasive video at different stages shows an accuracy rate of 98.86% for detecting adherent abrasive targets. It accurately extracts features such as the number and size of abrasive particles, facilitating wear fault analysis. Furthermore, building upon accurate detection of adherent abrasive targets, this invention further employs the SORT algorithm for real-time tracking of multiple abrasive targets, demonstrating excellent tracking performance for adherent abrasive targets. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 The results of motion abrasive target detection based on GMM;

[0053] Figure 2 The results of motion-adhesive abrasive particle target detection based on GMM;

[0054] Figure 3 To separate the abrasive particles that are stuck together during motion;

[0055] Figure 4 This is a schematic diagram of the IOU model;

[0056] Figure 5 Results of tracking moving abrasive particles;

[0057] Figure 6 This is a schematic diagram of the structure of a motion abrasive particle image acquisition platform;

[0058] Figure 7 The number of abrasive grains per frame of image;

[0059] Figure 8 The accuracy of abrasive target detection for each frame of image. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] The invention will now be further described with reference to the accompanying drawings.

[0062] The specific technical solution of the present invention is as follows:

[0063] This invention first detects moving abrasive particles based on a fusion of Gaussian mixture model and watershed algorithm. The specific operation steps are as follows:

[0064] S1: Motion abrasive target detection based on GMM

[0065] S1.1: Background model estimation: Perform background model estimation for each frame of the abrasive video;

[0066] The series of pixel values ​​at each coordinate point in each frame of an image that change over time can be viewed as a random process {I1,...,I...} t Furthermore, it is assumed that this random process satisfies the superposition of K Gaussian distributions. Typically, 3 to 5 Gaussian distributions are sufficient to describe the changes in complex scenes. Therefore, for a pixel I at a certain coordinate point in the image at time t... t The probability density function can be expressed as:

[0067]

[0068] Where K is the number of Gaussian distributions, N i It is the i-th Gaussian distribution at time t, ω i,t ,μ i,t and σ i,t 2 N are respectively i The weights, mean, and variance;

[0069] S1.2: For all pixels I in each frame of the image t Match (x,y) with the background model;

[0070] Pixel matching is |I t (x,y)-μ i,t |<2.5σ i,t 2 (2)

[0071] If a pixel matches the i-th Gaussian distribution, it means that the pixel matches the background model and is therefore considered a background pixel; otherwise, it is considered a particle target pixel.

[0072] S1.3: Model parameter update: Update the background model using the current image;

[0073] If pixel I t If (x,y) matches the i-th Gaussian distribution in the background model, then the weights, mean, and method of that Gaussian distribution are updated as follows:

[0074] ω i,t =(1-α)ω i,t-1 +α (3)

[0075] μ i,t =(1-ρ)ω i,t-1 +ρI t (x,y) (4)

[0076] σ i,t 2 =(1-ρ)σ i,t-1 2 +ρ(I t -μ i,t-1 ) 2 (5)

[0077] Where α is the background template update rate, α∈[0,1], and its magnitude affects the sensitivity of the model to the occurrence of moving pixels, ρ=α / ω i,t-1 ;

[0078] If the pixel does not match the background model, then use I t The Gaussian distribution with the mean (x,y) is replaced with the Gaussian distribution with the lowest weight;

[0079] S1.4: Foreground abrasive target extraction

[0080] The adaptive background model is obtained from steps S1.1 to S1.3. The current image and the background model are then matched. If a match is found, the point belongs to the background, and its pixel value is set to 0. Otherwise, the point belongs to the foreground target, and its pixel value is set to 255. This yields the abrasive target detection result, which is a binary image where the black areas represent the background and the white areas represent the abrasive targets. Figure 1 Show.

[0081] Figure 1 The results of the detection of moving abrasive targets, among which Figure 1 a, d, and g are the original images of the moving abrasive particles. Figure 1 b, e, and h are respectively Figure 1The abrasive target detection results for a, d, and g are shown. Based on the detection results, features such as the centroid position, contour, and size of the abrasive particles can be obtained. The circumscribed rectangle of each abrasive particle is drawn on the original image based on its centroid position and contour. The final target detection result is shown below. Figure 1 As shown in c, f, and i.

[0082] Abrasive grains flow at different heights within the microchannels. The abrasive grains in the lower layers are blocked by the upper layer of lubricating oil, so... Figure 1 The comparison of a, d, and g clearly shows that... Figure 1 The abrasive grains in d are lighter in color, closer to the background color, which increases the difficulty of detection. However, the detection results show that... Figure 1 From e and f, we can see that abrasive particles close to the background color can still be accurately detected. On the other hand, from Figure 1 From b and c, we can see that Figure 1 Even smaller abrasive particles in microchannels can be accurately detected. This demonstrates that the GMM algorithm can be used for moving abrasive particle target detection tasks and has high robustness to abrasive particles of different sizes and heights in microchannels.

[0083] Figure 1 g is the 3473rd frame of the abrasive video. Figure 2 'a' is the 3507th frame of the same video. Figure 1 In the image g, the two abrasive grains on the right are close together but not in contact. As the two abrasive grains continue to roll in the microchannel, the distance gradually decreases, and they eventually stick together, as shown in the image. Figure 2 As shown in a. From Figure 1 As can be seen from h and i, two abrasive grains can still be correctly detected when they are close together. However, when two abrasive grains are stuck together, the GMM algorithm will misdetect the two stuck abrasive grains as the same abrasive grain. Figure 2 As shown in b and c, target detection errors not only affect the statistical count of abrasive particles, but also, as the input for target tracking, the accuracy of target detection is a crucial factor affecting target tracking. Therefore, to ensure the detection accuracy of adherent abrasive particles, it is necessary to further segment the targets detected by the GMM algorithm.

[0084] S2: Segmentation of Adhesive Abrasive Particles Based on Watershed Algorithm

[0085] The watershed algorithm is a morphological segmentation method based on topological theory, and researchers have successfully applied it to the field of traditional ferrographic abrasive image segmentation. Unlike ferrographic abrasive images, moving abrasive images do not have a large number of stacked abrasive grains, so the segmentation difficulty is relatively lower. Furthermore, to avoid oversegmentation, this invention employs a distance transform-based marker watershed algorithm to address the problem of multiple abrasive grains adhering together.

[0086] The specific steps for segmentation are as follows:

[0087] S2.1: Perform distance transformation on the binary image of the abrasive target detection result from step S1: Calculate the Euclidean distance from all non-zero pixels in the binary image to the nearest zero pixel. This distance is used as the gray value of that pixel to obtain a distance-transformed image that visually shows the distance distribution from each pixel in the original binary image to the abrasive boundary. The distance-transformed image clearly shows the distance distribution from each pixel in the original binary image to the abrasive boundary, such as... Figure 3 As shown in a;

[0088] S2.2: Set an appropriate threshold for the distance-transformed image and extract the marked regions for each abrasive grain target; from Figure 3 b shows that the two adhered abrasive particles in the GMM detection results are marked as two regions;

[0089] S2.3: The watershed algorithm is used to automatically segment the adhered abrasive particles using the marked area obtained in step S2.2, such as... Figure 3 As shown in c;

[0090] S2.4: Utilize the position and contour of each segmented region to draw the target bounding box for each abrasive grain, and obtain the center position and size of the target detection box, such as... Figure 3 As shown in d.

[0091] from Figure 3 As can be seen from c, using the GMM target detection results, the watershed algorithm can retain individual abrasive targets while accurately segmenting adhered abrasive targets.

[0092] Based on the fusion of Gaussian mixture model and watershed algorithm for accurate detection of moving abrasive particles, the SORT tracker can provide the SORT tracker with accurate center position and size of the target detection box in real time, enabling more precise tracking of the target:

[0093] S3: Tracking of Moving Abrasive Targets

[0094] First, Kalman filtering is used to predict the positions of all abrasive targets in the current frame in the next frame, obtaining the predicted bounding boxes for all targets. Second, the intersection-over-union (IOU) cost matrix of the predicted bounding boxes of all abrasive targets in the current frame and the detection bounding boxes of all abrasive targets in the next frame is calculated. Then, the Hungarian algorithm is used to complete the optimal matching of predicted and detection bounding boxes between consecutive frames, completing data association and achieving target tracking.

[0095] Specifically as follows:

[0096] S3.1: Kalman Filter Prediction

[0097] In this step, the Kalman filter prediction is divided into two stages: prediction and update.

[0098] In the prediction phase, the prior estimate for time is obtained using the optimal estimate at time 1 and the state prediction equation. In the update phase, the prior estimate is corrected using the observed value at time 1 and the state update equation to obtain the optimal estimate for time 2.

[0099] First, define the state variable X of the abrasive target in the k-th frame image. k Sum of observed variables Z k for:

[0100] X k =[x,y,s,r,dx,dy,ds,dr] T (6)

[0101] Z k =[x,y,s,r] T (7)

[0102] Where (x,y) are the center coordinates of the abrasive target box, s and r are the area and aspect ratio of the bounding box, respectively, and dx,dy,ds and dr are the derivatives of the state components with time k.

[0103] In the prediction phase, the estimated value is first verified by the state prediction equation.

[0104]

[0105] Where A is the state transition matrix, W k-1 This is process noise;

[0106] During the update phase, the optimal estimate is obtained from the state update equation.

[0107]

[0108] Where H is the state observation matrix, Z k Let X be the observed value. K is the Kalman gain, which makes the true value X equal to the observed value. k and optimal estimate The error between them is minimal;

[0109] Then, perform data association:

[0110] Data association is a crucial step in multi-object tracking tasks, aiming to solve the matching problem of identical targets between consecutive frames, including the appearance of new targets, the disappearance of old targets, and ID matching of identical targets. The Intersection Over Union (IOU) between the predicted bounding box of the previous frame and the detected bounding box of the next frame can determine the correspondence between them. Therefore, SORT inputs the IOU cost matrix of the predicted and detected boxes into the Hungarian algorithm to obtain the optimal matching result of the predicted and detected boxes. IOU is used to measure the degree of overlap between the predicted and detected boxes, such as... Figure 4 As shown.

[0111] exist Figure 4 In the diagram, A represents the predicted bounding box of the abrasive target, B represents the detected bounding box of the abrasive target, and A∩B represents the overlap area between the predicted and detected bounding boxes. The formula for calculating IOU is:

[0112]

[0113] The tracking results of the abrasive particles collected by the online abrasive particle monitoring system are as follows: Figure 5 As shown in the video, each abrasive grain has a unique ID number, indicating that the SORT algorithm can track abrasive grains very well.

[0114] Example 2: To verify the effectiveness of this invention in detecting adherent moving abrasive particles, this example uses a constructed moving abrasive particle image acquisition platform for data acquisition. The moving abrasive particle image acquisition platform is as follows: Figure 6 As shown, it includes a microfluidic chip 6 (with a specific model, etc.), a microscope 4, a high-speed camera 5, an image storage device 7, a peristaltic pump 2, a microfluidic pipeline 3, and an oil tank 1. As shown in the figure, the peristaltic pump 2 is connected to the microfluidic chip 6 through the microfluidic pipeline 3. The peristaltic pump 2 pumps lubricating oil carrying abrasive particles from the oil tank 1 through the microfluidic pipeline 3 and delivers it to the microfluidic chip 6 so that it flows into its microchannels. The microfluidic chip 6 is placed on the stage of the microscope 4. The high-speed camera 5 captures images of moving abrasive particles flowing through the microchannels of the microfluidic chip 6 through the eyepiece. The high-speed camera 5 is electrically connected to the image storage device 7.

[0115] In this embodiment, the moving abrasive particle image acquisition platform, to avoid motion blur in the captured abrasive particle images, uses a peristaltic pump to precisely control the flow rate between 0.022 ml / min and 22 ml / min. The high-speed camera employs a Sony CMOS sensor with a resolution of pixels and a video frame rate of 30 fps. The system described above can acquire relatively clear videos of moving abrasive particles, which can be used for further moving abrasive particle target detection and tracking.

[0116] (1) The adhesion of a large number of moving abrasive particles usually occurs when there are many abrasive particles in the microchannel. The number of abrasive particles in the oil bath can directly affect the number of abrasive particles in the microchannel. Therefore, abrasive particle video can be obtained by injecting a large number of abrasive particles into the oil bath, and the video can be divided into sparse stage and dense stage according to the number of abrasive particles in the video. The video acquisition process is as follows: in the sparse stage, clear oil is injected into the oil bath, and images with sparse abrasive particles can be obtained in this stage; in the dense stage, lubricating oil carrying a large number of abrasive particles is injected into the oil bath, and images with dense abrasive particles can be obtained in this stage. The acquisition time for the two stages is about 2 minutes in total, and a total of 3620 frames of images are acquired. The first 3000 frames of images are selected for algorithm verification. Since the frame rate of the video is high and the difference between adjacent frames is not large, in order to avoid repeated calculations, one frame of every 5 frames is selected as a representative for algorithm verification, resulting in a total of 600 frames of images. Finally, the actual number of abrasive particles in each frame of the image is manually labeled, such as Figure 7 As shown. From Figure 7 It can be seen that there are fewer abrasive particles in frames 0 to 1400, so this is recorded as the sparse stage; there are more abrasive particles in frames 1400 to 2700, so this is recorded as the dense stage. All algorithms in this embodiment are implemented using Python 3.9.0 and the OpenCV-Python 4.5.5 vision software library. The computer's central processing unit is an Intel Core i7-10870H, and it has 16GB of memory.

[0117] In this embodiment, accuracy is defined as a quantitative evaluation metric for the algorithm. The formula for calculating accuracy is as follows:

[0118]

[0119] Accuracy represents the proportion of correctly detected abrasive particles in each frame of an image, and can objectively reflect the detection performance of the algorithm proposed in this paper.

[0120] (2) This embodiment compares the use of a simple GMM and a fusion of GMM and the watershed algorithm (GMM-Watershed), performing target detection on the aforementioned video and calculating the detection accuracy of each algorithm on each frame. To make the detection results clearer, [the following is omitted as it is not part of the main text]. Figure 7 The 600 frames of images were divided into 30 groups of 20 frames each. The average accuracy of each group was calculated, and the results are as follows: Figure 8 As shown.

[0121] from Figure 8As can be seen, both algorithms achieve high detection accuracy in the sparse stage because the number of abrasive particles in the image is relatively small, and they are less likely to adhere to each other. However, in the initial stage of the video, around frame 300, the abrasive particles are small and the oil flow is slow, causing the GMM algorithm to falsely detect small, slow-moving abrasive particles as background. Since the GMM-Watershed algorithm performs detection based on the GMM algorithm's detection, both algorithms experience false negatives. Secondly, around frame 1100, the number of abrasive particles gradually increases, and they begin to adhere to each other. The accuracy of the GMM algorithm starts to decrease, while the GMM-Watershed algorithm can accurately segment the adhered abrasive particles, achieving 100% accuracy. Finally, in the dense stage, the number of abrasive particles reaches its peak, and adhesion is severe. The accuracy of the GMM algorithm drops significantly, with an average accuracy as low as 71.01% between frames 2000 and 2100. This indicates that the GMM algorithm is not suitable for detecting targets with adhered abrasive particles. The average accuracy of the two methods at different stages is shown in Table 1.

[0122] Table 1: Average accuracy of GMM and GMM-Watershed at different stages

[0123]

[0124] As shown in Table 1, both the GMM algorithm and the GMM-Watershed algorithm have high accuracy in the sparse stage, but in the dense stage, the average accuracy of the GMM-Watershed algorithm is significantly higher than that of the GMM algorithm, reaching 97.58%. Therefore, the GMM-Watershed algorithm can greatly improve the accuracy of abrasive target detection when dealing with abrasive particle adhesion, with an overall average accuracy of 98.86%.

[0125] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for online detection and tracking of moving abrasive particles based on a fusion of Gaussian mixture model and watershed algorithm, characterized in that, It includes the following steps: S1: Motion abrasive target detection based on Gaussian mixture model S1.1: Background model estimation: Perform background model estimation for each frame of the abrasive video; The series of pixel values ​​at each coordinate point in each frame of an image that change over time can be viewed as a random process {I1,...,I...} t Furthermore, it is assumed that the random process satisfies the superposition of K Gaussian distributions. Therefore, for a pixel I at a certain coordinate point in the image at time t... t The probability density function can be expressed as: Where K is the number of Gaussian distributions, N i It is the i-th Gaussian distribution at time t, ω i,t ,μ i,t and σ i,t 2 They are N i The weights, mean, and variance; S1.2: For all pixels I in each frame of the image t Match (x,y) with the background model; S1.3: Model parameter update: Update the background model using the current image; If pixel I t If (x,y) matches the i-th Gaussian distribution in the background model, then the weights, mean, and method of that Gaussian distribution are updated as follows: oh i,t =(1-a)ω i,t-1 +a (3) m i,t =(1-ρ)ω i,t-1 +ρI t (x,y) (4) s i,t 2 =(1-ρ)σ i,t-1 2 +ρ(I t -m i,t-1 ) 2 (5) Where α is the background template update rate, α∈[0,1], and its magnitude affects the sensitivity of the model to the occurrence of moving pixels, ρ=α / ω i,t-1 ; If the pixel does not match the background model, then use I t The Gaussian distribution with the mean (x,y) is replaced with the Gaussian distribution with the lowest weight; S1.4: Foreground abrasive target extraction An adaptive background model is obtained from S1.1 to S1.

3. The current image and the background model are then matched to obtain the abrasive particle target detection result, which is a binary image. S2: Segmentation of Adhesive Abrasive Grains Based on Watershed Algorithm S2.1: Perform distance transformation on the binary image of the abrasive target detection result in step S1: calculate the Euclidean distance from all non-zero pixels in the binary image to the nearest zero pixel, and use this distance as the gray value of the pixel to obtain a distance-transformed image that can intuitively show the distance distribution from each pixel in the original binary image to the abrasive boundary. S2.2: Set an appropriate threshold for the distance transform image and extract the marked regions of each abrasive grain target; S2.3: The watershed algorithm is used to automatically segment the adhering abrasive particles using the marked area obtained in step S2.2; S2.4: Use the position and contour of each segmented region to draw the target bounding box of each abrasive grain, and obtain the center position and size of the target detection box; S3: Tracking of Moving Abrasive Targets S3.1: Kalman Filter Prediction In this step, the Kalman filter prediction is divided into two stages: prediction and update. S3.1.1 First, define the state variable X of the abrasive target in the k-th frame image. k Sum of observed variables Z k for: X k =[x,y,s,r,dx,dy,ds,dr] T (6) Z k =[x,y,s,r] T (7) Where (x,y) are the center coordinates of the abrasive target box, s and r are the area and aspect ratio of the bounding box, respectively, and dx,dy,ds and dr are the derivatives of the state components with time k. S3.1.2: Prediction Phase First, verify the estimated value using the state prediction equation. Where A is the state transition matrix, W k-1 This is process noise; S3.1.2: Update Phase The optimal estimate is obtained from the state update equation. Where H is the state observation matrix, Z k Let X be the observed value. K is the Kalman gain, which makes the true value X equal to the observed value. k and optimal estimate The error between them is minimal; S3.2: Data association.

2. The method for online detection and tracking of moving abrasive particles based on a fusion Gaussian mixture model and a watershed algorithm as described in claim 1, characterized in that, The number of Gaussian distributions in step S1.1 is 3-5.

3. The method for online detection and tracking of moving abrasive particles based on a fusion Gaussian mixture model and a watershed algorithm as described in claim 1, characterized in that, In step 1.2, pixel matching is |I t (x,y)-μ i,t |<2.5σ i,t 2 (2) If a pixel matches the i-th Gaussian distribution, it means that the pixel matches the background model and is therefore considered a background pixel; otherwise, it is considered a particle target pixel.

4. The method for detecting and tracking moving abrasive particles based on a fusion Gaussian mixture model and a watershed algorithm according to claim 1, characterized in that, In step S3.2, the IOU cost matrix of the predicted bounding box and the detection bounding box is input into the Hungarian algorithm to obtain the optimal matching result of the predicted bounding box and the detection bounding box. The formula for calculating IOU is: Among them, S A and S B These are the areas of the predicted bounding box and the detected bounding box, respectively.

Images