A method for tracking and verifying the trajectory of a bubble in a liquid
By combining DeepSort and YOLOv5 algorithms, the problem of frequent target ID changes caused by the shape changes of bubbles in liquids is solved, enabling continuous bubble tracking and trajectory verification, and improving the performance of multi-target tracking algorithms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN ENG UNIV
- Filing Date
- 2023-03-13
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies struggle to effectively track the movement of bubbles in liquids, especially in cases of shape changes and bursting, leading to frequent changes in the target ID and affecting the integrity of trajectory tracking.
By combining the DeepSort algorithm with the YOLOv5 algorithm, and optimizing the multi-target tracking algorithm through training datasets and similarity metrics, continuous tracking and trajectory verification of bubbles can be achieved.
It enables continuous tracking of bubbles with changing shape, provides reliable trajectory data, and improves the performance of multi-target tracking algorithms.
Smart Images
Figure CN116245918B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for trajectory tracking and verification of gas-liquid two-phase flow, particularly for tracking and verifying the trajectory of moving bubbles in a liquid. Specifically, it employs a multi-target tracking algorithm to track the trajectory of bubbles in the liquid, and uses trajectory similarity as a verification metric to improve the algorithm's performance. This invention belongs to the field of computer vision. Background Technology
[0002] Gas-liquid two-phase flow, as one of the most common forms of two-phase flow, is widely found in nature and engineering fields, such as thermal power engineering, nuclear power engineering, hydraulic engineering, shipbuilding, and aerospace. In electrochemical water treatment, the diameter, velocity, and density of bubbles have a significant impact on separation efficiency and effectiveness; appropriately adjusting bubble size and velocity can improve separation performance. In plate heat exchangers, water and air are typically used as the heat exchange medium in the cold aisle. Air in water exists in a bubble state, significantly affecting the heat exchange efficiency and flow resistance of the plate heat exchanger. During ship navigation, underwater exploding bubble loads can cause overall structural damage to the ship. Studying the dynamic characteristics of underwater bubble explosions is crucial for the development of underwater weaponry and ship protection. Therefore, in-depth research into the motion behavior and characteristics of bubbles has significant practical applications.
[0003] The trajectory of a bubble is a crucial indicator of its dynamic behavior. To obtain the trajectory of each bubble in a bubble flow, multi-target tracking is necessary. When bubbles flow in a liquid, they undergo deformation, breakup, and coalescence due to the combined effects of gravity, buoyancy, and surface tension. This leads to frequent changes in the target ID, increasing tracking difficulty and affecting the integrity of the trajectory tracking. Therefore, this invention employs the DeepSort algorithm, fully considering the shape changes and appearance information of bubbles during their movement, to achieve continuous bubble tracking. Furthermore, using bubble flow data with known real trajectory information, the algorithm's performance is optimized using trajectory similarity as a criterion, providing a set of valuable trajectory information for future research. Summary of the Invention
[0004] The purpose of this invention is to propose a method for tracking and verifying the trajectory of moving bubbles in liquids. This method overcomes the problem of frequent changes in target ID caused by changes in bubble shape in liquids, enabling continuous tracking of moving bubbles. Furthermore, it uses trajectory similarity as a verification index to optimize algorithm performance, thereby providing a reliable set of trajectory data.
[0005] The objective of this invention is achieved as follows:
[0006] A method for tracking and detecting the trajectory of air bubbles in a liquid, comprising the following steps:
[0007] Step 1: Acquire bubble flow data to obtain the actual motion trajectory information of each bubble;
[0008] Step 2: Train the YOLOv5 detection algorithm model; In the object detection process, the training dataset for the YOLOv5 algorithm consists of multi-bubble images obtained by segmenting bubble flow videos, including training and test sets;
[0009] Step 3: Train the DeepSort tracking algorithm model; By cropping the training dataset of the YOLOv5 algorithm, the training dataset of the DeepSort algorithm is obtained, and the DeepSort algorithm model is trained.
[0010] Step 4: Obtain the tracking trajectory; Based on the detection model trained by the YOLOv5 algorithm and the tracking model trained by the DeepSort algorithm, the bubble flow data is tracked to obtain the tracking trajectory;
[0011] Step 5: Improve algorithm performance using similarity metrics; calculate the similarity between the real trajectory and the tracking trajectory of each bubble in the bubble flow data, adjust the training parameters appropriately, and optimize the performance of the tracking algorithm.
[0012] Step two includes:
[0013] Step 2.1: Use the DarkLabel annotation tool to manually annotate the multi-bubble image to obtain an XML-formatted annotation dataset;
[0014] Step 2.2: Train the YOLOv5 algorithm model; Input the labeled dataset and multi-bubble images into the YOLOv5 network for training to obtain the trained YOLOv5 algorithm model, and use the model to perform target detection on the bubble flow data.
[0015] Step three includes:
[0016] Step 3.1: Using a Python program, based on the XML file generated by the annotation tool in Step 2.1, crop the multi-bubble image to obtain individual bubble images of different sizes, which will serve as the training dataset.
[0017] Step 3.2: Train the DeepSort algorithm model. Input the training dataset obtained in Step 3.1 into the DeepSort network for training to obtain the trained DeepSort algorithm model.
[0018] Step four specifically involves:
[0019] Step 4.1: Combine YOLOv5 with DeepSort to train a model for tracking bubble flow data. The DeepSort algorithm tracking process is as follows:
[0020] Step 4.1.1, Predicting the State: The Tracks generated in the previous iteration are predicted using Kalman filtering, and the mean and variance of the current iteration are calculated. Tracks are an object used for flow in the tracking process, including eight important attributes: mean, covariance matrix, ID of the tracked target, maximum number of consecutive misses, maximum lifetime, eigenvector, total number of observation updates, total number of frames since the first occurrence, total number of frames since the last observation update, and state. The state does not change and is: Tracks that have been confirmed as matched and Tracks that have not been confirmed as matched.
[0021] Step 4.1.2, First Match: Match the Tracks in 4.1.1 with the Detections detected by the target detector in this round, resulting in three states: unmatched Tracks, unmatched Detections, and matched Tracks; if the number of successful matches is less than three, it is considered an unconfirmed match Track; if it is three or more, it is considered a confirmed match Track.
[0022] Step 4.1.3, Second Match: There may be omissions in the matching in 4.1.2 above, so it needs to be merged with the unconfirmed matching Tracks results in 4.1.1, and the IOU score is used to perform a second match to obtain three definite status states: unmatched Tracks, unmatched Detections, and matched Tracks.
[0023] Step 4.1.4: Handling invalid objects: Among the unmatched Tracks, Tracks that have not been confirmed to match and Tracks that have been confirmed to match but whose total number of frames since the first occurrence exceeds a given threshold need to be deleted;
[0024] Step 4.1.5: Output the matching results and prepare data for the next iteration, merging the following three Tracks cases:
[0025] Step 4.1.5.1: Merge the matched Tracks from 4.1.3 and 4.1.4 above, perform state prediction using Kalman filtering, and simultaneously increment the total number of frames since the first occurrence to output the Tracks;
[0026] Step 4.1.5.2: For the unmatched Detections mentioned in 4.1.3 above, create new Tracks;
[0027] Step 4.1.5.3: Tracks that have been confirmed as matches in 4.1.4 above but whose total number of frames since the first occurrence exceeds a given threshold; these three types of tracks are combined together as the output of this iteration and as the input data for the next iteration.
[0028] Step 4.2: Draw the trajectory of the bubble obtained by tracking in the bubble flow. The algorithm is as follows:
[0029] Step 4.2.1: For the first frame image, calculate the center coordinates of each tracking box, use them as the center coordinates of the bubbles, and store them in list Lst1. The format of Lst1 is as follows:
[0030] Lst1 = [{1:[x 11 x 12 ]},{2:[x 21 x 22 ]},{3:[x 31 x 32 ]},...,{i:[x i1 x i2 ]}]
[0031] In this dictionary, the key i is the ID of the bubble, and the value is the center point coordinates of the bubble corresponding to the ID i. These points are plotted on the tracking result image of the current frame and output, and lst1 is added to the empty list lst2.
[0032] Step 4.2.2: For the j-th (j≠1) frame image, calculate the center coordinates of each tracking box, use them as the center coordinates of the bubble, and reassign them to Lst1. Lst1 (assuming j=2 at present) has the following format:
[0033] Lst1 = [{1:[x 13 x 14 ]},{2:[x 23 x 24 ]},{3:[x 33 x 34 ]},...,{i:[x i3 x i4 ]}]
[0034] Since no bubbles are added or disappear when j=2, the keys of each dictionary in list Lst1 remain unchanged. However, because the bubbles are moving, the values of each dictionary have changed.
[0035] Add lst1 to lst2. Lst2 (assuming j = 2) has the following format:
[0036] Lst2 = [[{1:[x 11 x12 ]},{2:[x 21 x 22 ]},{3:[x 31 x 32 ]},...,{i:[x i1 x i2 ]}],|{1:[x 13 x 14 ]},{2:[x 23 x 24 ]},{3:[x 33 x 34 ]},...,{i:[x i3 x i4 ]}]|
[0037] For each ID in frame j, if it appears in frame j-1, connect the coordinates of the two points corresponding to that ID in the j-th and j-1-th lists of Lst2. Repeat the above operation for the j-1-th and j-2-th lists until the first list. If it does not appear in the j-1-th list, do not draw a line. Check the j-2-th and j-3-th lists. If the ID does not appear in any of the three consecutive lists, it is determined that the ID has disappeared and all its trajectory information is deleted. Finally, plot the above curve on the tracking result image of the current frame and output it, which is the trajectory tracking image of the time before the bubble in the current frame.
[0038] Step 4.2.3: Perform the above operations for each frame of the image. To ensure the neatness of the trajectory drawing, when the length of Lst2 exceeds 30, delete the first element of the list and repeat the above operations. The resulting image is the trajectory tracking result of the first 30 frames for each bubble.
[0039] Step five includes:
[0040] Step 5.1: Calculate the discrete Friesian distance; curve P is the true trajectory of the bubble, and curve Q is the tracking trajectory of the bubble. Let σ(P) and σ(Q) represent the sets of midpoints of the two trajectories, respectively. Then, σ(P) = {u1, ..., u...} p} and σ(Q)={v1,...,v q}, and at the same time, we obtain the following set L of corresponding sequence point pairs:
[0041] L={(u1, v1), (u2, v2),..., (u 600 v 600 )}
[0042] The distance ||L|| between curves P and Q is defined as the maximum Euclidean distance between each pair of points in the sequence, and its expression is as follows:
[0043]
[0044] Then its discrete Friesian distance is defined as follows:
[0045] δ DF (P, Q) = min(||L||)
[0046] Step 5.2: Calculate the mean discrete Fréchet distance of multiple bubbles;
[0047] Step 5.3: Calculate trajectory similarity; based on the mean of discrete Friesian distance, calculate the similarity between the tracked trajectory and the true trajectory, and the formula is as follows:
[0048]
[0049] Where h is the mean discrete Friesian distance between the tracked trajectory and the real trajectory, a is the number of pixels occupied by the width of the video, and b is the number of pixels occupied by the height of the video.
[0050] Step 5.4: Optimize the multi-target tracking algorithm; Based on the similarity obtained in step 5.3, adjust the training parameters and continuously optimize the multi-target tracking algorithm to improve trajectory tracking accuracy.
[0051] Compared with the prior art, the beneficial effects of the present invention are:
[0052] This invention employs a multi-target tracking algorithm based on target detection to track moving bubbles in liquids, considering both the target's appearance and motion information, enabling continuous tracking of bubbles whose shape may change. Furthermore, trajectory similarity is used as a validation metric to optimize the multi-target tracking algorithm, resulting in an algorithm model with improved tracking performance and providing reliable trajectory data for subsequent research. Attached Figure Description
[0053] Figure 1a -b represents the target detection and tracking results images from the training set; where Figure 1a Images representing the target detection results in the training set; Figure 1b Images representing the target tracking results from the training set;
[0054] Figure 2a -c represents the actual trajectory and tracked trajectory images of a bubble in the bubble flow; where Figure 2a This is a frame image from the bubble flow; Figure 2b for Figure 2a The true trajectory image of bubble 1; Figure 2c for Figure 2a Image of the tracking trajectory of bubble 1;
[0055] Figure 3a -e represents a frame image and a portion of the bubble tracking trajectory image from the tracking results; where Figure 3a For a specific frame of the tracking results; Figure 3b for Figure 3a Trajectory image of bubble 2 in the middle; Figure 3c for Figure 3a Trajectory image of bubble 3 in the middle; Figure 3d for Figure 3a Trajectory image of bubble 4 in the middle; Figure 3e for Figure 3a Image of the tracking trajectory of bubble 5 in the middle;
[0056] Figure 4 This is the overall flowchart of the present invention. Detailed Implementation
[0057] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0058] In this embodiment, the bubble flow data for both the training and application phases are derived from a two-phase flow experiment conducted using a simulated closed loop. The vertical upward flow of the gas-liquid two-phase flow is controlled by adjusting pipeline valves. An integrated air compressor generates compressed air, which is stored in a 300L compressed air tank. The compressed air enters the bubble generator via a solenoid valve, then proceeds to the experimental section and the gas-liquid separator, before finally being discharged into the atmosphere. The experimental section is approximately 3.7m long and consists of acrylic tubes of varying lengths with an inner diameter of 50.8mm, connected by a data acquisition window. The data acquisition window features a planar design, enabling high-speed imaging. The acquisition frequency is 30fps, with a width*height of 576*1024 pixels. The water flow velocity is 2m / s, vertically upward, and the bubble flow velocity is similar to the water flow velocity.
[0059] A method for tracking and detecting the trajectory of air bubbles in a liquid includes the following steps:
[0060] Step 1: Training Phase
[0061] 1. Acquire bubble flow data. To improve the performance of the tracking algorithm, bubble flow data was acquired based on experimental simulation, where the actual trajectory information of each bubble was known.
[0062] 2. Training the YOLOv5 detection algorithm model. The dataset used for object detection consisted of 1871 multi-bubble images obtained from segmented bubble stream videos. Of these, 1496 were used for training and 375 for testing.
[0063] 2.1 The DarkLabel annotation tool was used to manually annotate the multi-bubble image to obtain an XML-formatted annotation dataset.
[0064] 2.2 Training the YOLOv5 Algorithm Model. The labeled dataset and multi-bubble images were input into the YOLOv5 network for training, resulting in a trained YOLOv5 algorithm model. This model was then used to perform object detection on the bubble flow data. A frame from the detection results is shown below. Figure 1a As shown.
[0065] 3. Training the DeepSort tracking algorithm model. The training dataset used in the target tracking process was obtained by processing the YOLOv5 algorithm training dataset, totaling 20,581 images. Of these, 18,522 images were used for training and 2,059 images were used for testing.
[0066] 3.1 Using a Python program, based on the XML file generated by the annotation tool in 2.1, the multi-bubble image is cropped to obtain individual bubble images of different sizes, which are used as the training dataset.
[0067] 3.2 Training the DeepSort algorithm model. Input the training dataset into the DeepSort network for training to obtain the trained DeepSort algorithm model.
[0068] 4. Obtain the tracking trajectory.
[0069] 4.1 Using YOLOv5 to train a model, DeepSort is then used to train the model to track bubble flow data. The DeepSort algorithm tracking process is as follows:
[0070] 4.1.1 Prediction Status: The Tracks generated in the previous iteration are predicted using Kalman filtering, and the mean and variance of the current iteration are calculated. Tracks are objects used for flow in the tracking process, including eight important attributes: mean, covariance matrix, target ID, maximum number of consecutive misses, maximum lifetime, eigenvector, total number of observation updates, total number of frames since the first occurrence, total number of frames since the last observation update, and status. The status remains unchanged, namely: Tracks with confirmed matches and Tracks with unconfirmed matches.
[0071] 4.1.2 First Match: The Tracks in 4.1.1 are matched with the Detections detected by the target detector in this round, resulting in three states: unmatched Tracks, unmatched Detections, and matched Tracks. Tracks with fewer than three successful matches are considered unconfirmed matches, while those with three or more successful matches are considered confirmed matches.
[0072] 4.1.3 Second Match: The matching in 4.1.2 above may have omissions, so it needs to be merged with the unconfirmed matching Tracks results in 4.1.1, and the IOU score is used to perform a second matching to obtain three definite states: unmatched Tracks, unmatched Detections, and matched Tracks.
[0073] 4.1.4 Handling invalid objects: Tracks that have not been confirmed to match and tracks that have been confirmed to match but whose total number of frames since the first occurrence exceeds a given threshold need to be deleted.
[0074] 4.1.5 Output the matching results and prepare data for the next iteration, merging the following three Tracks cases:
[0075] 4.1.5.1 The matched Tracks in 4.1.3 and 4.1.4 above are merged, and state prediction is performed by Kalman filtering. At the same time, the total number of frames since the first occurrence is increased by one, and the Tracks are output.
[0076] 4.1.5.2 For the unmatched Detections mentioned in 4.1.3 above, create new Tracks.
[0077] 4.1.5.3 Tracks that have been confirmed as matches in 4.1.4 above, but whose total number of frames since the first occurrence exceeds a given threshold.
[0078] These three tracks are combined as the output of this iteration and as the input data for the next iteration.
[0079] 4.2 The following algorithm is used to plot the trajectory of the bubble obtained by tracking in the bubble flow.
[0080] 4.2.1 For the first frame image, calculate the center coordinates of each tracking box, use them as the center coordinates of the bubbles, and store them in list Lst1. The format of Lst1 is as follows:
[0081] Lst1 = [{1:[x 11 x 12 ]},{2:[x 21 x 22 ]},{3:[x 31 x 32 ]},...,{i:[x i1 x i2 ]}]
[0082] In this dictionary, the key 'i' is the bubble's ID, and the value is the center point coordinates of the bubble with ID 'i'. These points are plotted on the tracking result image of the current frame and output, and 'lst1' is added to the empty list 'lst2'.
[0083] 4.2.2 For the j-th (j≠1) frame, calculate the center coordinates of each tracking box, use them as the center coordinates of the bubble, and reassign them to Lst1. Lst1 (assuming j=2) has the following format:
[0084] Lst1 = [{1:[x 13 x 14 ]},{2:[x 23 x 24 ]},{3:[x 33 x 34 ]},...,{i:[x i3 x i4 ]}]
[0085] Since no bubbles are added or disappear when j=2, the keys of each dictionary in list Lst1 remain unchanged. However, because the bubbles are moving, the values of each dictionary have changed.
[0086] Add lst1 to lst2. Lst2 (assuming j = 2) has the following format:
[0087] Lst2 = [[{1:[x 11 x 12 ]},{2:[x 21 x 22 ]},{3:[x 31 x 32 ]},...,{i:[x i1 x i2 ]}],[{1:[x 13 x 14 ]},{2:[x 23 x 24 ]},{3:[x 33 x 34 ]},...,{i:[x i3 x i4 ]}]]
[0088] For each ID in frame j, if it appears in frame j-1, connect the coordinates of the two points corresponding to that ID in the j-th and j-1-th lists of Lst2. Repeat the above operation for the j-1-th and j-2-th lists until the first list. If it does not appear in the j-1-th list, do not draw a line. Check the j-2-th and j-3-th lists. If the ID does not appear in any of the three consecutive lists, it is determined that the ID has disappeared and all its trajectory information is deleted.
[0089] Finally, the above curves are plotted on the tracking result image of the current frame and output, which is the trajectory tracking image of the moment before the bubble in the current frame.
[0090] 4.2.3 Perform the above operations for each frame of the image. To ensure the cleanliness of the trajectory drawing, when the length of Lst2 exceeds 30, delete the first element of the list and repeat the above operations. The resulting image is the trajectory tracking result for each bubble in the previous 30 frames (including the current frame). When j = 150, the bubble flow tracking and trajectory drawing results are as follows: Figure 1b As shown.
[0091] 5. Improve algorithm performance using similarity metrics. The Fréchet distance is used to calculate the similarity between the actual trajectory and the tracked trajectory of each bubble in the bubble flow data, serving as a validation metric. Simultaneously, training parameters are adjusted to optimize algorithm performance.
[0092] 5.1 Calculate the discrete Frescher distance. Figure 2a Given a frame of an image from a set of bubble flow data, for bubble 1, curve P represents the true trajectory of bubble 1 (e.g., ...). Figure 2b As shown), curve Q is the tracking trajectory of bubble 1 (as shown). Figure 2c As shown in the figure, curves P and Q are each composed of 600 trajectory points. Let σ(P) and σ(Q) represent the sets of points in the two trajectories, respectively, then we have σ(P) = {u1, ..., u2} p} and σ(Q)={v1,...,v q At the same time, we can obtain the set L of the following one-to-one corresponding sequence point pairs:
[0093] L={(u1, v1), (u2, v2),..., (u 600 v 600 )}
[0094] The distance ||L|| between curves P and Q is defined as the maximum Euclidean distance between each pair of points in the sequence, and its expression is as follows:
[0095]
[0096] Then its discrete Friesian distance is defined as follows:
[0097] δ DF (P, Q) = min(||L||)
[0098] For bubble 1, δ is obtained. DF (P, Q) = 5.70 (rounded to two decimal places, unit is pixels). Figure 2a There are a total of 9 bubbles, and their discrete Friesian distances are shown in the table below:
[0099]
[0100]
[0101] 5.2 Calculate the mean discrete Friesian distance for multiple bubbles.
[0102] The bubble flow dataset contains a total of 500 bubbles. The discrete Friesian distance for each bubble is calculated, and the mean discrete Friesian distance of the final bubble flow dataset is 5.23.
[0103] 5.3 Calculate trajectory similarity. Based on the mean of the discrete Friesian distance, the similarity between the tracked trajectory and the true trajectory is calculated, and the formula is given below:
[0104]
[0105] Where h is the mean discrete Friesian distance between the tracked trajectory and the real trajectory, a is the number of pixels occupied by the width of the video, and b is the number of pixels occupied by the height of the video. Since the bubble flow video is 576 pixels wide and 1024 pixels high, the similarity between the obtained tracked trajectory and the real trajectory is 99.55%.
[0106] 5.4 Optimize the multi-target tracking algorithm. Based on the similarity obtained in 5.3, adjust the training parameters and continuously optimize the multi-target tracking algorithm to improve trajectory tracking accuracy. After multiple adjustments, on the bubble flow dataset, the mean discrete Fréchet distance between the tracked trajectory and the real trajectory is 4.68 pixels, and the calculated similarity between the tracked trajectory and the real trajectory is 99.60%.
[0107] II. Application Phase
[0108] Using the algorithm model obtained during the training phase, the actual measured bubble flow data is tracked to obtain the tracking trajectory. At this point, the true motion trajectory information of each bubble in the bubble flow data is unknown. A frame of the tracking result is shown below. Figure 3a As shown, the tracking trajectories of some bubbles are as follows: Figure 3b , 3c As shown in 3d and 3e.
Claims
1. A method for tracking and detecting the trajectory of air bubbles in a liquid, characterized in that, Includes the following steps: Step 1: Acquire bubble flow data to obtain the actual motion trajectory information of each bubble; Step 2: Train the YOLOv5 detection algorithm model; In the object detection process, the training dataset for the YOLOv5 algorithm consists of multi-bubble images obtained by segmenting bubble flow videos, including training and test sets; Step 3: Train the DeepSort tracking algorithm model; By cropping the training dataset of the YOLOv5 algorithm, the training dataset of the DeepSort algorithm is obtained, and the DeepSort algorithm model is trained. Step 4: Obtain the tracking trajectory; Based on the detection model trained by the YOLOv5 algorithm and the tracking model trained by the DeepSort algorithm, the bubble flow data is tracked to obtain the tracking trajectory; Step 5: Improve algorithm performance using similarity metrics; calculate the similarity between the real trajectory and the tracking trajectory of each bubble in the bubble flow data, adjust the training parameters appropriately, and optimize the performance of the tracking algorithm; Step 5.1: Calculate the discrete Fréchet distance; Curve P represents the actual trajectory of the bubble, and curve Q represents the tracking trajectory of the bubble. and Let and represent the sets of midpoints of the two trajectories, then we have and At the same time, we obtain the following set of one-to-one corresponding sequence point pairs. : curve Distance between Defined as the maximum Euclidean distance between each pair of points in the sequence, the expression is as follows: Where i is the ID of the bubble; Then its discrete Frescher distance The definition is as follows: Step 5.2: Calculate the mean discrete Fréchet distance of multiple bubbles; Step 5.3: Calculate trajectory similarity; based on the mean of discrete Friesian distance, calculate the similarity between the tracked trajectory and the true trajectory, and the formula is as follows: in, To track the mean discrete Friesian distance between the trajectory and the true trajectory, This refers to the number of pixels occupied by the width of the video. The number of pixels occupied by the height of the video; Step 5.4: Optimize the multi-target tracking algorithm; Based on the similarity obtained in step 5.3, adjust the training parameters and continuously optimize the multi-target tracking algorithm to improve trajectory tracking accuracy.
2. The method for tracking and detecting the trajectory of air bubbles in a liquid according to claim 1, characterized in that, Step 2 includes: Step 2.1: Use the DarkLabel annotation tool to manually annotate the multi-bubble image to obtain an XML-formatted annotation dataset; Step 2.2: Train the YOLOv5 algorithm model; Input the labeled dataset and multi-bubble images into the YOLOv5 network for training to obtain the trained YOLOv5 algorithm model, and use the model to perform target detection on the bubble flow data.
3. The method for tracking and detecting the trajectory of air bubbles in a liquid according to claim 1, characterized in that, Step 3 includes: Step 3.1: Using a Python program, based on the XML file generated by the annotation tool in Step 2.1, crop the multi-bubble image to obtain individual bubble images of different sizes, which will serve as the training dataset. Step 3.2: Train the DeepSort algorithm model. Input the training dataset obtained in Step 3.1 into the DeepSort network for training to obtain the trained DeepSort algorithm model.
4. The method for tracking and detecting the trajectory of air bubbles in a liquid according to claim 1, characterized in that, Step 4 specifically involves: Step 4.1: Combine YOLOv5 with DeepSort to train a model for tracking bubble flow data. The DeepSort algorithm tracking process is as follows: Step 4.1.1, Predicting the State: The Tracks generated in the previous iteration are predicted using Kalman filtering, and the mean and variance of the current iteration are calculated. Tracks are an object used for flow in the tracking process, including eight important attributes: mean, covariance matrix, ID of the tracked target, maximum number of consecutive misses, maximum lifetime, eigenvector, total number of observation updates, total number of frames since the first occurrence, total number of frames since the last observation update, and state. The state does not change and is: Tracks that have been confirmed as matched and Tracks that have not been confirmed as matched. Step 4.1.2, First Match: Match the Tracks in 4.1.1 with the Detections detected by the target detector in this round, resulting in three states: unmatched Tracks, unmatched Detections, and matched Tracks; if the number of successful matches is less than three, it is considered an unconfirmed match Track; if it is three or more, it is considered a confirmed match Track. Step 4.1.3, Second Match: There may be omissions in the matching in 4.1.2 above, so it needs to be merged with the unconfirmed matching Tracks results in 4.1.1, and the IOU score is used to perform a second match to obtain three definite status states: unmatched Tracks, unmatched Detections, and matched Tracks. Step 4.1.4: Handling invalid objects: Among the unmatched Tracks, Tracks that have not been confirmed to match and Tracks that have been confirmed to match but whose total number of frames since the first occurrence exceeds a given threshold need to be deleted; Step 4.1.5: Output the matching results and prepare data for the next iteration, merging the following three Tracks cases: Step 4.1.5.1: Merge the matched Tracks from 4.1.3 and 4.1.4 above, perform state prediction using Kalman filtering, and simultaneously increment the total number of frames since the first occurrence to output the Tracks; Step 4.1.5.2: For the unmatched Detections mentioned in 4.1.3 above, create new Tracks; Step 4.1.5.3: Tracks that have been confirmed as matches in 4.1.4 above but whose total number of frames since the first occurrence exceeds a given threshold; these three types of tracks are combined together as the output of this iteration and as the input data for the next iteration. Step 4.2: Draw the trajectory of the bubble obtained by tracking in the bubble flow. The algorithm is as follows: Step 4.2.1: For the first frame image, calculate the center coordinates of each tracking box, use them as the center coordinates of the bubbles, and store them in a list. , The format is as follows: In this dictionary, the key i is the ID of the bubble, and the value is the ID. The coordinates of the center point corresponding to the bubble are plotted on the current frame's tracking result image and output. Add to empty list ; Step 4.2.2, for the first For each frame image, calculate the center coordinates of each tracking box, use these coordinates as the center coordinates of the bubble, and then reassign them to... , (assuming the current) The format is as follows: because At that time, no bubbles were added or disappeared, so the keys of each dictionary in list Lst1 did not change. However, because the bubbles were moving, the values of each dictionary had changed. Will Add to middle, (assuming the current) The format is as follows: For the Each ID within a frame, if it appears in the first... In the frame, Lst2 will be... The and the first Connecting the coordinates of the two points corresponding to the ID in the list, the first... The and the first Repeat the above operation on the list until the first list is reached; if it does not appear in the first list, continue the operation. In the middle, do not draw lines yet, check the first... , No. If an ID does not appear in any of the three consecutive lists, it is determined that the ID has disappeared and all its trajectory information is deleted. Finally, the above curve is plotted on the current frame tracking result image and output, which is the trajectory tracking image of the moment before the bubble in the current frame. Step 4.2.3: Perform the above operations for each frame of the image. To ensure the neatness of the trajectory drawing, when the length of Lst2 exceeds 30, delete the first element of the list and repeat the above operations. The resulting image is the trajectory tracking result of the first 30 frames for each bubble.