A method for identifying and tracking plasmoids

Abstract

The present invention discloses a method for identifying and tracking plasma clusters, comprising: detecting plasma clusters in a plasma cluster image based on a weight model to obtain the plasma cluster pixel coordinate position; tracking the plasma cluster pixel coordinate position based on a SORT multi-target tracking algorithm to obtain the running trajectory of the plasma cluster; it relates to the field of image processing technology, and solves the technical problem that multiple plasma clusters interfere with each other and cannot be accurately tracked; the coordinate position of plasma in the image is identified by a weight model, and it is tracked by a SORT multi-target tracking algorithm, and the difference in image grayscale gradient and the motion characteristics of the plasma cluster are fully considered, and the use of the above algorithm greatly improves the accuracy of plasma cluster identification and tracking.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular is a method for identifying and tracking plasma clusters. Background Art

[0002] The compact ring is a group of plasma with a self-organizing structure, high density and high speed. Because of its extremely high energy density and the ability to feed the tokamak core, it is considered to be one of the possible ways to feed the core of future fusion reactors. There are often multiple plasma groups in the plasma flow ejected by the compact ring injection device, and it is essential to obtain the parameters of each plasma group. The process of identifying and tracking plasma groups is to identify the plasma groups in the image and continuously track the trajectory of each plasma group. Through this process, the speed of each plasma group can be measured and calculated.

[0003] The existing method for measuring and calculating the velocity of plasma clusters is to use the data measured by fiber interferometers or magnetic probes at different positions for calculation. Since it is difficult to distinguish different plasma clusters, multiple plasma clusters interfere with each other and cannot be accurately tracked, making it difficult to determine the trajectory of each plasma cluster. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a method for identifying and tracking plasma clusters, which is used to solve the technical problem that multiple plasma clusters interfere with each other and cannot be accurately tracked. The present invention detects plasma clusters in a plasma cluster image based on a weight model to obtain the pixel coordinate positions of the plasma clusters, and tracks the pixel coordinate positions of the plasma clusters based on the SORT multi-target tracking algorithm to obtain the running trajectory of the plasma clusters, thereby solving the above-mentioned problem.

[0005] To achieve the above object, a first aspect of the present invention provides a method for identifying and tracking a plasma mass, comprising:

[0006] The plasma mass in the plasma mass image is detected based on the weight model to obtain the pixel coordinate position of the plasma mass; wherein the weight model is obtained based on deep learning training;

[0007] The pixel coordinate position of the plasma cluster is tracked based on the SORT multi-target tracking algorithm to obtain the trajectory of the plasma cluster.

[0008] The pixel coordinate position of the plasma mass is converted into the world coordinate system to obtain the world coordinate of the plasma mass.

[0009] Preferably, the training method of the weight model includes:

[0010] The plasma mass image is collected by a high-speed camera, and the plasma in the plasma mass image is annotated to obtain a plasma mass image data set; the plasma mass image data set is divided into a training set and a test set according to a preset ratio;

[0011] The training set is subjected to feature extraction and target classification by using the UNet neural network algorithm, and the network parameters are updated through iterative training using the Adam optimizer, and the weight model with the minimum loss on the test set is saved;

[0012] Preferably, the UNet neural network algorithm includes a feature extraction network (encoder) and a feature fusion network (encoder).

[0013] Preferably, the activation function corresponding to the UNet neural network algorithm is the Mish activation function, and the loss function adopts the Dice Loss algorithm;

[0014] The expression of the Mish activation function is: f(x)=x×tanh×[ln(1+e x )];

[0015] The mathematical expression of the Dice Loss algorithm used in the loss function is: Among them, P represents the predicted probability of the weighted model, G represents the true label category of the sample, and p i and g i Represent the elements in P and G respectively, and N represents the number of elements.

[0016] Preferably, the SORT multi-target tracking algorithm includes a prediction module based on a standard Kalman filter and a matching module based on a Hungarian algorithm;

[0017] Preferably, the tracking of the pixel coordinate position of the plasma mass based on the SORT multi-target tracking algorithm includes:

[0018] The SAUF connected domain labeling algorithm is used to label the pixel coordinates of the detected plasma clusters;

[0019] The trajectory of the plasma cluster is predicted by the prediction module in the SORT multi-target tracking algorithm to obtain the predicted trajectory;

[0020] The predicted trajectory of the plasma cluster is matched with the corresponding plasma cluster in the next frame through the matching module to obtain the running trajectory of the plasma cluster.

[0021] Preferably, the cost matrix corresponding to the matching module is calculated using Mahalanobis distance; the calculation expression of the Mahalanobis distance is: Among them, C i,j is the (i, j)th element of the cost matrix C, d i,jis the square of the Mahalanobis distance between the predicted position of the ith trajectory and the jth detection position, and θ is the proximity threshold, which is used to eliminate trajectories and detection results that cannot be paired.

[0022] Preferably, the step of converting the plasma mass pixel coordinate position into a world coordinate system comprises:

[0023] The homography matrix is ​​calculated based on the robust algorithm of RANSAC. The calculation expression of the homography matrix is: in,

[0024] Where [u,v] T Represents the pixel coordinate system, [X, Y, Z] T represents the world coordinate system, M represents the homography matrix, M1 is the intrinsic parameter matrix, M2 is the extrinsic parameter matrix, λ is the scale factor, f is the focal length, dx and dy are the horizontal and vertical actual physical sizes of the pixel respectively, u0 and v0 are the horizontal and vertical coordinates in the pixel coordinate system, R is the rotation matrix, and T is the transformation matrix;

[0025] The pixel coordinate system is converted into a world coordinate system through homography transformation.

[0026] Preferably, a method for identifying and tracking a plasma mass further comprises: calculating a velocity based on the world coordinates of the plasma mass;

[0027] The speed is calculated based on the world coordinates of the plasma group, including:

[0028] A high-speed camera is used to shoot a checkerboard calibration plate, and the pixel coordinates of the checkerboard corners are matched with the corresponding world coordinates. The homography matrix is ​​solved using a robust algorithm based on RANSAC. The pixel coordinate system in the image is converted to the world coordinate system through homography transformation, and the world coordinate sets {(X, Y, Z)} of several plasma clusters are obtained.

[0029] The point with the largest gray value in each plasma cluster is selected as the center point, and the speed of each plasma cluster can be calculated by dividing the distance between the coordinates of the center points of adjacent frames by the sampling time of each frame of the camera.

[0030] In the processing provided by the present invention, the difference in image grayscale gradients and the motion characteristics of plasma clusters are fully considered, thereby greatly improving the accuracy of plasma cluster identification and tracking; further, the speed of each plasma cluster can be accurately solved and obtained.

[0031] Compared with the prior art, the beneficial effects of the present invention are: the coordinate position of the plasma in the image is identified by a weight model, and it is tracked by the SORT multi-target tracking algorithm, which fully takes into account the differences in image grayscale gradients and the motion characteristics of plasma clusters. The above algorithm greatly improves the accuracy of plasma cluster identification and tracking; further, the speed of each plasma cluster can be accurately solved through the plasma cluster world coordinates, laying the foundation for automatic diagnosis and large-scale data analysis of compact ring injection devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A schematic flow chart of a method for identifying and tracking plasma clusters according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the UNet neural network structure mentioned in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the plasma cluster velocity mentioned in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Since it is difficult to accurately segment plasma clumps using traditional computer image segmentation techniques, the present invention uses a UNet neural network to segment plasma clumps in images. At the same time, in order to track plasma clumps in real time and capture their trajectories and evolutionary characteristics, the present invention uses a SORT multi-target tracking algorithm and improves the algorithm to improve the tracking accuracy of plasma clumps.

[0038] Specifically, in response to the need to calculate the motion trajectories and speeds of multiple plasma clusters in real time, the present invention proposes a visible light image diagnosis algorithm based on the UNet neural network and the SORT multi-target tracking algorithm. This method can identify and track plasma clusters in visible light images and calculate the speed of each plasma cluster, thereby creating a favorable environment for using image data for extensive data analysis.

[0039] See also Figure 1-Figure 3 According to a first aspect of the present invention, a method for identifying and tracking a plasma mass is provided, comprising:

[0040] The plasma mass in the plasma mass image is detected based on the weight model to obtain the pixel coordinate position of the plasma mass; wherein the weight model is obtained based on deep learning training;

[0041] In this embodiment, the training method of the weight model includes the following steps:

[0042] S11: Establish two coordinate systems: image pixel coordinate system [u, v] T and the real object world coordinate system [X,Y,Z] T ,Any point in the image pixel coordinate system can be represented by A(u,v), and any point B in the real object world coordinate system can be represented by B(X,Y,Z);

[0043] S12: Capture plasma mass images through a high-speed camera, and manually annotate the plasma in the plasma mass images to establish a plasma mass image dataset; divide the plasma mass image dataset into a training set and a test set in a ratio of 7:3;

[0044] S13: extracting features and classifying targets from the training set using the UNet neural network algorithm, adjusting the learning rate using the Adam algorithm, and continuously performing iterative training to update network parameters, and saving the weight model with the smallest loss on the test set;

[0045] In this embodiment, the method for acquiring the plasma image data set in S12 includes:

[0046] The plasma mass image dataset can come from the plasma mass images ejected by the EAST-CTI system taken by a high-speed camera, and the images can be grayscale images with 128×32 pixel values; then, images with obvious features are screened out from the plasma mass image dataset and manually annotated, and finally 325 training sets and 82 test sets are obtained, totaling 407 images.

[0047] Specifically, the dataset is annotated in json format using the annotation software Labelme, only the plasma cluster part is annotated, and the annotation in json format is converted into npy format which is easy to read.

[0048] Through the above processing, the plasma cluster image data set required in the implementation process of the present invention can be obtained.

[0049] In this embodiment, the Adam algorithm is used in S13 to adjust the learning rate and continuously perform iterative training to update the network parameters, and the weight model with the smallest loss on the test set is saved. The specific process includes the following steps:

[0050] S131: sending the data set into the network for feature extraction;

[0051] S132: Adam optimizer updates network parameters;

[0052] S133: judging whether the current optimal weight model is obtained based on the loss function; if yes, jumping to S134; if no, jumping to S135; that is, the smaller the loss, the better the weight model;

[0053] S134: Save the current optimal weight model to replace the previously saved model;

[0054] S135: Determine whether the entire iteration is completed; if yes, obtain the current optimal weight model; if not, jump to S131 for loop iteration.

[0055] That is to say, in the above process, the corresponding iterative training process needs to be repeated continuously until the weight model with the minimum loss is obtained, and the optimal weight model obtained by training is saved, referred to as the optimal model

[0056] The pixel coordinate position of the plasma cluster is tracked based on the SORT multi-target tracking algorithm to obtain the trajectory of the plasma cluster.

[0057] Furthermore, the network structure of the UNet neural network is as follows Figure 2 As shown in the figure, it includes two symmetrical left and right parts: a feature extraction network (encoder) and a feature fusion network (decoder). Among them, the feature extraction network on the left reduces the size of the input image by 16 times through convolution and pooling operations, thereby obtaining a four-layer feature map. At the same time, the feature fusion network on the right connects the corresponding cropped feature map in the contraction path with the upsampled output. Then, through continuous convolution, more accurate features can be extracted from the fused feature map. The specific implementation methods of the corresponding UNet neural network improvement may include:

[0058] Based on the original UNet neural network algorithm, the activation function and loss function were changed to improve the accuracy of segmentation. At the same time, the size of the input and output images was changed to meet the requirement of 128×32 pixel values ​​for input and output images.

[0059] Specifically, the activation function in the UNet neural network structure uses the Mish activation function, which can make the network easier to optimize and improve the generalization performance. Some smaller negative inputs can be retained as negative outputs to improve the interpretability and gradient flow of the network. Its mathematical formula is: f(x) = x×tanh×[ln(1+e x )];

[0060] The loss function uses the Dice Loss algorithm, which reduces the influence of background pixels by calculating the intersection ratio of the prediction and the true label, thereby effectively alleviating the sample imbalance problem, and can effectively punish low-confidence predictions and improve segmentation accuracy. Its mathematical formula is: Among them, P represents the predicted probability of the weighted model, G represents the true label category of the sample, and p i and g i Represent the elements in P and G respectively, and N represents the number of elements.

[0061] Through the above specific implementation method, it is possible to extract features from the images in the data set based on the UNet neural network.

[0062] The specific steps after obtaining the optimal weight model are as follows:

[0063] S21: sending the image to be detected into the optimal model obtained after training to detect the plasma group in the image and obtain the pixel coordinates of the plasma group, namely {(u, v)};

[0064] S22: using the SAUF connected domain labeling algorithm to label the detected plasma clusters, so that each plasma cluster in each frame is assigned a unique label, so that a single plasma cluster can be processed;

[0065] S23: The prediction module of the SORT multi-target tracking algorithm predicts the trajectory of the detected plasma clusters;

[0066] S24: The matching module of the SORT multi-target tracking algorithm matches the trajectory of the predicted plasma mass with the plasma mass detected in the next frame, thereby achieving tracking of the plasma mass.

[0067] The prediction module of the SORT multi-target tracking algorithm is based on the standard Kalman filter algorithm. The standard Kalman filter algorithm has a constant speed motion and linear observation model, and the basic principle includes two main steps: prediction and update.

[0068] Specifically, the prediction step is to generate the mode state forecast value at the current moment based on the state estimation value at the previous moment; the update step is to introduce the observation data and re-analyze the mode state using the minimum variance estimation method. The specific implementation methods of the corresponding prediction module improvement may include:

[0069] Based on the original standard Kalman filter algorithm, the trajectory initialization stage is improved to improve the accuracy of the initial prediction. Specifically, an estimate of the initial velocity is added in the trajectory initialization stage to predict the initial trajectory of the plasma group. The above estimate is calculated based on the outermost point of the boundary of the plasma flow propagating in the horizontal direction. The boundary can be extracted using a traditional image segmentation algorithm, such as the Canny algorithm. During the multi-target tracking process, the above estimate is continuously updated by the average value of all existing trajectory velocities.

[0070] The above specific implementation method can realize the corresponding prediction of the trajectory of the plasma cluster.

[0071] The matching module of the SORT multi-target tracking algorithm is based on the Hungarian algorithm. The Hungarian algorithm is a combinatorial optimization algorithm that solves the assignment problem in polynomial time. The cost matrix required by the algorithm can be obtained by using the Mahalanobis distance calculation, and its mathematical formula is: Among them, C i,j is the (i, j)th element of the cost matrix C, d i,j is the square of the Mahalanobis distance between the predicted position of the ith trajectory and the jth detection position, and θ is the proximity threshold, which is used to eliminate trajectories and detection results that cannot be paired.

[0072] The pixel coordinate position of the plasma mass is converted into the world coordinate system to obtain the world coordinate of the plasma mass;

[0073] Calculate the velocity based on the world coordinates of the plasma mass;

[0074] The specific methods for calculating its speed include:

[0075] A high-speed camera is used to shoot a checkerboard calibration plate, and the pixel coordinates of the checkerboard corners are matched with the corresponding world coordinates. The homography matrix is ​​solved using a robust algorithm based on RANSAC. The pixel coordinate system in the image is converted to the world coordinate system through homography transformation, and the world coordinate sets {(X, Y, Z)} of several plasma clusters are obtained.

[0076] In the above process of converting the pixel coordinate system in the image into the world coordinate system through homography transformation, the specific implementation method of the process of realizing the coordinate system conversion may include:

[0077] The pixel coordinate system of the image is converted into the world coordinate system where the object's real coordinates are located through homography transformation. The corresponding transformation matrix is ​​called the homography matrix, and its mathematical formula is: in,

[0078] Where [u,v] T Represents the pixel coordinate system, [X, Y, Z] T represents the world coordinate system, M represents the homography matrix, M1 is the intrinsic parameter matrix, M2 is the extrinsic parameter matrix, λ is the scale factor, f is the focal length, dx and dy are the horizontal and vertical actual physical sizes of the pixel respectively, u0 and v0 are the horizontal and vertical coordinates in the pixel coordinate system, R is the rotation matrix, and T is the transformation matrix;

[0079] The corresponding coordinate system conversion processing can be achieved through the above specific implementation method.

[0080] The point with the largest gray value in each plasma cluster is selected as the center point, and the speed of each plasma cluster can be calculated by dividing the distance between the coordinates of the center points of adjacent frames by the sampling time of each frame of the camera.

[0081] In the processing provided by the present invention, the difference in image grayscale gradients and the motion characteristics of plasma clusters are fully considered, thereby greatly improving the accuracy of plasma cluster identification and tracking; further, the speed of each plasma cluster can be accurately solved and obtained.

[0082] In summary, it can be seen that in the implementation process of the above-mentioned technical solution provided by the present invention, the differences in image grayscale gradients and the motion characteristics of plasma clusters are fully taken into consideration. The adoption of the above algorithm greatly improves the accuracy of plasma cluster identification and tracking; further, the speed of each plasma cluster can be accurately solved, laying the foundation for the automatic diagnosis and large-scale data analysis of the compact ring injection device.

[0083] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0084] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for identifying and tracking plasma clusters, characterized in that: include: The plasma mass in the plasma mass image is detected based on the weight model to obtain the pixel coordinate position of the plasma mass; wherein the weight model is obtained based on deep learning training; The pixel coordinate position of the plasma cluster is tracked based on the SORT multi-target tracking algorithm to obtain the trajectory of the plasma cluster. The pixel coordinate position of the plasma mass is converted into the world coordinate system to obtain the world coordinate of the plasma mass, and the speed is calculated based on the world coordinate of the plasma mass.

2. A method for identifying and tracking plasma clusters according to claim 1, characterized in that: The training method of the weight model comprises: Collecting plasma cluster images and annotating the plasma in the plasma cluster images to obtain a plasma cluster image dataset; dividing the plasma cluster image dataset into a training set and a test set; The UNet neural network algorithm is used to extract features and classify targets from the training set, and iterative training is performed to update network parameters, and the weight model with the smallest loss on the test set is saved.

3. A method for identifying and tracking plasma clusters according to claim 2, characterized in that: The activation function corresponding to the UNet neural network algorithm is the Mish activation function, and the loss function adopts the Dice Loss algorithm.

4. A method for identifying and tracking plasma clusters according to claim 3, characterized in that: The expression of the Mish activation function is: f(x)=x×tanh×[ln(1+e x )].

5. A method for identifying and tracking plasma clusters according to claim 3, characterized in that: The mathematical expression of the Dice Loss algorithm used in the loss function is: Among them, P represents the predicted probability of the weighted model, G represents the true label category of the sample, and p i and g i Represent the elements in P and G respectively, and N represents the number of elements.

6. A method for identifying and tracking plasma clusters according to claim 1, characterized in that: The SORT multi-target tracking algorithm includes a prediction module based on a standard Kalman filter and a matching module based on a Hungarian algorithm.

7. A method for identifying and tracking plasma clusters according to claim 6, characterized in that: The tracking of the pixel coordinate position of the plasma mass based on the SORT multi-target tracking algorithm includes: The SAUF connected domain labeling algorithm is used to label the pixel coordinates of the detected plasma clusters; The trajectory of the plasma cluster is predicted by the prediction module in the SORT multi-target tracking algorithm to obtain the predicted trajectory; The predicted trajectory of the plasma cluster is matched with the corresponding plasma cluster in the next frame through the matching module to obtain the running trajectory of the plasma cluster.

8. A method for identifying and tracking plasma clusters according to claim 6, characterized in that: The cost matrix corresponding to the matching module is calculated using the Mahalanobis distance; the calculation expression of the Mahalanobis distance is: Among them, C i,j is the (i, j)th element of the cost matrix C, d i,j is the square of the Mahalanobis distance between the predicted position of the ith trajectory and the jth detection position, and θ is the proximity threshold, which is used to eliminate trajectories and detection results that cannot be paired.

9. A method for identifying and tracking plasma clusters according to claim 1, characterized in that: The step of converting the plasma mass pixel coordinate position into the world coordinate system comprises: The homography matrix is ​​calculated based on the robust algorithm of RANSAC. The calculation expression of the homography matrix is: in, Where [u,v] T Represents the pixel coordinate system, [X, Y, Z] T represents the world coordinate system, M represents the homography matrix, M1 is the intrinsic parameter matrix, M2 is the extrinsic parameter matrix, λ is the scale factor, f is the focal length, dx and dy are the horizontal and vertical actual physical sizes of the pixel respectively, u0 and v0 are the horizontal and vertical coordinates in the pixel coordinate system, R is the rotation matrix, and T is the transformation matrix; The pixel coordinate system is converted into a world coordinate system through homography transformation.

10. A method for identifying and tracking plasma clusters according to claim 9, characterized in that: The homography matrix is ​​calculated using a robust algorithm based on RANSAC.

Images