Deep learning-based visible light video plasma breakdown prediction method and device

Abstract

The application discloses a visible light video plasma breakdown prediction method and device based on deep learning. The method comprises data acquisition and processing, which is used for training a visible light video plasma breakdown prediction model, and whether the plasma running video is broken is predicted through the prediction model. The data acquisition and processing comprises acquiring the plasma running video shot by a visible light camera, intercepting a video segment, intercepting a video segment with a set time length before the breakdown for the breakdown gun video, randomly intercepting a video segment with the same time length of the current flat segment for the non-breakdown gun, uniformly converting the intercepted video segments into the same frame rate, and pre-processing the video image. The application utilizes the visible light video signal to perform the breakdown prediction and realizes real-time prediction, has the potential of online deployment, and opens the possibility of future online breakdown warning and active control based on real-time video stream input.

Description

Technical Field

[0001] The present invention relates to the technical field of plasma rupture prediction, and in particular to a visible light video plasma rupture prediction method based on deep learning. Background Art

[0002] The plasma disruption prediction task involves inputting diagnostic signals into a model to learn disruption information, identify disruption precursors, and promptly trigger the disruption mitigation system to prevent disruption. Previous studies on disruption prediction have primarily relied on scalar zero-dimensional (0D) signals and one-dimensional (1D) profiles as inputs to the disruption predictor. These signals are screened by expert experience and are considered closely related to disruption. Current disruption prediction tasks lack the full utilization of high-dimensional visible light video signals. Visible light diagnostics, a common diagnostic tool for plasma operations, can monitor a variety of key disruption-related factors, such as plasma location and shape, impurity sputtering, hot spots, recycled particle radiation, MARFE, and other disruption-related plasma changes. Future tokamaks will be equipped with visible light / infrared systems covering the entire main chamber to maximize wall observation, providing valuable video data necessary for disruption event prediction and mitigation. Therefore, in preparation for ITER operations, intensified research is needed to demonstrate the feasibility of real-time disruption prediction based on video data. Summary of the Invention

[0003] The present invention proposes a deep learning-based visible light video plasma disruption prediction method, device, and storage medium, which can solve at least one of the technical problems in the background technology.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A deep learning-based visible light video plasma rupture prediction method includes data acquisition and processing to train a visible light video plasma rupture prediction model, and uses the prediction model to predict whether a plasma operation video will rupture;

[0006] Among them, data acquisition and processing include obtaining the plasma operation video taken by the visible light camera and intercepting the video clips. For the rupture cannon video, the video clip of the set time length before the rupture is intercepted. For the non-rupture cannon, the video clip of the same time length of the current flat-top section is randomly intercepted. The intercepted video clips are unified into the same frame rate, and then the video image is preprocessed.

[0007] Furthermore, the plasma rupture prediction model is constructed by a downsampling layer, a MetaFormer module, a BiLSTM, and a fully connected layer, wherein the token mixer in the MetaFormer module is implemented by an adaptive frequency filter (AFF) and a self-attention mechanism (Attention);

[0008] The model first extracts image features through the MetaFormer module to obtain a feature vector T. This feature vector is then input into the BiLSTM to learn the temporal information of the video. Finally, the fully connected layer outputs the prediction result. The BiLSTM can process both forward and backward information of the time series.

[0009] The model consists of N downsampling layers. The first downsampling layer consists of two 2D convolution kernels of the same size and a ReLU activation function. The next N-1 downsampling layers consist of two depthwise separable convolution kernels of different sizes and a ReLU activation function.

[0010] Each MetaForme module consists of a token mixer and a multilayer perceptron (MLP), preceded by layer normalization. The token mixer in the first N-1 MetaForme modules is implemented using adaptive frequency filtering. The input feature map undergoes a fast Fourier transform (FFT) to obtain a frequency-domain representation of the original image. After obtaining the corresponding frequency-domain representation, frequency filtering is used to preserve important frequency information. Finally, an inverse Fourier transform (IFT) is used to convert the filtered frequency representation into the updated original time-domain feature space. The token mixer in the final MetaForme module is implemented using a self-attention mechanism.

[0011] Furthermore, the frequency filter used by the model to filter frequency information is composed of two two-dimensional convolution kernels and a GELU activation function. The two different convolution kernels are calculated in parallel and the results are spliced ​​together. After the GELU activation function, they are input into two different convolution kernels for parallel calculation, and finally the filtered frequency information is output.

[0012] Furthermore, the self-attention mechanism can effectively capture the global dependencies in sequence data by modeling the relationship between different tokens.

[0013] First, the tokens are mapped into three different vectors: query Q, key vector K, and value vector V, which are used to calculate the attention scores between them. By calculating the dot product of query Q and key K, the attention score of each token can be obtained, reflecting the contribution of each token to the final task. Then, the result is normalized using the softmax function to obtain the attention weight of each token. Finally, the value vector V is weighted by these normalized attention weights and summed to generate the final attention output.

[0014] The calculation formula of the attention mechanism is shown in formula (2);

[0015] in, Indicates the dimension of the key vector K, which is used to scale the result of the dot product to prevent the gradient from being unstable due to excessive values;

[0016] .

[0017] Furthermore, the calculation method of query Q, key vector K and value vector V is shown in formula (1), where 、 and represents the trainable weight matrix, represents a sequence of input tokens;

[0018]

[0019]

[0020]

[0021] By calculating the dot product of the query Q and the key vector K, we can obtain the attention score of each token, which reflects the contribution of each token to the final task.

[0022] Furthermore, adaptive frequency filtering applies a two-dimensional fast Fourier transform to obtain tokens The corresponding frequency representation ;

[0023] The two-dimensional fast Fourier transform quickly converts the signal from the time domain to the frequency domain. The formula is expressed as :

[0024]

[0025] In formula (3), The frequency components of The features of different spatial positions in the process are obtained through a complexity of The transformation integrates the Then, a learnable mask is applied to filter Frequency representation of , thus achieving efficient token mixing;

[0026] Then, the filtered After inverse Fourier transform, the updated frequency representation in the original feature space is obtained , this process is expressed by formula (4):

[0027]

[0028] here, It is a learnable mask, which consists of two different sizes of convolution kernels 3×3 and 1×1, and is activated by the GELU function, plus two identical convolution kernels. After applying the mask, a skip connection is introduced to reduce the risk of overfitting. The symbol represents element-wise multiplication, and the symbol represents a skip connection. In addition, represents the inverse Fourier transform, which converts the filtered frequency domain back to the spatial domain.

[0029] Furthermore, the mask is implemented through a multi-branch structure, and the structure reparameterization technology is used to convert the multi-branch structure into a single-branch structure during inference. Specifically, the 1×1 convolution kernel is converted into a 3×3 convolution kernel, and then it is added to another 3×3 convolution kernel to form a single-branch structure. The combined 3×3 convolution kernel It is expressed by formula (5):

[0030]

[0031] Indicates that by filling The convolution kernel is converted to Convolution kernel; after padding, The central value is , while the surrounding values ​​are zero.

[0032] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0033] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0034] As can be seen from the above technical solution, the present invention's deep learning-based visible light video plasma fracture prediction method utilizes visible light video signals (consisting of continuous two-dimensional images) for fracture prediction. To cope with the high data throughput of video input while balancing real-time performance with accuracy, the present invention proposes a lightweight deep neural network that combines spatial and temporal feature extraction structures for feature learning and sequence modeling. The dataset is derived from plasma operation videos captured by multiple visible light cameras during tokamak operation. The wide-field-of-view, wide-angle visible light cameras distributed circumferentially in the tokamak enable comprehensive monitoring of plasma operation and detailed recording of dynamic processes. This present invention fully utilizes the advantages of these circumferentially distributed visible light cameras, creating a fracture predictor driven entirely by visible light video data.

[0035] This invention enables real-time plasma rupture prediction based on high-dimensional visible light video diagnostics, potentially enabling online deployment. This opens up the possibility of future online rupture warning and active control based on real-time video stream input. This highlights the potential of video data for rupture warning and suggests that future rupture warning databases should combine video signals with traditional signals to provide more comprehensive and robust features for rupture prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of the present invention;

[0037] Figure 2 This is a structural diagram of a spatial extractor according to an embodiment of the present invention;

[0038] Figure 3 This is a structural diagram of an adaptive frequency filter according to an embodiment of the present invention;

[0039] Figure 4 This is a diagram of the reparameterized structure of an embodiment of the present invention;

[0040] Figure 5 This is a diagram of the image fusion structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0042] like Figure 1As shown in the example, the deep learning-based visible light video plasma rupture prediction method described in this embodiment uses data acquisition and processing to construct a model training dataset. A plasma operation video captured by a visible light camera is captured and video clips are captured. For rupture cannon videos, a certain length of video clip is captured before the rupture. For non-rupture cannon videos, random clips of the same length of time during the current plateau are captured. These captured video clips are then unified to the same frame rate, and the video images are then preprocessed.

[0043] Among them, video image preprocessing is to crop the images in the processed video clips into a uniform size to obtain the preprocessed plasma rupture video, and use a sliding window with a fixed window size and sliding step to slice the preprocessed video clips, and obtain training samples after each video slice.

[0044] The neural network model for plasma burst prediction consists of a downsampling layer, a MetaFormer module, a BiLSTM, and a fully connected layer. The token mixer in the MetaFormer module is implemented using adaptive frequency filtering (AFF) and a self-attention mechanism. The model first extracts image features through the MetaFormer structure to obtain a feature vector T. This feature vector is input into the BiLSTM to learn the temporal information of the video. Finally, the model outputs the prediction result through the fully connected layer.

[0045] The model contains N downsampling layers. The first downsampling layer consists of two two-dimensional convolution kernels of the same size and a Relu activation function. The next N-1 downsampling layers consist of two depth-wise separable convolution kernels of different sizes and a Relu activation function.

[0046] Each MetaForme module consists of a token mixer and a multi-layer perceptron (MLP), preceded by layer normalization. The token mixer in the first N-1 MetaForme modules is implemented using adaptive frequency filtering. The input feature map undergoes a fast Fourier transform (FFT) to obtain a frequency-domain representation of the original image. Once this representation is obtained, frequency filtering is used to preserve important frequency information. Finally, an inverse Fourier transform (IFT) is used to convert the filtered frequency representation into the updated original time-domain feature space. This process effectively blends the global features of the image, focusing on information across the entire image. The token mixer in the final MetaForme module is implemented using a self-attention mechanism.

[0047] Among them, the frequency filter used to filter frequency information is composed of two two-dimensional convolution kernels and GELU activation function. After the two different convolution kernels are calculated in parallel, the results are spliced ​​together, and after passing through the GELU activation function, they are input into two different convolution kernels for parallel calculation, and finally the filtered frequency information is output.

[0048] In order to solve the computational consumption caused by the dual-branch structure in frequency filtering and speed up the reasoning speed of the model, the present invention uses structural reparameterization in the model reasoning stage to convert the dual-branch structure in frequency filtering into a single branch, thereby speeding up the reasoning speed of the model.

[0049] The self-attention mechanism effectively captures global dependencies in sequential data by modeling the relationships between different tokens. Tokens are first mapped into three different vectors: query (Q), key (K), and value (V), which are used to calculate attention scores between them. By calculating the dot product of Q and K, an attention score for each token is obtained, reflecting its contribution to the final task. The result is then normalized using the softmax function to obtain an attention weight for each token. Finally, the value vector (V) is weighted by these normalized attention weights and summed to generate the final attention output.

[0050] Among them, BiLSTM can simultaneously process the forward and backward information of the time series. Its bidirectional structure can effectively capture complete temporal features and contextual information, making it more suitable for tasks that require a comprehensive understanding of video data and capture the temporal dependencies in plasma video data.

[0051] Among them, the fully connected layer outputs the final rupture probability.

[0052] The following examples illustrate:

[0053] Dataset:

[0054] The video was sliced ​​into 100 ms time windows to obtain training samples to help the model learn the dynamic evolution of plasma disruption. For non-disruptive discharges, training samples were randomly selected from the flat-top phase and their labels were set to 0. For each disruptive discharge, a sliding window was performed with a step size of 10 ms starting from 200 ms before the disruption until the end of the disruption. The label of each sample was set to the probability of disruption at the last time point in the time window. The sample label is defined as shown in the following equation:

[0055]

[0056] in, t Indicates the time until rupture. In a rupture sample, the label of the first time slice is set to 0, while the label of the last time slice is set to 1 10 ms before rupture.

[0057] Deep Image Rupture Prediction Model

[0058] Plasma disruption is a dynamic process, making it crucial to simultaneously capture detailed information about both the instantaneous characteristics of the plasma and its temporal evolution. Previous disruption prediction models typically employ a hybrid neural network architecture combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to complete the prediction process. However, because plasma information is distributed across the entire image in D-shaped cross-sections, CNNs often perform poorly in visible light video-based disruption prediction. This phenomenon is primarily attributed to the limited receptive field of CNN neurons, which restricts their focus to local regions of the image and makes it difficult to effectively capture the global image features required for accurate prediction. To address this issue, the MetaFormer architecture, a general framework derived from the Transformer model that better processes global image features, was adopted. In the temporal dimension, LSTMs learn time series information by incorporating past states, but they process sequence data only in the forward direction, potentially losing early information in long sequences. In contrast, bidirectional LSTMs (BiLSTMs) process time series in both the forward and reverse directions. This bidirectional architecture effectively captures comprehensive temporal features and contextual information, making it more suitable for tasks requiring a comprehensive understanding of video data. Based on these considerations, this paper proposes a visible-light video-based rupture predictor. This model combines the capabilities of MetaFormer and BiLSTM to capture global image features of the plasma and the temporal dependencies of the video data. Furthermore, to improve the model's inference speed and realize its potential for online deployment, special attention was paid to its lightweight design.

[0059] like Figure 1As shown in the figure, the model consists of two main components: a spatial feature extractor and a temporal feature extractor. Each training sample is generated by slicing the video sequence with a sliding window of 100 ms and a step size of 10 ms. The spatial feature extractor processes each frame in the sample sequentially, generating a corresponding 1D feature vector. The temporal feature extractor then learns temporal information from all 1D feature vectors and outputs a final rupture probability. The spatial feature extractor consists of four stages operating at different scales. Each stage includes patch embedding and the use of MetaFormer. MetaFormer consists of two main modules: a token mixer and a multi-layer perceptron (MLP). Layer normalization (LN) is applied before each module, and skip connections are added after each patch. The token mixer combines tokens representing different spatial locations, enabling the model to capture global image information and learn relationships between them. The self-attention (SA) mechanism was originally developed for natural language processing tasks using the Transformer and has also demonstrated strong performance in vision tasks. It calculates attention weights between tokens and performs weighted summation to capture the long-range relationship of tokens. However, the SA-based token mixer has The computational complexity of , makes it difficult to process high-resolution feature maps containing a large number of tokens in the early stages of the model.

[0060] To optimize this problem, this paper proposes a lightweight token mixer with a computational complexity of , implemented by a global adaptive frequency filter (AFF) using the Fast Fourier Transform (FFT). This approach replaces the SA-based token mixer in the first three stages of the model to efficiently process high-resolution feature maps. In the last stage of the model, we only use the SA-based token mixer and include four layers of multi-head self-attention (MHSA) blocks. The specific structure is as follows Figure 2 shown.

[0061] Self-Attention Mechanism:

[0062] The self-attention (SA) mechanism effectively captures global dependencies in sequential data by modeling the relationships between tokens. First, tokens are mapped into three different vectors: query (Q), key (K), and value (V). These vectors are used to calculate the attention scores between tokens, such as Figure 4 The calculation method of Q, K and V is shown in formula (1), where 、 and represents the trainable weight matrix, Represents a sequence of input tokens.

[0063]

[0064]

[0065]

[0066] By calculating the dot product of Q and K, the attention score of each token can be obtained, which reflects the contribution of each token to the final task. The calculation formula of the attention mechanism is shown in formula (2). The dimension of the key vector (K) is used to scale the dot product result to prevent excessively large values ​​from causing gradient instability. Next, the result is normalized using the softmax function to obtain normalized attention weights for each token. Finally, the value vector (V) is weighted by these normalized attention weights and summed to obtain the final attention output.

[0067]

[0068] The multi-head self-attention (MHSA) mechanism extends the self-attention mechanism by linearly mapping the query, key, and value vectors into multiple subspaces. By learning different representations in parallel across multiple attention heads, these heads extract information from multiple perspectives, enhancing the model's expressiveness and ability to capture diverse features. Furthermore, by independently processing information in different subspaces, multi-head attention significantly improves computational efficiency, accelerating both training and inference. Therefore, the model employs the MHSA block and sets the number of self-attention heads to 16.

[0069] Adaptive frequency filtering:

[0070] This invention introduces a novel token mixer that combines frequency filtering design while maintaining excellent performance. Figure 3 As shown, a two-dimensional fast Fourier transform is applied to obtain the token The corresponding frequency representation The two-dimensional fast Fourier transform is an efficient algorithm that can quickly convert signals from the time domain to the frequency domain. The formula is expressed as :

[0071]

[0072] In formula (3), The frequency components of This process is done through a complexity of The transformation integrates the This also defines the overall time complexity of AFF. Next, a learnable mask is applied to filter The frequency representation of , thus achieving efficient token mixing. After inverse Fourier transform, the updated frequency representation in the original feature space is obtained This process can be expressed by formula (4):

[0073]

[0074] here, It is a learnable mask composed of two different sizes of convolution kernels (3×3 and 1×1), activated by the GELU function, and two identical convolution kernels, such as Figure 4 After applying the mask, skip connections are introduced to mitigate the risk of overfitting. Symbol represents element-wise multiplication, and the symbol represents a skip connection. In addition, represents the inverse Fourier transform, which converts the filtered frequency domain back to the spatial domain.

[0075] The mask is implemented through a multi-branch structure with convolution kernels of different sizes. During the model training process, this structure enables the model to learn multi-scale frequency information. However, this multi-branch approach usually leads to increased model complexity, which in turn leads to more parameters and computational costs, thus affecting the inference speed. In order to achieve faster inference while maintaining accuracy, the present invention adopts a structural reparameterization technique to convert the multi-branch structure into a single-branch structure during inference, such as Figure 4 Specifically, the 1×1 convolution kernel is converted into a 3×3 convolution kernel, which is then added to another 3×3 convolution kernel to form a single-branch structure. The combined 3×3 convolution kernel It can be expressed by formula (5):

[0076]

[0077] Here, Indicates that by properly filling The convolution kernel is converted to Convolution kernel. Specifically, after padding, The central value is By using a single-branch mask and a The convolution kernel increases inference speed by 14%, significantly improving the efficiency of the model in online deployment.

[0078] The following are the relevant results and analysis of the present invention

[0079] High-frequency video data:

[0080] During model testing, the test set samples are sliced ​​and fed into the model in chronological order. The predicted result for each sample is the probability of rupture at the last moment of the time slice. If the probability is greater than a given threshold, an alarm is issued, recorded as a true positive (TP). If it is less than the threshold, a missed alarm is recorded as a true negative (TN). For non-rupture shots, a probability greater than the threshold is a false alarm (FP), while a probability less than the threshold indicates accurate identification of the non-rupture and no alarm is issued, recorded as a true negative (TN). The threshold is determined by the validation set, which is fed into the model to calculate the F1 score at different thresholds. The threshold with the highest F1 score is considered the optimal threshold.

[0081]

[0082] Where precision is TP / (TP+FP) and recall is TP / (TP+FN). At the optimal threshold, the AUC value of the test set is 0.98.

[0083] Furthermore, on the test set, the model achieved an average warning time of 700ms. High-frequency video data demonstrated excellent alarm accuracy, validating the effectiveness of visible light video for plasma disruption prediction. While a high sampling rate can capture more dynamic evolution details, it also results in a longer model inference time of approximately 28ms, hindering real-time prediction and online deployment. Therefore, experiments were conducted using relatively low-resolution visible light video data.

[0084] Low-frequency video data:

[0085] Low-frequency video data comes from two visible light cameras, with separate training and testing for the model. At the optimal threshold, the AUC values ​​for the test set were both 0.97, and the average warning times for the models reached 284ms and 301ms, respectively. Notably, the model's inference time decreased significantly, to under 4.5ms. This is because when acquiring at 100fps, the interval between each frame is 10ms, which can lead to missed details. For some types of ruptures, which last for a short time, noticeable changes may only occur in one or two frames close to the rupture, resulting in a high probability of misclassification as a non-rupture. Consequently, the statistical average time is reduced. To fully utilize the monitoring advantages of the circumferentially distributed, wide-field-of-view visible light cameras and improve the warning time and accuracy of rupture warnings, a video image fusion experiment captured by different cameras was conducted.

[0086] Multi-window video image fusion:

[0087] Circumferentially distributed visible cameras provide different fields of view of the plasma state. To fully explore the significance of circumferential multi-field visible video images for rupture warning, rupture prediction research using multi-window image fusion was conducted. High-frequency data was downsampled to be consistent with low-frequency data.

[0088] This invention adopts feature-level image fusion technology to enhance the detectability and recognition rate of images by fully fusing feature maps after feature extraction. The specific fusion method is as follows Figure 5 As shown in the figure, first, adaptive frequency filtering is used to extract features from three different image types. These extracted feature maps are then concatenated to form a comprehensive feature map. This comprehensive feature map is then fed into a self-attention block for further processing, blending and optimizing the concatenated features. Finally, an LSTM network analyzes and predicts the processed features, outputting the fracture probability. This method not only fully utilizes the feature information from different image types but also improves the accuracy and stability of fracture probability prediction through the combination of the self-attention mechanism and LSTM.

[0089] On the test set, the model's average warning time reached 456ms, while its inference time was only approximately 7ms, demonstrating its potential for online deployment. At the same sampling rate, multi-window fusion achieves higher warning times than single-window fusion. This is because when plasma rupture precursors appear, only one or two cameras may be able to observe them. Cameras that cannot clearly observe them will issue warnings later. In single-window rupture studies, these phenomena manifest inconsistently, resulting in inconsistent warning times. With fusion, all information is included. Even if some cameras fail to detect rupture precursors in time, as long as at least one camera detects signs of rupture, the model will issue a warning, significantly improving model performance.

[0090] Deep learning models are often viewed as black boxes when solving complex problems, making it difficult to accurately explain their decision-making processes. To enhance model interpretability, this paper employs Grad-CAM (Gradient-weighted Class Activation Mapping) to visualize the model's regions of interest. Grad-CAM is an advanced technique for interpreting and visualizing deep learning models. It combines gradient information with feature maps to generate heat maps called class activation maps. This method utilizes the output of the model's final convolutional layer and the corresponding gradient information to intuitively display the important regions of interest in the image that the model is focusing on.

[0091] Specifically, Grad-CAM uses the model's final convolutional layer output and the corresponding gradient information to generate a heatmap that visually demonstrates the model's attention to the image. In this case, Grad-CAM shows that the model primarily focuses on the plasma's outline and internal structure. This visualization helps us understand how the model makes decisions based on the input image and provides insights into its inner workings.

[0092] Grad-CAM technology enables a more intuitive understanding of the image features underlying deep learning models' predictions, thereby enhancing model interpretability. This visualization method provides more information about the underlying model decisions, helping to further optimize model design and training, improving both performance and understandability. The application of Grad-CAM transforms deep learning models from unexplainable black boxes into more transparent and interpretable tools, providing a new avenue for in-depth exploration of the models' underlying mechanisms.

[0093] In summary, plasma disruption poses a significant safety challenge for future tokamaks and fusion reactors. Previous work on disruption prediction has primarily relied on scalar zero-dimensional (0D) signals and one-dimensional (1D) profiles as predictor inputs. This paper explores the feasibility of utilizing visible light video signals (composed of sequential 2D images) for disruption prediction. To address the high data throughput of video input and balance real-time performance with prediction accuracy, a lightweight deep neural network combining spatial and temporal feature extraction structures is proposed for feature learning and sequence modeling. The dataset is derived from a large field-of-view (FOV) wide-angle visible camera on a distributed tokamak. The model is trained and tested at different sampling rates using single-window data and multi-window fusion data, achieving state-of-the-art disruption prediction capabilities on multi-window fusion data. Notably, the model's inference time is only a few milliseconds, while the average warning time is as long as 450 milliseconds, opening the door to online disruption warning and active control based on real-time video stream input. These preliminary results highlight the potential of video data in rupture early warning and suggest that future rupture early warning databases should combine video signals with traditional signals to provide more comprehensive and robust features for rupture prediction.

[0094] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0095] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0096] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the deep learning-based visible light video plasma disruption prediction methods in the above embodiments.

[0097] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts of the above methods.

[0098] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0099] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0100] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0101] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for predicting plasma rupture from visible light video based on deep learning, characterized in that: This includes data acquisition and processing to train a visible light video plasma rupture prediction model, which can be used to predict whether a plasma operation video will rupture. Data acquisition and processing includes acquiring plasma operation videos captured by a visible light camera and capturing video clips. For rupture cannon videos, a video clip of a set length of time before the rupture is captured. For non-rupture cannons, a video clip of the same length of time during the current flat-top section is randomly captured. The captured video clips are unified into the same frame rate, and then the video images are pre-processed. The plasma rupture prediction model is constructed by a downsampling layer, a MetaFormer module, a BiLSTM, and a fully connected layer, wherein the token mixer in the MetaFormer module is implemented by an adaptive frequency filter (AFF) and a self-attention mechanism (Attention); The model first extracts image features through the MetaFormer module to obtain a feature vector T. This feature vector is then input into the BiLSTM to learn the temporal information of the video. Finally, the fully connected layer outputs the prediction result. The BiLSTM can process both forward and backward information of the time series. The model consists of N downsampling layers. The first downsampling layer consists of two 2D convolution kernels of the same size and a ReLU activation function. The next N-1 downsampling layers consist of two depthwise separable convolution kernels of different sizes and a ReLU activation function. Among them, each MetaForme module contains a token mixer and a multi-layer perceptron (MLP), with layer normalization acting before them. The token mixer in the first N-1 stage MetaForme module is implemented by adaptive frequency filtering. The input feature map is first subjected to a fast Fourier transform to obtain the frequency domain representation of the original image. After obtaining the corresponding frequency domain representation, frequency filtering is used to retain important frequency information. Finally, the filtered frequency representation is converted into the updated original time domain feature space using an inverse Fourier transform. The token mixer in the last stage MetaForme module is implemented using the self-attention mechanism Attention.

2. The method for predicting plasma disruption using visible light video based on deep learning according to claim 1, wherein: The frequency filter used by the model to filter frequency information consists of two two-dimensional convolution kernels and a GELU activation function. The two different convolution kernels are calculated in parallel, and the results are spliced ​​together. After the GELU activation function, they are input into two different convolution kernels for parallel calculation, and finally the filtered frequency information is output.

3. The method for predicting plasma disruption using visible light video based on deep learning according to claim 1, characterized in that: The self-attention mechanism Attention can effectively capture the global dependencies in sequence data by modeling the relationship between different tokens; First, the tokens are mapped into three different vectors: query Q, key vector K, and value vector V, which are used to calculate the attention score between them. By calculating the dot product of query Q and key K, the attention score of each token is obtained, which reflects the contribution of each token to the final task. Then, the result is normalized using the softmax function to obtain the attention weight of each token. Finally, the value vector V is weighted by these normalized attention weights and summed to generate the final attention output; The calculation formula of the attention mechanism is shown in formula (2); in, Indicates the dimension of the key vector K, which is used to scale the result of the dot product to prevent the gradient from being unstable due to excessive values; 。 4. The method for predicting plasma disruption using visible light video based on deep learning according to claim 3, characterized in that: The calculation method of query Q, key vector K and value vector V is shown in formula (1), where 、 and represents the trainable weight matrix, represents a sequence of input tokens; By calculating the dot product of the query Q and the key vector K, the attention score of each token is obtained, which reflects the contribution of each token to the final task.

5. The method for predicting plasma disruption using visible light video based on deep learning according to claim 1, characterized in that: Adaptive frequency filtering applies a two-dimensional fast Fourier transform to obtain tokens The corresponding frequency representation ; The two-dimensional fast Fourier transform quickly converts the signal from the time domain to the frequency domain. The formula is expressed as : In formula (3), The frequency components of The features of different spatial positions in the process are obtained through a complexity of The transformation integrates the Then, a learnable mask is applied to filter Frequency representation of , thus achieving efficient token mixing; Then, the filtered After inverse Fourier transform, the updated frequency representation in the original feature space is obtained , this process is expressed by formula (4): here, It is a learnable mask, which consists of two different sizes of convolution kernels 3×3 and 1×1, and is activated by the GELU function, plus two identical convolution kernels. After applying the mask, a skip connection is introduced to reduce the risk of overfitting. The symbol represents element-wise multiplication, and the symbol represents a skip connection. In addition, represents the inverse Fourier transform, which converts the filtered frequency domain back to the spatial domain.

6. The method for predicting plasma disruption using visible light video based on deep learning according to claim 5, characterized in that: The mask is implemented through a multi-branch structure, which uses a structural reparameterization technique to convert the multi-branch structure into a single-branch structure during inference. Specifically, the 1×1 convolution kernel is converted into a 3×3 convolution kernel, and then added to another 3×3 convolution kernel to form a single-branch structure. The combined 3×3 convolution kernel It is expressed by formula (5): Indicates that by filling The convolution kernel is converted to Convolution kernel; after padding, The central value is , while the surrounding values ​​are zero.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 6.

Images