Modeling method of 4-level pulse amplitude modulation high-speed transmitter based on non-autoregressive Transformer

Abstract

The present invention discloses a modeling method for a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer. The method predicts the behavior of a 4-level pulse amplitude modulation transmitter by establishing a deep learning model with an encoder-decoder architecture. The encoder generates a context vector by processing disordered non-sequential inputs, including input signal parameters and link parameters, and passes it to the decoder. The decoder combines the context vector generated by the encoder with the transmitter output signal sequence to generate a category probability distribution for each point in the sequence. The model is trained using a random masking strategy and inference is performed using non-autoregressive decoding and filtering. The model can predict the output sequence in parallel and perform a filtering process to predict the output signal. Compared with traditional simulation methods, the present invention achieves a significant acceleration effect, which is particularly evident when processing multi-link systems.

Description

Technical Field

[0001] The present invention relates to the field of high-speed communication link analysis, and in particular to a modeling method for a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer. Background Art

[0002] With the development of electronic and communication technologies, high-speed serial links play a vital role in achieving low-latency and high-bandwidth communications. They are widely used in emerging data-driven applications such as artificial intelligence (AI), 5G mobile networks, and automotive technologies. To meet the energy-efficient, cost-effective, and high-performance system requirements of these applications, chiplet-based high-density heterogeneous integration (HDHI) technology has gradually become mainstream and is widely used in various applications.

[0003] High-speed serial links are key components of these advanced high-speed systems. To meet the growing demand for high bandwidth and efficient communications, these links feature highly dense signal paths, ranging from hundreds to thousands, and operate at high frequencies and high data rates, facing complex signal integrity (SI) issues such as crosstalk, signal attenuation, and electromagnetic interference (EMI). The transmitter (TX) is one of the most critical and resource-intensive components in a high-speed link, and its performance has a direct impact on the quality of the initial transmission signal and the integrity of the entire link. Degradation of the transmitter's output signal can cause significant distortion at the final receiver, leading to errors in the recovery of the transmitted signal. To maintain high-quality and high-speed output signals and ensure signal integrity along the transmission path, the transmitter needs to operate at high frequencies and minimize timing errors caused by variations in process, voltage, and temperature (PVT) and load conditions.

[0004] The main shortcomings of existing technologies are as follows: 1. Accurate transistor-level models, such as SPICE models, require long calculation times due to the complex internal circuit details; empirical behavioral models, such as those based on the current source model (CSM) and I / O buffer information specification (IBIS), are fast but generally have limited accuracy or ability to model complex interdependencies. 2. Compared to non-return-to-zero (NRZ) signals, four-level pulse amplitude modulation (PAM4) signals increase the complexity of signal processing. Their smaller level intervals are more sensitive to noise and interference, reducing noise margins and requiring higher signal-to-noise ratio processing capabilities. They are also more susceptible to nonlinear effects such as distortion and inter-symbol interference (ISI) in high-frequency transmission.

[0005] Based on the above problems, a 4-level pulse amplitude modulation high-speed transmitter modeling method based on non-autoregressive Transformer is proposed to achieve efficient modeling of 4-level pulse amplitude modulation high-speed communication link transmitter. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention proposes a 4-level pulse amplitude modulation high-speed transmitter modeling method based on a non-autoregressive Transformer. The method can consider the influence of link parameters and multi-link signal crosstalk on the output signal of the transmitter circuit, and efficiently model the output of the transmitter circuit for the 4-level pulse amplitude modulation signal.

[0007] The object of the present invention is achieved through the following technical solutions:

[0008] A 4-level pulse amplitude modulation high-speed transmitter modeling method based on a non-autoregressive Transformer, the method comprising the following steps:

[0009] Step 1: Obtain training data through circuit simulation: Input different PAM4 signals under different link parameters and obtain the transmitter circuit output signal as training data;

[0010] Step 2: Build a Transformer-based deep learning model: Build a deep learning model using an encoder-decoder architecture, consisting of a non-sequential encoder and a Transformer sequential decoder.

[0011] Step 3: Model training based on random masking: For an input signal of a given length, the length of the output signal sequence can be predetermined and does not change with the input. A random masking strategy is used to train the deep learning model, enabling the model to predict elements at the mask position.

[0012] Step 4, model inference based on non-autoregressive decoding and filtering: input link parameters and the input signal of the interfered link into the deep model, input K = 0, and obtain the intrinsic output signal under this set of link parameters and the input signal itself through non-autoregressive decoding and filtering; then for all other interfering links, input link parameters and the interfering link input signal respectively, input K = 1, and obtain the crosstalk output signal under this set of link parameters and the interfering input signal through non-autoregressive decoding and filtering; add the intrinsic output signal with all other crosstalk output signals to obtain the output signal of the transmitter in multiple communication links affected by other signals.

[0013] Furthermore, the link parameters in step 1 include the equalizer coefficient H0, the power supply voltage V h , Transmitter load capacitance C L , S parameters S of the two transmission lines tl , the equivalent resistance Z0 of the receiving end and the pull-up voltage V p , PAM4 input signal x sIt is a sequence of "0" and "1". Each waveform transmits a 2-bit signal. There are 4 level values, representing "00", "01", "10", and "11". It is input to the link transmitter in a trapezoidal wave mode and is transmitted through the highest level voltage V h , signal period t p , the ratio of rise / fall time to signal period r rf Description; Step 1 specifically includes:

[0014] Step 1.1, obtain the intrinsic output signal sample when there is no crosstalk:

[0015] Step 1.2, obtain crosstalk signal samples:

[0016] Step 1.3, convert the continuous output voltage signal sequence into a discrete voltage category sequence: for the intrinsic output and crosstalk output, their entire voltage range is divided into discrete voltage categories by a step size of Δv. I or Δv C Divide into several voltage categories and build a dictionary of intrinsic outputs and a dictionary of crosstalk outputs to map voltage values ​​to their nearest category.

[0017] Furthermore, the step 1.1 specifically includes:

[0018] Step 1.1.1, input signal x s applied to the first link while keeping the second link input low;

[0019] Step 1.1.2, record the output signal of the first link as the intrinsic output signal sample y intr .

[0020] Furthermore, the step 1.2 specifically includes:

[0021] Step 1.2.1, input signal x s Applied to the second link while keeping the first link input high;

[0022] Step 1.2.2: record the interference signal of the second link to the first link and subtract the intrinsic output signal y from it. intr , get the crosstalk signal sample C ij .

[0023] Furthermore, the step 1.3 specifically includes:

[0024] Step 1.3.1, dictionary construction: For intrinsic output, define its voltage range as According to the step size Δv I Divide the voltage range and generate a series of discrete voltage categories to construct the dictionary D I :

[0025]

[0026] For the crosstalk output, its voltage range is defined as According to the step size Δv C Divide the voltage range and generate a series of discrete voltage categories to construct the dictionary D C :

[0027]

[0028] Voltage value v k Corresponding to the k+1th category Class k+1 , where Class0 is assigned to D I and D C A special one <mask>Elements, representing masks;

[0029] Step 1.3.2, voltage value to voltage category mapping: For each output voltage value, map each voltage value to its nearest voltage value v k , and then use the corresponding dictionary to find its nearest voltage category Class k+1 , thereby converting the continuous voltage signal sequence into a discrete voltage category sequence, and using the discrete voltage category sequence as the training target of the deep learning model in subsequent training.

[0030] Furthermore, the non-sequential encoder in step 2 is used to process the disordered non-sequential input, generate a context vector, and pass it into the Transformer sequence decoder. The encoding method of each input feature is as follows:

[0031] Step 2.1.1, use a Boolean variable K to distinguish between the predicted intrinsic output signal and the crosstalk output signal, and convert the binary variable K through a 2×d model The embedding matrix is ​​encoded as a d model Vector of dimension K e , K e represents the K-encoded vector, d model is the dimension of the encoding vector, which is the same as the model dimension of the Transformer sequence decoder;

[0032] Step 2.1.2, for the scalar features H0, V h 、C L , Z0, V p , t p 、r rf Standardization and encoding: scalar features H0, V h 、C L , Z0, V p , t p 、r rf They are standardized by subtracting their respective means and dividing by their standard deviations to reduce scale differences and accelerate convergence; the standardized features are converted to d through a multi-layer perceptron layer. model dimensional vector; each feature passes through a multilayer perceptron layer with different parameters and the same structure; each multilayer perceptron layer has two hidden layers, each containing 16 neurons, the hidden layer uses the ReLU activation function, and the output layer uses the linear activation function; finally, the scalar features H0 and V h 、C L , Z0, V p , t p 、r rf The encoded vectors are expressed as

[0033] Step 2.1.3, input signal sequence encoding: PAM4 signal has 4 levels, with a total of 12 types of directional level transitions; first traverse the input sequence x of length m s , when x i <x i+1 Mark the rising edge at position i+1 Or when x i >x i+1 Mark the falling edge at position i when Where i = 0, ..., m-2; since the signal is at a low level before and after transmission, for a non-zero start bit x0, a rising edge is marked at position 1 For non-zero trailing bits x m-1 , marking a falling edge at position m 0 represents an invalid position; each type of transition occurs at most m' times, where if m is even, m' is defined as m / 2, and if m is odd, m' is defined as (m+1) / 2; pad with 0 to ensure uniform length; in this way, x s is converted into m' discrete position indexes, and then through 12 (m+1)×4 m The embedding matrix of dimension is embedded into a set of continuous 4 m dimensional vector; the position index embedding vectors of these different types of transformations are respectively input with a dimension of 4 according to different transformation types. m Each multilayer perceptron layer has 2 hidden layers, each layer contains 16 neurons, the hidden layer uses the ReLU activation function, and the output layer uses the linear activation function. After stacking the multilayer perceptron layers, we finally get Contains 12×m′ vectors, each of dimension d model ;

[0034] Step 2.1.4, S-parameter encoding: By decomposing the S-parameter matrix into real and imaginary matrices, taking into account the symmetry of the S-parameters themselves, the 4×4 S-parameter matrix in the two-transmission-line system is reconstructed into an S-parameter matrix with 10 effective elements and a shape of 2×5; each element s in the S-parameter matrix is ​​translated and logarithmic transformation is applied:

[0035] s scaled =log(s+1.1*min(S tl ))

[0036] Among them S tl Represents the S parameter matrix of the transmission line, min(S tl ) represents the smallest element in the matrix, s is S tl Elements, s scaled is the scaling result after translation and logarithmic transformation; two consecutive convolutional neural network layers are used to process the S parameter matrix of each frequency point. The first convolutional neural network layer expands the two input matrix channels corresponding to the real and imaginary parts into 16 output channels, and the second convolutional neural network layer further expands it to 32 output channels. The size of each convolution kernel is 1, and the convolutional neural network layer uses the ReLU activation function; the final feature map is flattened and then passed through a linear layer to generate a dimension of (len f ,d model )of Indicates S tl The encoded vector, where d model is the dimension of the encoding vector, len f is the number of S parameter frequency points;

[0037] Step 2.1.5, encode features Combined into a set of unordered dimensional vectors X e , length is 8+len f +12*m′, input to the Transformer sequence decoder, and interact with the output sequence.

[0038] Furthermore, the Transformer sequence decoder in step 2 is used to combine the context vector generated by the non-sequential encoder and the category sequence output by the transmitter to generate a category probability distribution for each point in the sequence. The specific steps are as follows:

[0039] Step 2.2.1, Input Embedding and Position Encoding: First, embed the category sequence after the output signal is converted, converting each category into a fixed-dimensional vector representation; add the position encoding to the embedding vector to preserve the position information in the input sequence; the position encoding can be generated by sine and cosine functions:

[0040]

[0041] where d model is the embedding dimension, i is the embedding dimension index, pos is the position in the sequence, PE (pos,k) Represents the position encoding value at the k-th dimension index at position pos. The encoding of the even-numbered dimension index at position pos is calculated using the sine function, and the encoding of the odd-numbered dimension index is calculated using the cosine function.

[0042] Step 2.2.2, Multi-head Self-Attention Mechanism: The input vector X is separated into the query vector Q, key vector K and value vector V through linear transformation:

[0043] Q=XW Q ,K=XW K ,V=XW V

[0044] Where W Q ,W K ,W V The linear transformation matrices representing the query vector Q, key vector K, and value vector V are used to transform the input vector X into the query vector Q, key vector K, and value vector V respectively;

[0045] Calculate the dot product of the query vector Q and the key vector K, divide it by the scaling factor, and use the softmax function to obtain the attention weight:

[0046]

[0047] where d k represents the dimension of the key vector K, Attention(Q,K,V) represents the output of the attention function, and the query vector Q is used to compare with the transpose K of each key vector K. T Compute the dot product and divide by the scaling factor After that, the scores are used to measure the importance of the corresponding value vector V; after processing by the softmax function, these scores are converted into weights, and then these weights are multiplied by the value vector V to obtain the weighted output, that is, the result after attention concentration. Softmax(*) represents the softmax function;

[0048] The attention mechanism is split into multiple heads, which are calculated independently and then the results are merged:

[0049] MultiHead(Q,K,V)=Concat(head1,…,head h )W O ,

[0050] in

[0051] in represents the linear transformation matrix used in the i-th head, which is used to transform the original query vector Q, key vector K and value vector V into the corresponding space processed by the head; head1,…,head h Represents the output of each attention head, Concat(head1,…,head h )W O Indicates that the output vectors of all heads are concatenated and then transformed by a linear transformation W O Merge the information of these different heads, MltiHead(Q, K, V) is the output of the multi-head attention mechanism; and perform layer normalization after residual connection;

[0052] Step 2.2.3, multi-head cross attention: The input vector X and the encoding vector from the non-sequential encoder, i.e., the context vector C, are linearly transformed to generate the query vector Q, key vector K, and value vector V respectively:

[0053] Q=XW Qc ,K=CW Kc ,V=CW Vc

[0054] Where W Qc ,W Kc ,W Vc are the linear transformation matrices in the cross-attention, which transform the input vector X and the context vector C into the corresponding query vector Q, key vector K and value vector V;

[0055] Calculate the dot product of the query vector Q and the key vector K, divide it by the scaling factor, and use the softmax function to obtain the attention weight:

[0056]

[0057] where d k represents the dimension of the key vector K, Attention(Q,K,V) represents the output of the attention function, the query vector Q from the input vector X is used to compare with the transpose K of each key vector K converted from the context vector C. T Compute the dot product and divide by the scaling factor Afterwards, the scores obtained are used to measure the importance of the corresponding value vector V; after processing through the softmax function, these scores are converted into weights, and then these weights are multiplied with the value vector V to generate a weighted output, which is the result of cross attention concentration;

[0058] Cross attention also uses a multi-head mechanism, and the results are merged after independent calculations:

[0059] MultiHead(Q,K,V)=Concat(head1,…,head h )W Oc ,

[0060] in

[0061] Each of the heads Indicates the use of different linear transformation matrices The query vector Q, key vector K and value vector V are converted to the corresponding space processed by the head; the output of each attention head is concatenated and subjected to a linear transformation W Oc Merge to form the final cross-attention output; finally, the output is connected through the residual and layer normalized;

[0062] Step 2.2.4, bit-by-bit feedforward network: process the vector of each position independently, through two layers of linear transformation and ReLU activation function:

[0063] FFN(x)=max(0,xW1+b1)W2+b2

[0064] The FFN is a two-layer network structure that operates independently at each position. It first performs a linear transformation xW1+b1 on the encoding vector x at each position, then introduces nonlinear processing through the ReLU activation function max(0,z), and then performs a second linear transformation max(0,xW1+b1)W2+b2. Here W1 and W2 are weight matrices, and b1 and b2 are bias terms. All of these parameters are learnable.

[0065] Step 2.2.5, Linear Output Layer: Pass the final output through a linear layer and map it to the dimension of the dictionary size to obtain the probability distribution of the element at each position in the output sequence.

[0066] Furthermore, the specific training process in step 3 includes:

[0067] Step 3.1, randomly select mask elements: randomly sample the number n of mask elements from a uniform distribution between 1 and n mask , and then randomly select n mask elements as masked elements Y mask and replace its value with <mask>, i.e. Class0;

[0068] Step 3.2, input random mask sequence: The Transformer sequence decoder receives the random mask sequence and the context vector, and generates an output sequence of the same length as the random mask sequence in one decoding;

[0069] Step 3.3, Optimization objective: By optimizing the prediction and all Y mask The cross entropy loss between the target elements of the elements is used for training:

[0070]

[0071] where y i Indicates that in Y mask A target element in the set, which is selected as the mask during training, that is, the part that the model needs to predict, X is the input feature, Z is the unmasked target element, P(y i |X,Z) represents the deep learning model for the masked label y i The probability estimate of the true category, L CE is the cross entropy loss function, which calculates the negative logarithm of the predicted probability of the deep learning model for the masked target element as the difference between the model's predicted value and the actual target value. The deep learning model is optimized by the stochastic gradient descent algorithm.

[0072] Furthermore, the specific steps of each reasoning process in step 4 include:

[0073] Step 4.1, input fully masked sequence: At the beginning of inference, a fully masked sequence is input into the Transformer sequence decoder;

[0074] Step 4.2: Parallel prediction of masked elements: The Transformer sequence decoder predicts the category of all masked elements simultaneously by selecting the category with the highest probability from the category set for each element.

[0075] Step 4.3, reconstruct the output signal: use the dictionary to convert the predicted category into the actual voltage signal value;

[0076] Step 4.4, filtering the output signal: Apply the Savitzky-Golay filter to the first decoded and reconstructed signal sequence to smooth the waveform unevenness and correct the errors, and finally obtain a smooth output signal.

[0077] The present invention provides a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method that efficiently captures the long-term sequence dependencies of the output signals of nonlinear transmitter circuits while simultaneously considering the impact of link parameters on transmitter performance, including crosstalk between multiple links. Compared to traditional circuit simulation methods, the present invention's non-autoregressive Transformer-based model achieves high-precision and high-speed transmitter modeling by capturing long-term sequence dependencies and using parallel prediction techniques. This method significantly improves the efficiency of transmitter modeling and analysis, making it particularly suitable for high-speed and high-density link systems. It helps improve data transmission quality, reduce bit error rates, and enhance the efficiency of circuit design and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is a flow chart of the present invention;

[0079] Figure 2 It is a schematic diagram of the overall structure of the deep learning model of the present invention;

[0080] Figure 3 1 is a schematic diagram of the structure of a non-sequential encoder of the present invention;

[0081] Figure 4 Schematic diagram of the Transformer sequence decoder structure of the present invention;

[0082] Figure 5 It is a curve comparison diagram of the prediction results based on the non-autoregressive Transformer of the present invention and the simulation results of traditional simulation software. DETAILED DESCRIPTION

[0083] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0084] like Figure 1-Figure 5 As shown, the specific implementation steps of the present invention's 4-level pulse amplitude modulation high-speed transmitter modeling method based on non-autoregressive Transformer are as follows:

[0085] Step 1: Obtain training data through circuit simulation: Input different PAM4 signals under different link parameters and obtain the transmitter circuit output signal as training data. Link parameters include equalizer coefficient H0, power supply voltage V h (same as the highest level voltage of the PAM4 signal), the transmitting end load capacitance C L , S parameters S of the two transmission lines tl , the equivalent resistance Z0 of the receiving end and the pull-up voltage V p , PAM4 input signal x s It is a sequence of "0" and "1". Each waveform transmits a 2-bit signal. There are 4 level values ​​(representing "00", "01", "10", and "11"), which are input to the link transmitter in a trapezoidal wave mode. h , signal period t p , the ratio of rise / fall time to signal period r rf The key to this process is to obtain the training data of the model. The steps are as follows:

[0086] Step 1.1, acquisition of intrinsic output signal samples when there is no crosstalk:

[0087] Step 1.1.1, input signal x s applied to the first link while keeping the second link input low.

[0088] Step 1.1.2, record the output signal of the first link as the intrinsic output signal sample y intr .

[0089] Step 1.2, obtaining crosstalk signal samples:

[0090] Step 1.2.1, input signal x s Applied to the second link while keeping the first link input high.

[0091] Step 1.2.2: record the interference signal of the second link to the first link and subtract the intrinsic output signal y from it. intr , get the crosstalk signal sample C ij .

[0092] Step 1.3, convert the continuous output voltage signal sequence into a discrete voltage category sequence: Convert the continuous output voltage signal sequence into a discrete voltage category sequence: For the intrinsic output and crosstalk output, respectively, their entire voltage range is divided into steps of Δv I or Δv C Divide into several voltage categories and build a dictionary of intrinsic output and crosstalk output to map voltage values ​​to their nearest category. The specific steps are as follows:

[0093] Step 1.3.1, dictionary construction: For intrinsic output, define its voltage range as According to the step size Δv I Divide the voltage range and generate a series of discrete voltage categories to construct the dictionary D I :

[0094]

[0095] For the crosstalk output, its voltage range is defined as According to the step size Δv C Divide the voltage range and generate a series of discrete voltage categories to construct the dictionary D C :

[0096]

[0097] Voltage value v k Corresponding to the k+1th category Class k+1 , where Class0 is assigned to D I and D C A special one <mask>Elements representing masks.

[0098] Step 1.3.2, voltage value to voltage category mapping: For each output voltage value, map each voltage value to its nearest voltage value v k , and then use the corresponding dictionary (intrinsic output or crosstalk output) to find its nearest voltage category Class k+1 , thereby converting the continuous voltage signal sequence into a discrete voltage category sequence, and using the discrete voltage category sequence as the training target of the model in subsequent training.

[0099] The training data acquisition process in step 1 requires separately obtaining output signal samples with and without crosstalk, converting the continuous output voltage signal sequence into a discrete voltage category sequence, and mapping the voltage value to its nearest category by constructing a dictionary.

[0100] Step 2: Build a deep learning model based on Transformer: Figure 2 As shown in the figure, a deep learning model using an encoder-decoder architecture is established, including a non-sequential encoder and a Transformer sequence decoder. The specific structure and steps are as follows:

[0101] Step 2.1, non-sequential encoder: process disordered non-sequential input, generate context vector, and pass it to the decoder. Figure 3 As shown in Figure 2, the encoding methods of each input feature are as follows:

[0102] In step 2.1.1, a Boolean variable K is used to distinguish between the predicted intrinsic output signal and the crosstalk output signal. Figure 3 As shown in ①a, the binary variable K is passed through a 2×d model The embedding matrix is ​​encoded as a d model Vector of dimension K e , K e represents the K-encoded vector, d model is the dimension of the encoding vector, which is the same as the model dimension of the Transformer sequence decoder.

[0103] Step 2.1.2, for the scalar features H0, V h 、C L , Z0, V p , t p 、r rf Standardization and coding: Figure 3 As shown in ①b, for the scalar characteristics H0, V h 、C L , Z0, V p , t p 、r rf The features are normalized by subtracting their respective means and dividing by their standard deviations to reduce scale differences and accelerate convergence. The normalized features are converted to d model dimensional vector. Each feature is passed through an MLP with different parameters but the same structure. Each MLP has two hidden layers, each containing 16 neurons. The hidden layer uses the ReLU activation function, and the output layer uses the linear activation function. Finally, the scalar features H0 and V are obtained. h 、C L , Z0, V p , t p 、r rf The encoded vectors are expressed as

[0104] Step 2.1.3, input signal sequence encoding: PAM4 signal has 4 levels, with a total of 12 types of directional level transitions (such as from 0 to 1, from 0 to 2, from 0 to 3...). First, traverse the input sequence x of length m s , mark the rising edge at position i+1 (When x i <x i+1 ), or mark the falling edge at position i (When x i >x i+1 When (i=0,…,m-2). Since the signal is at a low level before and after transmission, for a non-zero start bit x0, a rising edge is marked at position 1. For non-zero trailing bits x m-1 , marking a falling edge at position m 0 represents an invalid position. Each type of transition occurs at most m' times, where m'=m / 2 (m is an even number) or m'=(m+1) / 2 (m is an odd number), padded with 0 to ensure uniform length. In this way, x s It is converted into m′ discrete position indexes and then passed through 12 (m+1)×4 m The embedding matrix of dimension is embedded into a set of continuous 4 m dimensional vector. The position index embedding vectors of these different types of transformations are respectively input with a dimension of 4 according to different transformation types. m Each MLP has 2 hidden layers, each containing 16 neurons. The hidden layer uses the ReLU activation function, and the output layer uses the linear activation function. After stacking the MLP layers, we finally get Contains 12×m′ vectors, each of dimension d model .

[0105] Step 2.1.4, S parameter encoding: Figure 3 As shown in ③, by decomposing the S parameter matrix into real and imaginary matrices, and considering the symmetry of the S parameters themselves, the 4×4 S parameter matrix in the two-transmission line system is reconstructed into an S parameter matrix with 10 effective elements and a shape of 2×5. Each element s in the S parameter matrix is ​​translated and logarithmic transformation is applied:

[0106] s scaled =log(s+1.1*min(S tl ))

[0107] Among them S tl Represents the S parameter matrix of the transmission line, min(S tl )) represents the minimum element in the matrix, s is S tl Elements, s scaled The result of the scaling after translation and logarithmic transformation is obtained. Two consecutive convolutional neural network (CNN) layers are used to process the S parameter matrix of each frequency point. The first CNN expands the two input channels (corresponding to the real and imaginary matrices) into 16 output channels, and the second CNN further expands it to 32 output channels. The size of each convolution kernel is 1, and the CNN uses the ReLU activation function. The final feature map is flattened and then passed through a linear layer to generate a dimension of (len f ,d model )of in Indicates S tl The encoded vector, where d model is the dimension of the encoding vector, len f is the number of S parameter frequency points.

[0108] Step 2.1.5, encode features Combined into a set of unordered dimensional vectors X e , length is 8+len f +12*m′, input to the decoder, interacts with the output sequence.

[0109] Step 2.2, Transformer sequence decoder: Figure 4 As shown in Figure 1, the Transformer sequence decoder combines the context vector generated by the encoder and the category sequence output by the transmitter to generate a category probability distribution for each point in the sequence. The specific steps are as follows:

[0110] Step 2.2.1, Input Embedding and Positional Encoding: First, embed the converted category sequence of the output signal, converting each category into a fixed-dimensional vector representation. Add the positional encoding to the embedding vector to preserve the position information in the input sequence. The positional encoding can be generated using the sine and cosine functions:

[0111]

[0112] where d model is the embedding dimension (usually equal to the model dimension), i is the embedding dimension index, pos is the position in the sequence, PE (pos,k) Represents the position encoding value at the k-th dimension index at position pos. The encoding of the even-numbered dimension index at position pos is calculated using the sine function, and the encoding of the odd-numbered dimension index is calculated using the cosine function.

[0113] Step 2.2.2, multi-head attention mechanism module: Generate query vector (Q), key vector (K) and value vector (V) from the input vector X through linear transformation:

[0114] Q=XW Q ,K=XW K ,V=XW V

[0115] Where W Q ,W K ,W V The linear transformation matrices representing the query vector Q, key vector K, and value vector V are used to transform the input vector X into the query vector Q, key vector K, and value vector V respectively.

[0116] Calculate the dot product of the query vector Q and the key vector K and divide it by the scaling factor, then pass it through the softmax function to get the attention weight:

[0117]

[0118] where d k represents the dimension of the key vector K, Attention(Q,K,V) represents the output of the attention function, and the query vector Q is used to compare with the transpose K of each key vector K. T Compute the dot product and divide by the scaling factor After that, the scores are used to measure the importance of the corresponding value vector V; after processing by the softmax function, these scores are converted into weights, and then these weights are multiplied by the value vector V to obtain the weighted output, that is, the result after attention concentration. Softmax(*) represents the softmax function; the softmax function σ(z) i The definition is as follows:

[0119]

[0120] The denominator is all The sum of all output values ​​is ensured to be 1, thus forming a probability distribution.

[0121] The attention mechanism is divided into multiple heads for parallel calculation, and then the results are spliced ​​together:

[0122] MultiHead(Q,K,V)=Concat(head1,…,head h )W O ,

[0123] in

[0124] in represents the linear transformation matrix used in the i-th head, which is used to transform the original query vector Q, key vector K and value vector V into the corresponding space processed by the head; head1,…,head h Represents the output of each attention head, Concat(head1,…,head h )W O Indicates that the output vectors of all heads are concatenated and then transformed by a linear transformation W O The information of these different heads is merged, and MltiHead(Q, K, V) is the output of the multi-head attention mechanism; and layer normalization is performed after residual connection.

[0125] Step 2.2.3, multi-head cross attention: The input vector X and the encoding vector from the encoder, i.e., the context vector C, are linearly transformed to generate the query vector Q, key vector K, and value vector V respectively:

[0126] Q=XW Qc ,K=CW Kc ,V=CW Vc

[0127] Where W Qc ,W Kc ,W Vc are the linear transformation matrices in the cross-attention, which transform the input vector X and the context vector C into the corresponding query vector Q, key vector K and value vector V.

[0128] Calculate the dot product of the query vector Q and the key vector K, divide it by the scaling factor, and use the softmax function to obtain the attention weight:

[0129]

[0130] where d k represents the dimension of the key vector K, Attention(Q,K,V) represents the output of the attention function, the query vector Q from the input vector X is used to compare with the transpose K of each key vector K converted from the context vector C. T Compute the dot product and divide by the scaling factor The scores are used to measure the importance of the corresponding value vector V. After processing through the softmax function, these scores are converted into weights, and then these weights are multiplied by the value vector V to generate a weighted output, which is the result of cross attention concentration.

[0131] Cross attention also uses a multi-head mechanism, and the results are merged after independent calculations:

[0132] MultiHead(Q,K,V)=Concat(head1,…,head h )W Oc ,

[0133] in

[0134] Each of the heads Indicates the use of different linear transformation matrices The query vector Q, key vector K and value vector V are converted to the corresponding space processed by the head. The output of each attention head is concatenated and subjected to a linear transformation W. Oc The final cross-attention output is then merged and the output is then processed through a residual connection and layer normalization.

[0135] Step 2.2.4, bit-wise feedforward neural network (FNN): The vector of each position independently passes through two layers of linear transformation and a ReLU activation function:

[0136] FFN(x)=max(0,xW1+b1)W2+b2

[0137] The FFN is a two-layer network structure that operates independently at each position. It first performs a linear transformation xW1+b1 on the encoding vector x at each position, then introduces nonlinear processing through the ReLU activation function max(0,z), and then performs a second linear transformation max(0,xW1+b1)W2+b2. Here, W1 and W2 are weight matrices, and b1 and b2 are bias terms. All of these parameters are learnable.

[0138] Step 2.2.5, Linear Output Layer: Pass the final output through a linear layer and map it to the dimension of the dictionary size to obtain the probability distribution of the element at each position in the output sequence.

[0139] Step 3: Model training based on random masking: For an input signal of a given length, the length of the output signal sequence can be predetermined (assuming it is n) and does not change with the input. The model is trained using a random masking strategy to enable the model to predict elements at the masked position. The specific training steps are as follows:

[0140] Step 3.1, randomly select mask elements: randomly sample the number n of mask elements from a uniform distribution between 1 and n mask , and then randomly select n mask elements as masked elements Y mask and replace its value with <mask>, that is Class0.

[0141] Step 3.2, input random mask sequence: The decoder receives the random mask sequence and the context vector, and generates an output sequence of the same length as the random mask sequence in one decoding.

[0142] Step 3.3, Optimization objective: By optimizing the prediction and all Y mask The cross entropy loss between the target elements of the elements is used for training:

[0143]

[0144] where y i Indicates that in Y mask A target element in the set, which is selected as the mask during training, that is, the part that the model needs to predict, X is the input feature, Z is the unmasked target element, P(y i |X, Z) represents the model pair for the masked label y i The probability estimate of the true category, L CE is the cross entropy loss function, which calculates the negative logarithm of the model's predicted probability for the masked target element as the difference between the model's predicted value and the actual target value. The deep learning model is optimized by the stochastic gradient descent algorithm.

[0145] In step 3, a random mask strategy is used for training to enable the model to have the ability to predict elements at mask positions.

[0146] Step 4: Model inference based on non-autoregressive decoding and filtering: Input the link parameters and the input signal of the interfered link to the model, input K = 0, and obtain the intrinsic output signal under the set of link parameters and the input signal itself through non-autoregressive decoding and filtering; then, for all other interfering links, input the link parameters and the interfering link input signal respectively, input K = 1, and obtain the crosstalk output signal under the set of link parameters and the interfering input signal through non-autoregressive decoding and filtering. The intrinsic output signal is accumulated with all other crosstalk output signals to obtain the output signal of the transmitter in the multi-communication link affected by other signals. The specific steps of each inference process are as follows:

[0147] Step 4.1, input fully masked sequence: At the beginning of inference, a fully masked sequence is input into the decoder.

[0148] Step 4.2, parallel prediction of masked elements: The decoder predicts the categories of all masked elements simultaneously by selecting the category with the highest probability from the category set for each element.

[0149] Step 4.3, reconstruct the output signal: use the dictionary to convert the predicted category into the actual voltage signal value.

[0150] Step 4.4, filtering the output signal: Apply the Savitzky-Golay filter to the first decoded and reconstructed signal sequence to smooth the waveform unevenness and correct the errors, and finally obtain a smooth output signal.

[0151] In step 4, the non-autoregressive inference method can be used to output the sequence results in one decoding, avoiding multiple loop predictions, and using the characteristics of waveform continuity to eliminate errors by filtering.

[0152] For a 4-level pulse amplitude modulation signal with a pulse period of 60 to 150 ps and a signal transition time ratio of 10% to 20%, tests were conducted under different link parameters. The average relative error of the intrinsic output of 2000 test samples was only 0.95%, and the average error of the crosstalk output was only 1.93%. Figure 5 A comparison chart of the prediction of the 4-level pulse amplitude modulation transmitter signal based on the non-autoregressive Transformer and the simulation results of the traditional simulation software is shown for a 2-link system, given the link parameters and the input signal of each link. In the test of the 2-link system, the average error of the method of the present invention compared with the actual simulation results for 1000 groups of test samples is only 0.96%. Compared with the traditional simulation software which takes 3.30s to complete the simulation of the two-link system and obtain the transmitter output signal, the method of the present invention only takes 7.23ms, achieving an acceleration of up to 456 times. In the test of the 16-link system, the average error of the method of the present invention compared with the actual simulation results is only 1.07%. The traditional simulation software takes 20.2s to complete the simulation, while the method of the present invention only takes 21.4ms, achieving an acceleration of up to 944 times. The method of the present invention shows high accuracy and excellent simulation efficiency in modeling the output signal of the 4-level pulse amplitude modulation transmitter.

[0153] In summary, this invention significantly improves the model's efficiency in predicting the output voltage waveform of a 4-level pulse amplitude modulation transmitter through efficient encoding and decoding. Compared to traditional simulation software, this significantly increases simulation speed while ensuring high accuracy of simulation results. By integrating the encoder and decoder, this invention significantly improves the efficiency and accuracy of modeling and analysis of 4-level pulse amplitude modulation transmitters. This method is particularly suitable for high-speed and high-density link systems, helping to improve data transmission quality and reduce bit error rates.

[0154] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.< / mask> < / mask> < / mask> < / mask>

Claims

1. A 4-level pulse amplitude modulation high-speed transmitter modeling method based on non-autoregressive Transformer, characterized in that: The method comprises the following steps: Step 1: Obtain training data through circuit simulation: Input different PAM4 signals under different link parameters and obtain the transmitter circuit output signal as training data; Step 2: Build a Transformer-based deep learning model: Build a deep learning model using an encoder-decoder architecture, consisting of a non-sequential encoder and a Transformer sequential decoder. Step 3: Model training based on random masking: For an input signal of a given length, the length of the output signal sequence is predetermined and does not change with the input. A random masking strategy is used to train the deep learning model, enabling the model to predict elements at the mask position. Step 4, model inference based on non-autoregressive decoding and filtering: input link parameters and the input signal of the interfered link into the deep model, input K = 0, and obtain the intrinsic output signal under this set of link parameters and the input signal itself through non-autoregressive decoding and filtering; then for all other interfering links, input link parameters and the interfering link input signal respectively, input K = 1, and obtain the crosstalk output signal under this set of link parameters and the interfering input signal through non-autoregressive decoding and filtering; add the intrinsic output signal with all other crosstalk output signals to obtain the output signal of the transmitter in multiple communication links affected by other signals.

2. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 1, characterized in that: The link parameters in step 1 include the equalizer coefficient H0, the power supply voltage V h , Transmitter load capacitance C L , S parameters S of the two transmission lines tl , the equivalent resistance Z0 of the receiving end and the pull-up voltage V p , PAM4 input signal x s It is a sequence composed of "0" and "1". Each waveform transmits a 2-bit signal. There are 4 level values, representing "00", "01", "10", and "11". It is input to the link transmitter in a trapezoidal wave mode and is transmitted through the highest level voltage V h , signal period t p , the ratio of rise / fall time to signal period r rf Description; Step 1 specifically includes: Step 1.1, obtain the intrinsic output signal sample when there is no crosstalk: Step 1.2, obtain crosstalk signal samples: Step 1.3, convert the continuous output voltage signal sequence into a discrete voltage category sequence: for the intrinsic output and crosstalk output, their entire voltage range is divided into discrete voltage categories by a step size of Δv. I or Δv C Divide into several voltage categories and build a dictionary of intrinsic outputs and a dictionary of crosstalk outputs to map voltage values ​​to their nearest category.

3. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 2, characterized in that: The step 1.1 specifically includes: Step 1.1.1, input signal x s applied to the first link while keeping the second link input low; Step 1.1.2, record the output signal of the first link as the intrinsic output signal sample y intr .

4. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 2, characterized in that: The step 1.2 specifically includes: Step 1.2.1, input signal x s Applied to the second link while keeping the first link input high; Step 1.2.2: record the interference signal of the second link to the first link and subtract the intrinsic output signal y from it. intr , get the crosstalk signal sample C ij .

5. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 2, characterized in that: The step 1.3 specifically includes: Step 1.3.1, dictionary construction: For intrinsic output, define its voltage range as According to the step size Δv I Divide the voltage range and generate a series of discrete voltage categories to construct the dictionary D I : For the crosstalk output, its voltage range is defined as According to the step size Δv C Divide the voltage range and generate a series of discrete voltage categories to construct the dictionary D C : Voltage value v k Corresponding to the k+1th category Class k+1 , where Class0 is assigned to D I and D C A special one <mask> Elements, representing masks;< / mask> Step 1.3.2, voltage value to voltage category mapping: For each output voltage value, map each voltage value to its nearest voltage value v k , and then use the corresponding dictionary to find its nearest voltage category Class k+1 , thereby converting the continuous voltage signal sequence into a discrete voltage category sequence, and using the discrete voltage category sequence as the training target of the deep learning model in subsequent training.

6. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 2, characterized in that: The non-sequential encoder in step 2 is used to process disordered non-sequential input, generate a context vector, and pass it into the Transformer sequence decoder. The encoding method of each input feature is as follows: Step 2.1.1, use a Boolean variable K to distinguish between the predicted intrinsic output signal and the crosstalk output signal, and convert the binary variable K through a 2×d model The embedding matrix is ​​encoded as a d model Vector of dimension K e , K e represents the K-encoded vector, d model is the dimension of the encoding vector, which is the same as the model dimension of the Transformer sequence decoder; Step 2.1.2, for the scalar features H0, V h 、C L , Z0, V p , t p 、r rf Standardization and encoding: scalar features H0, V h 、C L , Z0, V p , t p 、r rf They are standardized by subtracting their respective means and dividing by their standard deviations to reduce scale differences and accelerate convergence; the standardized features are converted to d through a multi-layer perceptron layer. model dimensional vector; each feature passes through a multilayer perceptron layer with different parameters and the same structure; each multilayer perceptron layer has two hidden layers, each containing 16 neurons, the hidden layer uses the ReLU activation function, and the output layer uses the linear activation function; finally, the scalar features H0 and V h 、C L , Z0, V p , t p 、r rf The encoded vectors are expressed as Step 2.1.3, input signal sequence encoding: PAM4 signal has 4 levels, with a total of 12 types of directional level transitions; first traverse the input sequence x of length m s , when x i <x i+1 Mark the rising edge at position i+1 Or when x i >x i+1 Mark the falling edge at position i when Where i = 0, ..., m-2; since the signal is at a low level before and after transmission, for a non-zero start bit x0, a rising edge is marked at position 1 For non-zero trailing bits x m-1 , marking a falling edge at position m 0 represents an invalid position; each type of transition occurs at most m′ times, where if m is even, m′ is defined as m / 2, and if m is odd, m′ is defined as (m+1) / 2; pad with 0 to ensure uniform length; In this way, x s is converted into m' discrete position indexes, and then through 12 (m+1)×4 m The embedding matrix of dimension is embedded into a set of continuous 4 m dimensional vector; the position index embedding vectors of these different types of transformations are respectively input with a dimension of 4 according to different transformation types. m Each multilayer perceptron layer has 2 hidden layers, each layer contains 16 neurons, the hidden layer uses the ReLU activation function, and the output layer uses the linear activation function. After stacking the multilayer perceptron layers, we finally get Contains 12×m′ vectors, each of dimension d model ; Step 2.1.4, S-parameter encoding: By decomposing the S-parameter matrix into real and imaginary matrices, taking into account the symmetry of the S-parameters themselves, the 4×4 S-parameter matrix in the two-transmission-line system is reconstructed into an S-parameter matrix with 10 effective elements and a shape of 2×5; each element s in the S-parameter matrix is ​​translated and logarithmic transformation is applied: s scaled =log(s+1.1*min(S tl )) Among them S tl Represents the S parameter matrix of the transmission line, min(S tl ) represents the smallest element in the matrix, s is S tl Elements, s scaled is the scaling result after translation and logarithmic transformation; two consecutive convolutional neural network layers are used to process the S parameter matrix of each frequency point. The first convolutional neural network layer expands the two input matrix channels corresponding to the real and imaginary parts into 16 output channels, and the second convolutional neural network layer further expands it to 32 output channels. The size of each convolution kernel is 1, and the convolutional neural network layer uses the ReLU activation function; the final feature map is flattened and then passed through a linear layer to generate a dimension of (len f ,d model )of Indicates S tl The encoded vector, where d model is the dimension of the encoding vector, len f is the number of S parameter frequency points; Step 2.1.5, encode features Combined into a set of unordered dimensional vectors X e , length is 8+len f +12*m′, input to the Transformer sequence decoder, and interact with the output sequence.

7. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 2, characterized in that: In step 2, the Transformer sequence decoder is used to combine the context vector generated by the non-sequential encoder and the category sequence output by the transmitter to generate a category probability distribution for each point in the sequence. The specific steps are as follows: Step 2.2.1, Input Embedding and Position Encoding: First, embed the category sequence after the output signal is converted, and convert each category into a fixed-dimensional vector representation; add the position encoding to the embedding vector to preserve the position information in the input sequence; the position encoding is generated by sine and cosine functions: where d model is the embedding dimension, i is the embedding dimension index, pos is the position in the sequence, PE (pos,k) Represents the position encoding value at the k-th dimension index at position pos. The encoding of the even dimension index at position pos is calculated using the sine function, and the encoding of the odd dimension index is calculated using the cosine function; Step 2.2.2, Multi-head Self-Attention Mechanism: The input vector X is separated into the query vector Q, key vector K and value vector V through linear transformation: Q=XW Q ,K=XW K ,V=XW V Where W Q ,W K ,W V The linear transformation matrices representing the query vector Q, key vector K, and value vector V are used to transform the input vector X into the query vector Q, key vector K, and value vector V respectively; Calculate the dot product of the query vector Q and the key vector K, divide it by the scaling factor, and use the softmax function to obtain the attention weight: where d k represents the dimension of the key vector K, Attention(Q,K,V) represents the output of the attention function, and the query vector Q is used to compare with the transpose K of each key vector K. T Compute the dot product and divide by the scaling factor After that, the scores are used to measure the importance of the corresponding value vector V; after processing by the softmax function, these scores are converted into weights, and then these weights are multiplied by the value vector V to obtain the weighted output, that is, the result after attention concentration; softmax(*) represents the softmax function; The attention mechanism is split into multiple heads, which are calculated independently and then the results are merged: MultiHead(Q,K,V)=Concat(head1,…,head h )W O , in in represents the linear transformation matrix used in the i-th head, which is used to transform the original query vector Q, key vector K and value vector V into the corresponding space processed by the head; head1,…,head h Represents the output of each attention head, Concat(head1,…,head h )W O Indicates that the output vectors of all heads are concatenated and then transformed by a linear transformation W O Merge the information of these different heads, MltiHead(Q, K, V) is the output of the multi-head attention mechanism; and perform layer normalization after residual connection; Step 2.2.3, multi-head cross attention: The input vector X and the encoding vector from the non-sequential encoder, i.e., the context vector C, are linearly transformed to generate the query vector Q, key vector K, and value vector V respectively: Q=XW Qc ,K=CW Kc ,V=CW Vc Where W Qc ,W Kc ,W Vc are the linear transformation matrices in the cross-attention, which transform the input vector X and the context vector C into the corresponding query vector Q, key vector K and value vector V; Calculate the dot product of the query vector Q and the key vector K, divide it by the scaling factor, and use the softmax function to obtain the attention weight: where d k represents the dimension of the key vector K, Attention(Q,K,V) represents the output of the attention function, the query vector Q from the input vector X is used to compare with the transpose K of each key vector K converted from the context vector C. T Compute the dot product and divide by the scaling factor Afterwards, the scores obtained are used to measure the importance of the corresponding value vector V; after processing through the softmax function, these scores are converted into weights, and then these weights are multiplied with the value vector V to generate a weighted output, which is the result of cross attention concentration; Cross attention also uses a multi-head mechanism, and the results are merged after independent calculations: MultiHead(Q,K,V)=Concat(head1,…,head h )W Oc , in Each of the heads Indicates the use of different linear transformation matrices The query vector Q, key vector K and value vector V are converted to the corresponding space processed by the head; the output of each attention head is concatenated and subjected to a linear transformation W Oc Merge to form the final cross-attention output; finally, the output is connected through the residual and layer normalized; Step 2.2.4, bit-by-bit feedforward network: process the vector of each position independently, through two layers of linear transformation and ReLU activation function: FFN(x)=max(0,xW1+b1)W2+b2 FFN is a two-layer network structure that operates independently at each position. It first performs a linear transformation xW1+b1 on the encoding vector x at each position, then introduces nonlinear processing through the ReLU activation function max(0,z), and then performs a second linear transformation max(0,xW1+b1)W2+b2. Here W1 and W2 are weight matrices, and b1 and b2 are bias terms. All of these parameters are learnable. Step 2.2.5, Linear Output Layer: Pass the final output through a linear layer and map it to the dimension of the dictionary size to obtain the probability distribution of the element at each position in the output sequence.

8. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 1, characterized in that: The specific training process in step 3 includes: Step 3.1, randomly select mask elements: randomly sample the number n of mask elements from a uniform distribution between 1 and n mask , and then randomly select n mask elements as masked elements Y mask and replace its value with <mask> , i.e. Class0;< / mask> Step 3.2, input random mask sequence: The Transformer sequence decoder receives the random mask sequence and the context vector, and generates an output sequence of the same length as the random mask sequence in one decoding; Step 3.3, Optimization objective: By optimizing the prediction and all Y mask The cross entropy loss between the target elements of the elements is used for training: where y i Indicates that in Y mask A target element in the set, which is selected as the mask during training, that is, the part that the model needs to predict, X is the input feature, Z is the unmasked target element, P(y i |X,Z) represents the deep learning model for the masked label y i The probability estimate of the true category, L CE is the cross entropy loss function, which calculates the negative logarithm of the predicted probability of the deep learning model for the masked target element as the difference between the model's predicted value and the actual target value. The deep learning model is optimized by the stochastic gradient descent algorithm.

9. The method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer according to claim 1, characterized in that: The specific steps of each reasoning process in step 4 include: Step 4.1, input fully masked sequence: At the beginning of inference, a fully masked sequence is input into the Transformer sequence decoder; Step 4.2: Parallel prediction of masked elements: The Transformer sequence decoder predicts the category of all masked elements simultaneously by selecting the category with the highest probability from the category set for each element. Step 4.3, reconstruct the output signal: use the dictionary to convert the predicted category into the actual voltage signal value; Step 4.4, filtering the output signal: Apply the Savitzky-Golay filter to the first decoded and reconstructed signal sequence to smooth the waveform unevenness and correct the errors, and finally obtain a smooth output signal.

Images