Non-autoregressive transformer-based modeling method for 4-level pulse amplitude modulation high-speed transmitter

Abstract

Disclosed in the present invention is a non-autoregressive Transformer-based modeling method for a 4-level pulse amplitude modulation high-speed transmitter. The method involves establishing a deep learning model having an encoder-decoder architecture to predict the behavior of a 4-level pulse amplitude modulation transmitter. An encoder processes unordered non-sequential inputs, including an input signal parameter and link parameters, to generate a context vector and then transmit same to a decoder. The decoder uses both the context vector generated by the encoder and a transmitter output signal sequence to generate a categorical probability distribution for each point in the sequence one by one. The model is trained using a random masking strategy, and inference is performed by means of non-autoregressive decoding and filtering, so that the model can perform parallel prediction on an output sequence, and perform a filtering process to predict an output signal. Compared to traditional simulation methods, the present invention achieves a significant acceleration effect, particularly when processing multi-link systems.

Description

A Modeling Method for a 4-Level Pulse Amplitude Modulation High-Speed ​​Transmitter Based on Non-Autoregressive Transformer Technical Field

[0001] This invention relates to the field of high-speed communication link analysis, and in particular to a modeling method for a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer. Background Technology

[0002] With the development of electronic and communication technologies, high-speed serial links play a crucial role in achieving low-latency and high-bandwidth communication, and are widely used in emerging data-driven applications such as artificial intelligence (AI), 5G mobile networks, and automotive technology. To meet the demands of these applications for energy efficiency, cost-effectiveness, and high-performance systems, chip-based high-density heterogeneous integration (HDHI) technology has gradually become mainstream and is widely used in various applications.

[0003] High-speed serial links are a critical component of these advanced high-speed systems. To meet the ever-increasing demands for high bandwidth and efficient communication, these links feature highly dense signal paths, ranging from hundreds to thousands, and operate at high frequencies and high data rates, facing complex signal integrity (SI) challenges such as crosstalk, signal attenuation, and electromagnetic interference (EMI). In high-speed links, the transmitter (TX) is one of the most crucial and resource-intensive components, its performance directly impacting the quality of the initial transmitted signal and the integrity of the entire link. Degradation of the transmitter's output signal can cause significant distortion at the final receiver, leading to errors in signal recovery. To maintain high-quality and high-speed output signals and ensure signal integrity along the transmission path, the transmitter needs to operate at high frequencies and minimize timing errors caused by variations in process technology, voltage, temperature (PVT), and load conditions.

[0004] The main shortcomings of existing technologies are: 1. Accurate transistor-level models, such as the SPICE model, have long computation times due to the complexity of internal circuit details; empirical behavioral models, such as the current source model (CSM) and I / O buffer information specification (IBIS) models, although fast, usually have limited accuracy or capability in modeling complex interdependencies. 2. Compared with non-return-to-zero (NRZ) coded signals, four-level pulse amplitude modulation (PAM4) signals increase the complexity of signal processing. Their smaller level intervals are more sensitive to noise and interference, reducing noise margin and requiring higher signal-to-noise ratio processing capabilities. In high-frequency transmission, they are more susceptible to nonlinear effects, such as distortion and inter-symbol interference (ISI).

[0005] To address the aforementioned issues and achieve efficient modeling of high-speed communication link transmitters with 4-level pulse amplitude modulation, a modeling method for 4-level pulse amplitude modulation high-speed transmitters based on non-autoregressive Transformers is proposed. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a modeling method for a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer. This method can consider the influence of link parameters and crosstalk between multiple links on the transmitter circuit output signal, and efficiently model the transmitter circuit output for 4-level pulse amplitude modulation signals.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A modeling method for a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer, comprising the following steps:

[0009] Step 1: Obtain training data through circuit simulation: Input different PAM4 signals under different link parameters and obtain the output signal of the transmitter circuit as training data;

[0010] Step 2, build a Transformer-based deep learning model: Build a deep learning model with an encoder-decoder architecture, including a non-sequential encoder and a Transformer sequential decoder;

[0011] Step 3, Model training based on random mask: For a given input signal of a certain length, the length of the output signal sequence can be predetermined and does not change with the input. The deep learning model is trained using a random mask strategy so that the model has the ability to predict the mask position elements.

[0012] Step 4, Model Inference Based on Non-Autoregressive Decoding and Filtering: Input link parameters and the input signal of the interfered link to the deep model, with input K=0. After non-autoregressive decoding and filtering, the intrinsic output signal under the circuit parameters and its own input signal is obtained. Then, for all other interfering links, input link parameters and the input signal of the interfering link respectively, with input K=1. After non-autoregressive decoding and filtering, the crosstalk output signal under the circuit parameters and the interfering input signal is obtained. The intrinsic output signal is added to all other crosstalk output signals to obtain the output signal of the transmitter in the multi-communication link that is affected by crosstalk from other signals.

[0013] Furthermore, the link parameters in step 1 include the equalizer coefficient H0 and the power supply voltage V. h Transmitter load capacitor C L S-parameters of the two transmission lines tlThe equivalent resistance Z0 of the receiver and the pull-up voltage V of the receiver p PAM4 input signal x s It is a sequence of "0"s and "1"s, with each waveform transmitting 2 bits of signal. There are 4 level values, representing "00", "01", "10", and "11" respectively. It is input to the link transmitter in trapezoidal waveform and transmitted through the highest level voltage V. h Signal period t p The percentage of rise / fall time relative to the signal period (r) rf Description; Step 1 specifically includes:

[0014] Step 1.1, obtain the intrinsic output signal sample without crosstalk:

[0015] Step 1.2, Obtain crosstalk signal samples:

[0016] Step 1.3: Convert the continuous output voltage signal sequence into a discrete voltage category sequence: For intrinsic output and crosstalk output, convert their entire voltage range into a step size Δv. I or Δv C The voltage values ​​are divided into several categories, and a dictionary of intrinsic outputs and a dictionary of crosstalk outputs are constructed to map the voltage values ​​to their nearest category.

[0017] Further, step 1.1 specifically includes:

[0018] Step 1.1.1, input signal x s Apply to the first link while keeping the input of the second link low;

[0019] Step 1.1.2: Record the output signal of the first link as the intrinsic output signal sample y. intr .

[0020] Further, step 1.2 specifically includes:

[0021] Step 1.2.1, input signal x s It is applied to the second link while keeping the input of the first link high.

[0022] Step 1.2.2: Record the interference signal from the second link to the first link, and subtract the intrinsic output signal y from it. intr The crosstalk signal sample C is obtained. ij .

[0023] Furthermore, step 1.3 specifically includes:

[0024] Step 1.3.1, Dictionary Construction: For intrinsic outputs, define their voltage range as follows: Based on step size ΔvI By dividing the voltage range and generating a series of discrete voltage categories, a dictionary D can be constructed. I :

[0025] For crosstalk output, its voltage range is defined as follows: Based on step size Δv C By dividing the voltage range and generating a series of discrete voltage categories, a dictionary D can be constructed. C :

[0026] Voltage value v k Corresponding to the (k+1)th class k+1 Class0 is assigned to D I and D C A special one <mask>Element, representing a mask;

[0027] Step 1.3.2, Voltage value to voltage class mapping: For each output voltage value, map each voltage value to its nearest voltage value v k , using the corresponding dictionary to find its nearest voltage class Class k+1 , so as to convert the continuous voltage signal sequence into a discrete voltage class sequence, which is used as the training target of the deep learning model in subsequent training.

[0028] Further, the non-sequence encoder in step 2 is used to process unordered non-sequence input to generate a context vector, which is input into the Transformer sequence decoder. The specific encoding method of each input feature is as follows:

[0029] Step 2.1.1, using a Boolean variable K to distinguish between the case of predicting intrinsic output signal and crosstalk output signal, encoding the binary variable K through a 2xd model embedding matrix into a d model dimensional vector K e , K e represents the encoded vector of K, d model is the dimension of the encoded vector, which is the same as the model dimension of the Transformer sequence decoder;

[0030] Step 2.1.2, standardization and encoding of scalar features H0, V h , C L , Z0, V p , t p , r rf : Standardize the scalar features H0, V h , C L , Z0, V p , t p , r rf by subtracting the mean and dividing by the standard deviation respectively to reduce the scale difference and speed up the convergence; the normalized features are converted into d model dimensional vectors through multi-layer perceptron layers; each feature is processed through a multi-layer perceptron layer with different parameters but the same structure; each multi-layer perceptron layer has two hidden layers, each containing 16 neurons, and the hidden layers use ReLU activation function, and the output layer uses linear activation function; finally, the encoded vectors of the scalar features H0, V h , C L , Z0, V p , t p , r rf are obtained, respectively represented as

[0031] Step 2.1.3, input signal sequence encoding: PAM4 signal has 4 levels, a total of 12 types of directional level transitions; first traverse the input sequence x of length m s , when x i ,<x i+1 mark the rising edge at position i+1 , or when x i , >x i+1 mark the falling edge at position i , where i=0,…,m-2; since the signal is at low level before and after transmission, for the non-zero starting bit x0, mark a rising edge at position 1 , for the non-zero ending bit x m-1 , mark a falling edge at position m , 0 represents invalid position; each type of transition can occur at most m' times, where m' is defined as m / 2 if m is even, and (m+1) / 2 if m is odd; fill with 0 to ensure uniform length; in this way, x s is converted into m' discrete position indexes, and then embedded into a set of continuous 4 m dimensional vectors through a 12(m+1) x 4 m dimensional embedding matrix; the position index embedding vectors of these different types of transitions are respectively passed through input dimension 4 m multilayer perceptron layers; each multilayer perceptron layer has 2 hidden layers, each containing 16 neurons, the hidden layer uses ReLU activation function, and the output layer uses linear activation function; after stacking the multilayer perceptron layers, the final output is obtained , containing 12xm' vectors, each vector has dimension d model ;

[0032] Step 2.1.4, S parameter encoding: by decomposing the S parameter matrix into real and imaginary matrices, considering the symmetry of the S parameter itself, the 4x4 S parameter matrix in the two transmission line system is reconstructed into an S parameter matrix with 10 effective elements, and the shape is reconstructed into 2x5; each element s in the S parameter matrix is translated and a logarithmic transformation is applied: s scaled =log(s+1.1*min(S tl ))

[0033] where S tl represents the S parameter matrix of the transmission line, min(S tl ) represents the minimum element in the matrix, s is an element of S tl , s scaled is the scaling result after translation and logarithmic transformation; the S parameter matrix of each frequency point is processed using two consecutive convolutional neural network layers, the first convolutional neural network layer expands the input two corresponding real and imaginary matrix channels into 16 output channels, and the second convolutional neural network layer further expands it into 32 output channels, each convolution kernel size is 1, and the convolutional neural network layer uses a ReLU activation function; the final feature map is flattened and passed through a linear layer to generate a dimension of (len f ,d model ) of represents the encoded vector of S tl , where d model is the dimension of the encoded vector, and len f is the number of S parameter frequency points;

[0034] Step 2.1.5, combine the encoded features into a set of unordered dimension vectors X e with a length of 8+len f +12*m' and input into the Transformer sequence decoder to interact with the output sequence.

[0035] Further, the Transformer sequence decoder in step 2 is used to combine the context vector generated by the non-sequence encoder and the category sequence output by the transmitter to generate a category probability distribution for each point in the sequence, and the specific steps are as follows:

[0036] Step 2.2.1, input embedding and position encoding: first, the output signal is converted into a category sequence, and each category is converted into a fixed dimension vector representation; position encoding is added to the embedding vector to preserve the position information in the input sequence; position encoding can be generated by sine and cosine functions:

[0037] where d model is the embedding dimension, i is the embedding dimension index, pos is the position in the sequence, PE (pos,k) represents the position encoding value at the kth dimension index at position pos, the encoding of the even dimension index at position pos is calculated using the sine function, and the encoding of the odd dimension index is calculated using the cosine function;

[0038] Step 2.2.2, multi-head self-attention mechanism: the input vector X is separated into query vector Q, key vector K and value vector V through linear transformation: Q=XW Q , K=XW K , V=XW V

[0039] where W Q ,W K ,W V represent linear transformation matrices of query vector Q, key vector K and value vector V, respectively, for transforming input vector X into query vector Q, key vector K and value vector V, respectively;

[0040] The dot product of query vector Q and key vector K is calculated, and the attention weight is obtained by using softmax function after dividing by the scaling factor:

[0041] where d k represents the dimension of key vector K, Attention(Q,K,V) represents the output of the attention function, and query vector Q is used to calculate the dot product with the transpose of each key vector K T The dot product is calculated and divided by the scaling factor , and the score obtained is used to measure the importance of the corresponding value vector V; after processing by the softmax function, the scores are converted into weights, and then the weights are multiplied by the value vector V to obtain the weighted output, i.e. the result after attention concentration, softmax(*) represents the softmax function;

[0042] The attention mechanism is split into multiple heads, and the results are combined after independent calculation: MultiHead(Q,K,V)=Concat(head 1,…,head h )W O , where

[0043] where indicates the linear transformation matrix used in the i-th head, which is used to convert the original query vector Q, key vector K and value vector V into the corresponding space processed by the head; head 1,…,head h indicates the output of each attention head, Concat(head 1,…,head h )W O indicates that the output vectors of all heads are spliced, and then a linear transformation W O is used to combine the information of different heads, and MltiHead(Q,K,V) is the output of the multi-head attention mechanism; and after residual connection, layer normalization is performed;

[0044] Step 2.2.3, multi-head cross attention: input vector X and the encoding vector from the non-sequence encoder, i.e. context vector C, generate query vector Q, key vector K and value vector V through linear transformation, respectively:

[0045] Q=XW Qc ,K=CW Kc ,V=CW Vc

[0046] where W Qc ,W Kc ,W Vc are linear transformation matrices in cross-attention, which transform input vector X and context vector C into corresponding query vector Q, key vector K and value vector V;

[0047] The dot product of query vector Q and key vector K is calculated, and after dividing by a scaling factor, the attention weight is obtained using the softmax function:

[0048] where d k represents the dimension of key vector K, and Attention(Q,K,V) represents the output of the attention function. Query vector Q from input vector X is used to calculate the dot product with the transpose of each key vector K T transformed from context vector C, and then divide by a scaling factor to obtain the score, which measures the importance of the corresponding value vector V; after processing by the softmax function, the scores are converted into weights, and then the weights are multiplied by the value vector V to generate the weighted output, which is the result after cross-attention;

[0049] Cross-attention also adopts a multi-head mechanism, and the results are independently calculated and combined: MultiHead(Q,K,V) = Concat(head 1,…,head h )W Oc , where

[0050] where each head indicates that different linear transformation matrices are used to transform query vector Q, key vector K and value vector V into the corresponding space for processing in this head; the output of each attention head is combined by concatenation and a linear transformation W Oc to form the final cross-attention output; finally, the output is processed by residual connection and layer normalization;

[0051] Step 2.2.4, bit-by-bit feed-forward network: independently process the vector at each position by two layers of linear transformation and ReLU activation function: FFN(x) = max(0,xW1+b1)W2+b2

[0052] where FFN is a two-layer network structure running independently for each position, which first performs a linear transformation on the encoding vector x for each position, xW1+ b1, then introduces a nonlinear processing through the ReLU activation function max(0, z), and then performs a second linear transformation max(0, xW1+ b1)W2+ b2. Here W1, W2 are weight matrices, and b1, b2 are bias terms, all of which are learnable;

[0053] Step 2.2.5, linear output layer: pass the final output through a linear layer to map to the dimension of the dictionary size to obtain the probability distribution of each position element in the output sequence.

[0054] Further, the specific training process in step 3 includes:

[0055] Step 3.1, randomly select mask elements: randomly sample the number of mask elements n from a uniform distribution from 1 to n mask , and randomly select n mask elements as masked elements Y mask , and replace their values with <mask>i.e. Class 0;

[0056] Step 3.2, input random mask sequence: the Transformer sequence decoder receives a sequence of random masks and a context vector, and generates an output sequence of the same length as the random mask sequence in one decoding;

[0057] Step 3.3, optimization objective: training by optimizing the cross-entropy loss between the prediction and all Y mask elements of the target element:

[0058] where y i represents a target element in the set Y mask , which is selected as a mask during training, i.e. the part that the model needs to predict, X is the input feature, Z is the unmasked target element, P(y i | X, Z) represents the probability estimate of the deep learning model for the true class of the masked label y i , L CE is the cross-entropy loss function, which calculates the negative logarithm of the predicted probability of the deep learning model for the masked target element as the difference between the model prediction value and the actual target value, and the deep learning model is optimized by the stochastic gradient descent algorithm.

[0059] Further, the specific steps of each inference process in step 4 include:

[0060] Step 4.1, input a fully masked sequence: at the beginning of inference, a fully masked sequence is input into the Transformer sequence decoder;

[0061] Step 4.2, predict elements at mask in parallel: the Transformer sequence decoder simultaneously predicts the class of all masked elements, specifically by selecting the class with the highest probability from the class set for each element;

[0062] Step 4.3, reconstruct the output signal: convert the predicted class to the actual voltage signal value using the dictionary;

[0063] Step 4.4, filter the output signal: apply the Savitzky-Golay filter to the first decoded and reconstructed signal sequence to smooth the unevenness of the waveform and correct errors, and finally obtain a smooth output signal.

[0064] The beneficial effects of the present application are: the present application provides a 4-level pulse amplitude modulation high-speed transmitter modeling method based on non-autoregressive Transformer, which can efficiently capture the long-term sequence dependence of the nonlinear transmitter circuit output signal, and consider the influence of link parameters on the performance of the transmitter, including crosstalk between multiple links. Compared with the traditional circuit simulation method, the non-autoregressive Transformer model based on the present application realizes high-precision and high-speed transmitter modeling through capturing long-term sequence dependence and parallel prediction technology, which significantly improves the efficiency of transmitter modeling and analysis, and is particularly suitable for high-speed and high-density link systems, which helps to improve data transmission quality, reduce bit error rate, and improve the efficiency of circuit design and optimization. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 is a flowchart of the present application;

[0066] Figure 2 is a schematic diagram of the overall structure of the deep learning model of the present application;

[0067] Figure 3 is a schematic diagram of the non-sequence encoder structure of the present application;

[0068] Figure 4 is a schematic diagram of the Transformer sequence decoder structure of the present application;

[0069] Figure 5 is a curve comparison diagram of the prediction results based on the non-autoregressive Transformer of the present application and the simulation results of the traditional simulation software. DETAILED DESCRIPTION

[0070] The specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0071] As shown in Figures 1-5, the specific implementation steps of the 4-level pulse amplitude modulation high-speed transmitter modeling method based on non-autoregressive Transformer of the present application are as follows:

[0072] Step 1: Obtain training data by circuit simulation: input different PAM4 signals under different link parameters to obtain transmitter circuit output signals as training data. Link parameters include equalizer coefficients H0, power supply voltage V h (the same as the highest level voltage of PAM4 signal), sending end load capacitance C L , S parameters S tl of two transmission lines, receiving end equivalent resistance Z0, and receiving end pull-up voltage V p , PAM4 input signal x s is a sequence composed of "0" and "1", each waveform transmits 2bit signal, and there are 4 level values (representing "00", "01", "10", "11" respectively), which are input to the link sending end in the form of trapezoidal wave, and the highest level voltage V h , signal period t p , rise / fall time relative signal period ratio r rf The process is critical to obtain the training data of the model, as follows:

[0073] Step 1.1, obtaining the intrinsic output signal sample without crosstalk:

[0074] Step 1.1.1, apply the input signal x s to the first link while keeping the second link input low.

[0075] Step 1.1.2, record the output signal of the first link as the intrinsic output signal sample y intr .

[0076] Step 1.2, obtaining the crosstalk signal sample:

[0077] Step 1.2.1, apply the input signal x s to the second link while keeping the first link input high.

[0078] Step 1.2.2, record the interference signal of the second link to the first link, and subtract the intrinsic output signal y intr from it to get the crosstalk signal sample C ij .

[0079] Step 1.3, convert the continuous output voltage signal sequence into a discrete voltage category sequence: For intrinsic output and crosstalk output, divide their entire voltage range by step size Δv I or Δv C into several voltage categories, and construct a dictionary for intrinsic output and a dictionary for crosstalk output to map voltage values to their nearest categories. The specific steps are as follows:

[0080] Step 1.3.1, dictionary construction: for intrinsic output, define its voltage range as According to the step size Δv I , divide the voltage range to generate a series of discrete voltage categories, i.e. construct the dictionary D I :

[0081] For crosstalk output, define its voltage range as According to the step size Δv C , divide the voltage range to generate a series of discrete voltage categories, i.e. construct the dictionary D C :

[0082] voltage value v k corresponding to the k+1th class Class k+1 where Class0 is assigned to D I and one of D C a special <mask>Element, representing the mask.

[0083] Step 1.3.2, voltage value to voltage class mapping: for each output voltage value, map each voltage value to its nearest voltage value v k , using the corresponding dictionary (intrinsic output or crosstalk output) to find its nearest voltage class Class k+1 , thereby converting the continuous voltage signal sequence into a discrete voltage class sequence, which is used as the training target of the model in subsequent training.

[0084] The training data acquisition process in step 1 needs to separately acquire output signal samples without crosstalk and with crosstalk, and convert the continuous output voltage signal sequence into a discrete voltage class sequence, mapping the voltage value to its nearest class by constructing a dictionary.

[0085] Step 2, build a deep learning model based on Transformer: as shown in Figure 2, build a deep learning model with an encoder-decoder architecture, including a non-sequence encoder and a Transformer sequence decoder, the specific structure and steps are as follows:

[0086] Step 2.1, non-sequence encoder: process unordered non-sequence input, generate context vector, pass into decoder. As shown in Figure 3, the specific encoding method of each input feature is as follows:

[0087] Step 2.1.1, use a Boolean variable K to distinguish between predicting intrinsic output signal and crosstalk output signal. As shown in Figure 3, ①a, encode the binary variable K through a 2xd model embedding matrix into a d model dimensional vector K e , K e represents the encoded vector of K, d model is the dimension of the encoded vector, which is the same as the model dimension of the Transformer sequence decoder.

[0088] Step 2.1.2, standardization and encoding of scalar features H0, V h , C L , Z0, V p , t p , r rf : as shown in Figure 3, ①b, standardize and encode scalar features H0, V h , C L , Z0, V p , t p , r rf The scale difference is reduced and the convergence is accelerated by normalizing each feature by subtracting the respective mean and dividing by the standard deviation. The normalized features are converted into d model dimensional vectors by a multilayer perceptron (MLP) layer. Each feature is converted by a parameterized MLP with the same structure. Each MLP has two hidden layers with 16 neurons each, the hidden layers use a ReLU activation function, and the output layer uses a linear activation function. The final output is a scalar feature H0, V h , C L , Z0, V p , t p , r rf , which is represented as

[0089] Step 2.1.3, Input signal sequence encoding: The PAM4 signal has 4 levels, and there are 12 types of directional level transitions (e.g., from 0 to 1, from 0 to 2, from 0 to 3,...). First, traverse the input sequence x s , and mark an upward edge at position i+1 (when x i < x i+1 ), or mark a downward edge at position i (when x i > x i+1 ), where i = 0,..., m-2. Since the signal is at the low level before and after transmission, for the non-zero starting bit x0, mark an upward edge at position 1 , and for the non-zero ending bit x m-1 , mark a downward edge at position m 0 represents an invalid position. Each type of transition can occur at most m' times, where m' = m / 2 (m is even) or m' = (m+1) / 2 (m is odd), and 0 is padded to ensure uniform length. In this way, x s is converted into m' discrete position indices, which are then embedded into a set of continuous 4 m dimensional vectors by 12 (m+1) x 4 m dimensional embedding matrices. The position index embedding vectors of different types of transitions are respectively passed through input dimension 4 m MLPs for different types of transitions. Each MLP has 2 hidden layers with 16 neurons each, the hidden layers use a ReLU activation function, and the output layer uses a linear activation function. After stacking the MLP layers, the final output is , which contains 12 x m' vectors, each with dimension d model .

[0090] Step 2.1.4, S-parameter encoding: As shown in Figure 3 ③, by decomposing the S-parameter matrix into real and imaginary matrices, and considering the symmetry of the S-parameters themselves, the 4×4 S-parameter matrix in the two transmission line system is reconstructed into a 2×5 S-parameter matrix with 10 effective elements. Each element s in the S-parameter matrix is ​​translated and a logarithmic transformation is applied: s scaled =log(s+1.1*min(S) tl ))

[0091] Where S tl Denotes the S-parameter matrix of the transmission line, min(S tl Let s represent the smallest element in the matrix S. tl The element, s scaled The result is the scaled result after translation and logarithmic transformation. Two consecutive convolutional neural network (CNN) layers process the S-parameter matrix at each frequency point. The first CNN expands the two input channels (corresponding to the real and imaginary parts of the matrix) into 16 output channels, and the second CNN further expands them to 32 output channels. Each convolutional kernel has a size of 1, and the CNN uses the ReLU activation function. The final flattened feature map is then passed through a linear layer to generate a dimension of (len). f ,d model ))of in S represents tl The encoded vector, where d model len represents the dimension of the encoded vector. f This represents the number of S-parameter frequency points.

[0092] Step 2.1.5, encode the features Combined into a set of unordered dimensional vectors X e The length is 8+len f +12*m′, is input into the decoder and interacts with the output sequence.

[0093] Step 2.2, Transformer Sequence Decoder: As shown in Figure 4, the Transformer sequence decoder combines the context vector generated by the encoder and the class sequence output by the transmitter to generate a class probability distribution for each point in the sequence. The specific steps are as follows:

[0094] Step 2.2.1, Input Embedding and Position Encoding: First, the category sequence after the output signal is transformed is embedded, converting each category into a fixed-dimensional vector representation. Position encoding is added to the embedding vector to preserve positional information in the input sequence. Position encoding can be generated using sine and cosine functions:

[0095] Where d model is the embedding dimension (usually equal to the model dimension), i is the embedding dimension index, and pos is the position in the sequence, PE (pos,k) represents the position encoding value at the kth dimension index at position pos, the encoding of even dimension indices at position pos is calculated using a sine function, and the encoding of odd dimension indices is calculated using a cosine function.

[0096] Step 2.2.2, multi-head attention mechanism module: generate query vector (Q), key vector (K) and value vector (V) from input vector X through linear transformation respectively: Q=XW Q ,K=XW K ,V=XW V

[0097] where W Q ,W K ,W V represent the linear transformation matrix of query vector Q, key vector K and value vector V respectively, which is used to convert input vector X into query vector Q, key vector K and value vector V respectively.

[0098] Calculate the dot product of query vector Q and key vector K and divide by the scaling factor, then get the attention weight through the softmax function:

[0099] where d k represents the dimension of key vector K, Attention(Q,K,V) represents the output of the attention function, and query vector Q is used to calculate the dot product with the transpose of each key vector K T and divide by the scaling factor , the resulting score is used to measure the importance of the corresponding value vector V; after processing by the softmax function, these scores are converted into weights, and then these weights are multiplied by the value vector V to get the weighted output, i.e. the result after attention concentration, softmax(*) represents the softmax function; the softmax function σ(z) i is defined as follows:

[0100] where the denominator is the sum of all , which ensures that the sum of all output values is 1, thus forming a probability distribution.

[0101] Split the attention mechanism into multiple heads and calculate in parallel, then concatenate the results: MultiHead(Q,K,V)=Concat(head 1,…,head h )W O , where

[0102] where denotes the linear transformation matrix used in the i-th head to transform the original query vector Q, key vector K and value vector V into the corresponding space processed by the head, respectively; head 1, …, head h denotes the output of each attention head, Concat(head 1, …, head h )W O denotes the output of each attention head, Concat(head 1, …, head O )W Qc combines the information of different heads, and MltiHead(Q, K, V) is the output of the multi-head attention mechanism; and after the residual connection, the layer normalization is performed.

[0103] Step 2.2.3, multi-head cross-attention: the input vector X and the encoding vector from the encoder, i.e., the context vector C, are transformed into the query vector Q, the key vector K and the value vector V through linear transformation, respectively: Q = XW Kc ,K = CW Vc ,V = CW Qc Kc Vc are linear transformation matrices in cross-attention, which transform the input vector X and the context vector C into the corresponding query vector Q, key vector K and value vector V.

[0104] The dot product of the query vector Q and the key vector K is calculated, and after being divided by the scaling factor, the attention weight is obtained using the softmax function:

[0105]

[0106] where d k represents the dimension of the key vector K, and Attention(Q, K, V) represents the output of the attention function. The query vector Q from the input vector X is used to calculate the dot product with the transpose K T of each key vector K transformed from the context vector C, and the obtained score is used to measure the importance of the corresponding value vector V. After being processed by the softmax function, the scores are converted into weights, and then the weights are multiplied with the value vector V to generate the weighted output, i.e., the result after cross-attention.

[0107] Cross-attention also adopts a multi-head mechanism, and the results are combined after independent calculation: MultiHead(Q, K, V) = Concat(head 1, …, head h )W Oc , where

[0108] where each head​​​ denotes using different linear transformation matrices The query vector Q, key vector K and value vector V are converted to the corresponding space of this head processing. The output of each attention head is concatenated (Concat) and passed through a linear transformation W Oc and merged to form the final cross-attention output. Finally, the output is passed through a residual connection and layer normalization.

[0109] Step 2.2.4, Feed-forward neural network (FNN) per position: The vector of each position is independently passed through two linear transformations and a ReLU activation function: FFN(x) = max(0, xW1 + b1)W2 + b2

[0110] where FFN is a two-layer network structure that runs independently for each position, which first performs a linear transformation xW1 + b1 on the encoding vector x of each position, then introduces non-linear processing through the ReLU activation function max(0, z), and then performs a second linear transformation max(0, xW1 + b1)W2 + b2. Here, W1, W2 are weight matrices, and b1, b2 are bias terms, all of which are learnable.

[0111] Step 2.2.5, Linear output layer: The final output is passed through a linear layer to map to the dimension of the dictionary size to obtain the probability distribution of each position element in the output sequence.

[0112] Step 3, Model training based on random mask: For an input signal of a given length, the length of the output signal sequence can be predetermined (assuming n) and does not change with the input. The model is trained using a random mask strategy, enabling the model to predict the mask position elements. The specific training steps are as follows:

[0113] Step 3.1, Randomly select mask elements: Randomly sample the number of mask elements n from a uniform distribution from 1 to n mask , and randomly select n mask elements as the masked elements Y mask , and replace their values with <mask>i.e. Class 0.

[0114] Step 3.2, input random mask sequence: the decoder receives a sequence of random masks and context vectors, and generates an output sequence of the same length as the random mask sequence in one decoding.

[0115] Step 3.3, optimization target: training by optimizing the cross-entropy loss between the prediction and all target elements of the target element: mask

[0116] where y i represents a target element in the set Y mask , which is selected as a mask during training, i.e. the part that the model needs to predict, X is the input feature, Z is the unmasked target element, P(y i |X, Z) represents the model's probability estimate for the true class of the masked label y i , L CE is the cross-entropy loss function, which calculates the negative logarithm of the model's prediction probability as the difference between the model's prediction and the actual target value, and the deep learning model is optimized by the stochastic gradient descent algorithm.

[0117] In step 3, the random mask strategy is used to train the model to have the ability to predict the elements at the mask position.

[0118] Step 4, model inference based on non-autoregressive decoding and filtering: input the link parameters, interference link input signals, input K=0, and get the intrinsic output signal under the circuit parameters and input signal through non-autoregressive decoding and filtering; then for all other interference links, input the link parameters, interference link input signals, input K=1, and get the crosstalk output signal under the circuit parameters and interference input signal through non-autoregressive decoding and filtering. Add the intrinsic output signal and all other crosstalk output signals to get the output signal of the transmitter in the multi-communication link under the interference of other signals. The specific steps of each inference process are as follows:

[0119] Step 4.1, input full mask sequence: at the beginning of inference, a completely masked sequence is input into the decoder.

[0120] Step 4.2, predict elements at mask simultaneously: the decoder simultaneously predicts the class of all mask elements, specifically by selecting the class with the highest probability from the class set for each element.

[0121] Step 4.3, reconstruct the output signal: use the dictionary to convert the predicted class to the actual voltage signal value.

[0122] ​Step 4.4, filter output signal: apply the Savitzky-Golay filter to the first decoded and reconstructed signal sequence to smooth the unevenness of the waveform and correct errors, and finally obtain a smooth output signal.

[0123] In step 4, the non-autoregressive inference method can output the sequence result in one decoding, avoid multiple loop prediction, and use the characteristics of waveform continuity to eliminate errors.

[0124] For a 4-level pulse amplitude modulation signal with a pulse period of 60-150 ps and a signal conversion time ratio of 10%-20%, 2000 test samples were tested under different circuit parameters, and the average relative error of the intrinsic output was only 0.95%, and the average error of the crosstalk output was only 1.93%. Figure 5 shows a comparison between the non-autoregressive Transformer prediction 4-level pulse amplitude modulation transmitter signal and the simulation results of the traditional simulation software for a 2-link system, given the circuit parameters and the input signal of each link. In the test of the 2-link system, the average error of the method of the present application compared with the true simulation results for 1000 test samples was only 0.96%, and compared with the traditional simulation software which needs 3.30s to complete the simulation of the two-link system to obtain the transmitter output signal, the method of the present application only needs 7.23ms, achieving a speedup of up to 456 times. In the test of the 16-link system, the average error of the method of the present application compared with the true simulation results was only 1.07%, and the traditional simulation software needs 20.2s to complete the simulation, and the method of the present application only needs 21.4ms, achieving a speedup of up to 944 times. The method of the present application shows high accuracy and superior simulation efficiency in modeling the 4-level pulse amplitude modulation transmitter output signal.

[0125] In summary, the present application significantly improves the efficiency of the model in predicting the 4-level pulse amplitude modulation transmitter output voltage waveform through efficient encoding and decoding methods, and compared with the traditional simulation software, the simulation speed is significantly improved, while ensuring the high precision of the simulation results. The present application significantly improves the efficiency and accuracy of 4-level pulse amplitude modulation transmitter modeling and analysis through the cooperative work of the encoder and the decoder, and is particularly suitable for high-speed and high-density link systems, which helps to improve the data transmission quality and reduce the bit error rate.

[0126] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.< / mask> < / mask> < / mask> < / mask>

Claims

1. A method for modeling a 4-level pulse amplitude modulation high-speed transmitter based on a non-autoregressive Transformer, characterized in that, The method comprises the following steps: Step 1, obtaining training data through circuit simulation: input different PAM4 signals under different link parameters to obtain the output signal of the transmitter circuit as the training data; Step 2, establishing a deep learning model based on Transform: a deep learning model adopting an encoder-decoder architecture is established, including a non-sequence encoder and a Transform sequence decoder; Step 3, model training based on random mask: for an input signal of a given length, the length of the output signal sequence can be determined in advance and does not change with the input, and the deep learning model is trained by adopting a random mask strategy, so that the model has the ability to predict the mask position element; Step 4, model inference based on non-autoregressive decoding and filtering: input the link parameters and the disturbed link input signal into the deep model, input K=0, and obtain the intrinsic output signal under the circuit parameters and the input signal itself through non-autoregressive decoding and filtering; then for all other interference links, input the link parameters and the interference link input signal, input K=1, and obtain the crosstalk output signal under the circuit parameters and the interference input signal through non-autoregressive decoding and filtering; the intrinsic output signal and all other crosstalk output signals are added to obtain the output signal of the transmitter disturbed by other signals in the multi-communication link.

2. The method according to claim 1, wherein, The link parameters in step 1 include equalizer coefficients H0, power supply voltage V h , transmitting end load capacitance C L , S parameters S tl of the two transmission lines, receiving end equivalent resistance Z0 and receiving end pull-up voltage V p , PAM4 input signal x s is a sequence composed of "0" and "1", each waveform transmits 2bit signal, and there are 4 level values respectively representing "00", "01", "10" and "11", which are input into the link transmitting end in the form of trapezoidal wave, and the highest voltage V h , signal period t p , and rising / falling time relative signal period ratio r rf are described; and step 1 specifically comprises: Step 1.1, obtaining intrinsic output signal samples without crosstalk: Step 1.2, obtaining crosstalk signal samples: Step 1.

3. Transforming the continuous output voltage signal sequence into a discrete voltage category sequence: for intrinsic output and crosstalk output, respectively, divide their entire voltage range into several voltage categories with a step size of Δv I or Δv C Construct a dictionary for intrinsic output and a dictionary for crosstalk output to map the voltage value to its nearest category.

3. The method of claim 2, wherein the method is based on a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method. The step 1.1 specifically comprises: Step 1.1.1, input signal x s is applied to the first link while the second link input is held low. Step 1.1.2, record the output signal of the first link as the eigen output signal sample y intr .

4. The method of claim 2, wherein the method is based on a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method. The step 1.2 specifically comprises: Step 1.2.1, input signal x s is applied to the second link while the first link input is kept high. Step 1.2.2, record the interference signal of the second link to the first link and subtract the eigen output signal y from it, to obtain the crosstalk signal sample C intr . ij ​ 5. The method of claim 2, wherein the method is based on a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method, characterized in that, The step 1.3 specifically comprises: Step 1.3.1, Dictionary Construction: For the intrinsic output, define its voltage range as According to the step size Δv I The voltage range is divided, and a series of discrete voltage categories are generated, i.e. the dictionary D is constructed I : For the crosstalk output, its voltage range is defined as According to the step size Δv C The voltage range is divided to generate a series of discrete voltage classes, i.e. to construct the dictionary D C : Voltage value v k Corresponding to the k+1th class Class k+1 Where Class0 is assigned to D I And one of D C A special <mask>The element represents a mask;< / mask> Step 1.3.2, Voltage value to voltage class mapping: For each output voltage value, map each voltage value to its nearest voltage value v k , and find its nearest voltage class Class using the corresponding dictionary k+1 , thereby converting the continuous voltage signal sequence into a discrete voltage class sequence, which is used as the training target of the deep learning model in the subsequent training.

6. The method of claim 2, wherein the method is based on a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method. The non-sequence encoder in the step 2 is used to process unordered non-sequence input to generate a context vector, which is transmitted into the Transform sequence decoder, and each input feature encoding mode is specifically as follows: Step 2.1.1, using a Boolean variable K to distinguish the case of predicting the intrinsic output signal and the crosstalk output signal, a binary variable K is encoded into a d model dimensional vector K model through a 2 x d e embedding matrix, K e represents the encoded vector of K, d model is the dimension of the encoded vector, which is the same as the model dimension of the Transformer sequence decoder; Step 2.1.2, for scalar features H0, V h C L Z0, V p t p r rf Standardization and encoding: for scalar features H0, V h C L Z0, V p t p r rf Standardization is performed by subtracting the respective means and dividing by the standard deviation to reduce scale differences and accelerate convergence; the standardized features are then transformed into d through a multi-layer perceptron layer. model A vector of dimensionality; each feature is passed through a multilayer perceptron layer with different parameters but identical structure; each multilayer perceptron layer has two hidden layers, each containing 16 neurons, the hidden layers use the ReLU activation function, and the output layer uses a linear activation function; finally, scalar features H0 and V are obtained. h C L Z0, V p t p r rf The encoded vectors are represented as follows: Step 2.1.3, input signal sequence encoding: PAM4 signal has 4 levels, a total of 12 types of directional level transitions; first traverse the input sequence x of length m s , when x i < x i+1 , mark the rising edge at position i+1 or when x i > x i+1 marking a falling edge at position i where i = 0,..., m-2; since the signal is at low level before and after transmission, for a non-zero start bit x0, a rising edge is marked at position 1 For non-zero end bit x m-1 Mark a falling edge at position m 0 represents an invalid position; each type of conversion appears at most m' times, where if m is even, m' is defined as m / 2, and if m is odd, m' is defined as (m+1) / 2; 0 is filled to ensure uniform length; In this way, x s is converted to m' discrete position indices, which are then embedded into a set of continuous 4 m dimensional vectors by a 12(m+1) x 4 m dimensional embedding matrix; the position index embedding vectors for different types of conversion are passed through separate multi-layer perceptron layers with input dimension 4 m ; each multi-layer perceptron layer has 2 hidden layers, each containing 16 neurons, with the hidden layers using a ReLU activation function and the output layer using a linear activation function; after stacking the multi-layer perceptron layers, the final output is obtained contains 12 x m' vectors, each of dimension d model ; Step 2.1.4, S parameter encoding: by decomposing the S parameter matrix into a real part and a virtual part matrix, considering the symmetry of the S parameter itself, the 4*4 S parameter matrix in the two transmission line system is reconstructed into a 2*5 S parameter matrix with 10 effective elements; each element s in the S parameter matrix is translated and a logarithmic transformation is applied: s scaled = log(s + 1.1*min) S tl )) where S tl represents the S-parameter matrix of the transmission line, min(S tl ) represents the minimum element in the matrix, s is an element of S tl , s scaled is the scaling result after translation and logarithmic transformation; two consecutive convolutional neural network layers are used to process the S-parameter matrix of each frequency point, the first convolutional neural network layer expands the input two corresponding to the real and imaginary matrix channels to 16 output channels, and the second convolutional neural network layer further expands it to 32 output channels, each convolution kernel size is 1, and the convolutional neural network layer uses a ReLU activation function; the final feature map is flattened and passed through a linear layer to generate a f with a dimension of (len model , d S tl Encoded vector, where d model is the dimension of the encoded vector, len f is the number of S-parameter frequency points; Step 2.1.5, encode features combining into a set of unordered dimension vectors X e of length 8 + len f + 12 * m', into a Transformer sequence decoder that interacts with an output sequence.

7. The method according to claim 2, wherein, The Transform sequence decoder in the step 2 is used to combine the context vector generated by the non-sequence encoder and the category sequence of the transmitter output to generate a category probability distribution for each point in the sequence, and the specific steps are as follows: Step 2.2.1, input embedding and position encoding: first, the output signal is converted into an embedded category sequence, converting each category into a fixed-dimensional vector representation; position encoding is added to the embedded vector to retain the position information in the input sequence; position encoding can be generated by sine and cosine functions: where d model is the embedding dimension, i is the embedding dimension index, pos is the position in the sequence, PE (pos,k) represents the position encoding value at the kth dimension index at position pos, the encoding of even dimension indices at position pos is calculated using a sine function, and the encoding of odd dimension indices at position pos is calculated using a cosine function; Step 2.2.2, multi-head self-attention mechanism: the input vector X is separated into query vector Q, key vector K and value vector V through linear transformation: Q = XW Q K = XW K V = XW V where W Q ,W K ,W V respectively represent linear transformation matrices of the query vector Q, the key vector K and the value vector V, for converting the input vector X into the query vector Q, the key vector K and the value vector V respectively. The dot product of the query vector Q and the key vector K is computed, divided by a scaling factor, and passed through a softmax function to obtain the attention weights: where d k denotes the dimension of the key vectors K, Attention(Q, K, V) denotes the output of the attention function, the query vector Q is used to compute a similarity with the transpose K T computes the dot product and divides by the scaling factor After that, the obtained score is used to measure the importance of the corresponding value vector V; after processing by the softmax function, these scores are converted into weights, and then these weights are multiplied by the value vector V to obtain the weighted output, i.e. the result after attention concentration; softmax(*) represents the softmax function; The attention mechanism is split into multiple heads, and the results are calculated independently and then combined: MultiHead(Q, K, V) = Concat(head1,..., head h ) W O , wherein wherein denotes the linear transformation matrix used in the i-th head for transforming the original query vector Q, key vector K and value vector V into the corresponding space processed by this head, respectively; head1, …, head h denotes the output of each attention head, Concat(head1, …, head h )W O denotes the concatenation of the output vectors of all heads and then a linear transformation W O combines the information of these different heads, MltiHead(Q, K, V) is the output of the multi-head attention mechanism; and after the residual connection, layer normalization is performed. Step 2.2.3, Multi-head cross-attention: The input vector X and the encoded vector C from the non-sequence encoder, i.e., the context vector, are transformed linearly to generate the query vector Q, the key vector K, and the value vector V, respectively: Q = XW Qc K = CW Kc V = CW Vc where W Qc ,W Kc ,W Vc are linear transformation matrices in cross-attention that transform the input vector X and the context vector C into the respective query vector Q, key vector K, and value vector V; The dot product of the query vector Q and the key vector K is computed, divided by a scaling factor, and the attention weights are obtained using a softmax function: where d k denotes the dimension of the key vector K, Attention(Q, K, V) denotes the output of the attention function, the query vector Q from the input vector X is used to attend to each of the transposed key vectors K T The dot product is computed and divided by the scaling factor The resulting scores are then used to measure the importance of the corresponding value vectors V; after processing through the softmax function, these scores are converted into weights, which are then multiplied with the value vectors V to generate the weighted output, i.e., the result after cross-attention concentration: Cross-attention also adopts a multi-head mechanism, and the results are calculated independently and then combined: MultiHead(Q, K, V) = Concat(head1,..., head h ) W Oc , wherein wherein each head denotes the use of different linear transformation matrices transforming the query vector Q, the key vector K and the value vector V into the corresponding space of the head processing; the output of each attention head is spliced and linearly transformed W Oc merging to form the final cross-attention output; finally, the output is connected by a residual connection and normalized by a layer Step 2.2.4, Position-wise feed-forward network: The vector at each position is processed independently through two linear transformations and a ReLU activation function: FFN(x) = max(0, xW1 + b1)W2 + b2 where FFN is a two-layer network structure that operates independently at each position. It first performs a linear transformation on the encoding vector x at each position, xW1 + b1, then introduces nonlinearity through the ReLU activation function max(0, z), and finally performs a second linear transformation max(0, xW1 + b1)W2 + b2; here, W1 and W2 are weight matrices, and b1 and b2 are bias terms, all of which are learnable; Step 2.2.5, Linear output layer: The final output is mapped to the dimension of the dictionary size through a linear layer to obtain the probability distribution of each element in the output sequence.

8. The method of claim 1, wherein the method is based on a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method. The specific training process in step 3 includes: Step 3.1, Randomly select mask elements: Randomly sample the number n of mask elements from a uniform distribution over 1 to n mask , and randomly select n mask more elements as masked elements Y mask , and replace their values with <mask>i.e., Class0;< / mask> Step 3.2, Input random mask sequence: The Transformer sequence decoder receives a random mask sequence and a context vector, and generates an output sequence of the same length as the random mask sequence in one decoding; Step 3.3, optimization objective: train by optimizing the cross-entropy loss between the predicted and all Y mask elements of the target elements: where y i represents one target element in the set Y mask , which is selected as the mask, i.e., the part that the model needs to predict, X is the input feature, Z is the unmasked target element, P(y i | X, Z) represents the probability estimate of the deep learning model for the true class of the masked label y i , L CE is the cross-entropy loss function, which calculates the negative logarithm of the predicted probability of the deep learning model for the masked target element as the difference between the model prediction value and the actual target value, and the deep learning model is optimized by the stochastic gradient descent algorithm.

9. The method of claim 1, wherein the method is based on a non-autoregressive Transformer-based 4-level pulse amplitude modulation high-speed transmitter modeling method. The specific steps of each inference process in step 4 include: Step 4.1, Input full mask sequence: At the beginning of inference, a completely masked sequence is input into the Transformer sequence decoder; Step 4.2, Predict the class of each masked element in parallel: The Transformer sequence decoder simultaneously predicts the class of all masked elements by selecting the class with the highest probability from the class set for each element; Step 4.3, Reconstruct the output signal: Convert the predicted class to the actual voltage signal value using the dictionary; Step 4.4, Filter the output signal: Apply the Savitzky-Golay filter to the first decoded and reconstructed signal sequence to smooth the unevenness of the waveform and correct errors, ultimately obtaining a smooth output signal.

Images