Embedded system-oriented feedback deep neural network digital pre-distortion method and system

Abstract

The invention discloses a feedback deep neural network digital pre-distortion method for an embedded system, and the method comprises the steps: constructing a deep neural network with a multi-stage cascade structure, each stage of cascade structure comprising a delay unit and a full connection layer, and the full connection layer having a feedback input mechanism; pre-estimating parameters of the power amplifier to obtain characteristic parameters of the power amplifier; based on the power amplifier characteristic parameters, training the deep neural network to obtain a pre-distortion model; in the training and forward reasoning process of the deep neural network, a Dropout mechanism is applied to each full connection layer in real time, and the calculation complexity is dynamically reduced; and performing pre-distortion processing on an input signal by using the pre-distortion model. The invention also discloses a pre-distortion system for realizing the method. According to the method, the calculation complexity is remarkably reduced while the pre-distortion performance is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of reducing amplifier nonlinear distortion, and particularly to a feedback deep neural network digital predistortion method and system for an embedded system. Background Art

[0002] In a 5G communication system, in order to improve spectral efficiency, the modulation method adopted has a high peak-to-average power ratio, which poses higher requirements on the linearity of power amplifiers. Usually, predistortion processing is adopted to improve the linearity of power amplifiers.

[0003] The basic principle of predistortion processing is to introduce a predistortion signal opposite to the nonlinear characteristics of the power amplifier at the input end of the power amplifier, so that after the input signal passes through the predistorter, a distortion opposite to the nonlinear distortion of the power amplifier is generated. Therefore, after being amplified by the power amplifier, the nonlinear distortion generated by the power amplifier can be cancelled, and the output signal can be as close to linear as possible.

[0004] When traditional digital predistortion technology processes broadband signals, due to the existence of memory effects, its modeling accuracy and system performance both face severe challenges. The main problems include: 1. It is difficult for traditional neural network structures to accurately capture the dynamic characteristics of power amplifiers; 2. The training efficiency of neural networks is low; 3. The high computational complexity of deep neural networks makes it difficult to be implemented in embedded systems; 4. For broadband signals, the existing modeling accuracy and system performance are insufficient. Summary of the Invention

[0005] Aiming at the defects of the above-mentioned existing technologies, the present invention provides a feedback deep neural network digital predistortion method for an embedded system applicable to 5G broadband signals, which improves the system training efficiency and the modeling accuracy of the predistortion system, and enables the power amplifier to have better linearity.

[0006] The technical solution of the present invention is as follows:

[0007] A feedback deep neural network digital predistortion method for an embedded system includes the following steps:

[0008] Construct a deep neural network with a multi-stage cascade structure, where each stage of the cascade structure includes a delay unit and a fully connected layer, and the fully connected layer has a feedback input mechanism;

[0009] Estimate the parameters of the power amplifier to obtain the characteristic parameters of the power amplifier;

[0010] Based on the characteristic parameters of the power amplifier, train the deep neural network to obtain a predistortion model;

[0011] During the training and forward inference processes of the deep neural network, the Dropout mechanism is applied to each fully connected layer in real time to dynamically reduce the computational complexity;

[0012] The input signal is pre-distorted using the pre-distortion model.

[0013] Furthermore, the output of each cascaded structure is expressed as

[0014] y k (n) = f[W k ·x k (n) + F k ·y k (n - 1) + b k

[0015] where: y k (n) is the output of the k-th cascaded structure at time n, x k (n) is the delayed input vector [x(n - k), x(n - k - 1),..., x(n - k - M)] T , W k is the forward weight matrix, F k is the feedback weight matrix, b k is the bias vector, f[·] is the activation function, and M is the memory depth of each cascaded structure.

[0016] Furthermore, when estimating the parameters of the power amplifier, the input-output relationship of the power amplifier is expressed as

[0017]

[0018] where: y PA (n) is the output signal of the power amplifier at time n, x(n) is the input signal of the power amplifier, h p,m is the characteristic parameter of the power amplifier to be estimated, p is the order, and m is the time delay.

[0019] Furthermore, the least squares algorithm is used as the parameter estimation method when estimating the parameters of the power amplifier.

[0020] Furthermore, during the training of the deep neural network, the Dropout mechanism is used to randomly disconnect some neurons. During the forward inference process of the deep neural network, the Dropout mechanism is as follows:

[0021] For the k-th layer network, the Dropout decision at time n is expressed as

[0022] m k (n) = σ(αR xy (n) + βR yy ​(n) + γ)

[0023] where: m k (n) is the Dropout mask of the k-th layer at time n, σ(·) is the sigmoid function, R xy (n) is the cross-correlation function of the input signal and the output signal, R yy (n) is the autocorrelation function of the output signal, and α, β, γ are adjustable weight parameters;

[0024]

[0025] where: x(·) is the input signal, y(·) is the output signal, L is the observation window length of the input signal, and M is the observation window length of the output feedback signal.

[0026] Another technical solution of the present invention is:

[0027] A feedback-type digital predistortion system for an embedded system, comprising:

[0028] A network construction module for constructing a deep neural network with a multi-stage cascade structure, each stage of the cascade structure includes a delay unit and a fully-connected layer, and the fully-connected layer has a feedback input mechanism;

[0029] A parameter estimation module for estimating the parameters of the power amplifier to obtain the characteristic parameters of the power amplifier;

[0030] A network training module for training the deep neural network based on the characteristic parameters of the power amplifier to obtain a predistortion model;

[0031] A real-time Dropout control module for applying the Dropout mechanism to each fully-connected layer in real time during the training and forward inference processes of the deep neural network to dynamically reduce the computational complexity;

[0032] A predistortion processing module for performing predistortion processing on the input signal by using the predistortion model.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] Improve the modeling accuracy of the predistortion system for the nonlinear characteristics of the power amplifier;

[0035] Significantly reduce the system computational complexity, reduce the hardware resource requirements and power consumption, and are suitable for implementation in embedded systems;

[0036] Through the online adaptive training mechanism, realize the real-time optimization and fast convergence of the system;

[0037] Enhance the system's ability to process broadband signals. Brief Description of the Drawings

[0038] Figure 1 This is a schematic structural diagram of the system for implementing the feedback deep neural network digital predistortion method for an embedded system in an embodiment of the present invention.

[0039] Figure 2 This is a real-time Dropout implementation block diagram of the feedback deep neural network digital predistortion system for an embedded system in an embodiment of the present invention. Detailed Embodiments

[0040] The present invention will be further described below in conjunction with embodiments, but it is not intended to limit the present invention.

[0041] The feedback deep neural network digital predistortion system for an embedded system in this embodiment includes:

[0042] A network construction module for constructing a deep neural network with a multi-stage cascade structure. Each stage of the cascade structure includes a delay unit and a fully connected layer, and the fully connected layer has a feedback input mechanism.

[0043] A parameter estimation module for estimating the parameters of the power amplifier to obtain the characteristic parameters of the power amplifier.

[0044] A network training module for training the deep neural network based on the characteristic parameters of the power amplifier to obtain a predistortion model.

[0045] A real-time Dropout control module for applying the Dropout mechanism to each fully connected layer in real time during the training and forward inference processes of the deep neural network to dynamically reduce the computational complexity. The real-time Dropout control module includes: a neuron dynamic selection unit for determining the neurons participating in the calculation in real time; a computational complexity control unit for dynamically adjusting the Dropout ratio according to the system resources; and a performance monitoring unit for ensuring that the predistortion performance meets the requirements.

[0046] A predistortion processing module for performing predistortion processing on the input signal using the predistortion model.

[0047] As a preferred embodiment, the digital predistortion system may further include an error generation unit for calculating the error between the predistortion output and the desired output; a delay compensation unit for compensating for the system signal processing delay; and a resource monitoring unit for monitoring the usage of system computing resources.

[0048] The following will be combined with Figure 1 and Figure 2 shown to make a detailed description of the implementation of the feedback deep neural network digital predistortion method for an embedded system by this digital predistortion system.

[0049] First, a deep neural network with a multi - level cascaded structure is constructed. In the deep neural network with a multi - level cascaded structure, each level of the network contains multiple cascaded Z^-1 delay units and a fully - connected layer, and each fully - connected layer has a feedback input structure. The feedback input structure obtains the layer output signal at the previous moment as the feedback signal. This structure can effectively capture the non - linear memory effect of the power amplifier by introducing time - delay input and output feedback, and improve the modeling accuracy of the predistortion system.

[0050] In the deep neural network with a multi - level cascaded structure, the output of the neural network of the k - th level cascaded structure is expressed as

[0051] y k (n)=f[W k ·x k (n)+F k ·y k (n - 1)+b k

[0052] where: y k (n) is the output of the k - th level cascaded structure at time n, x k (n) is the delayed input vector [x(n - k), x(n - k - 1),..., x(n - k - M)] T , W k is the forward weight matrix, F k is the feedback weight matrix, b k is the bias vector, f[·] is the activation function, and M is the memory depth of each level of the cascaded structure.

[0053] Secondly, a power amplifier model is used to estimate the parameters of the power amplifier and obtain the power amplifier characteristic parameters. The power amplifier model parameter estimation uses the ideal transmitted signal as the input and the power amplifier feedback signal as the training sequence, and estimates the power amplifier model parameters through an adaptive algorithm.

[0054] The power amplifier model is such that the input - output relationship of the power amplifier can be described by the following mathematical expression:

[0055]

[0056] where: y PA (n) is the output signal of the power amplifier at time n, x(n) is the input signal of the power amplifier, h p,m is the power amplifier characteristic parameter to be estimated, which describes the contributions of different orders and memory depths, p is the order, P usually takes values from 3 to 7 to meet the accuracy requirements, m is the time delay, and M is the memory depth, representing the length of the historical signal considered, which is usually related to the signal bandwidth.

[0057] ​This expression reflects two key characteristics of the power amplifier:

[0058] 1. Nonlinear characteristic: Described by terms of different orders p, where |x(n - m)|^{p - 1} represents amplitude nonlinearity.

[0059] 2. Memory effect: Described by the time delay m, considering the influence of input signals at different times.

[0060] Based on the aforementioned mathematical expression, the parameter estimation problem is transformed into a least - squares problem

[0061]

[0062] The RLS (Recursive Least Squares) algorithm is adopted to achieve real - time parameter estimation:

[0063] Parameter update equation:

[0064] Gain vector calculation:

[0065] Covariance matrix update: P(n) = λ -1 [P(n - 1) - k(n)x T (n)P(n - 1)]

[0066] where: λ is the forgetting factor, and its value range is (0, 1]. A smaller λ value provides faster tracking ability, and a larger λ value provides better steady - state performance.

[0067] The convergence criterion of the algorithm is

[0068]

[0069] where ∈ is a preset convergence threshold.

[0070] As a preferred embodiment, the algorithm convergence can be monitored regularly, the forgetting factor can be adjusted dynamically, and matrix regularization can be performed when necessary to ensure numerical stability.

[0071] Used to calculate the output of the power amplifier based on the power amplifier characteristic parameters, and a predistortion model is obtained by training a deep neural network. Batch normalization can be used to optimize the training process.

[0072] The weighted mean square error (MSE) is adopted as the loss function during the training process:

[0073]

[0074] where: θ represents the set of network parameters, y pred (n) is the model prediction output, y target (n) is the target output, αn is the sample weight coefficient, λ is the L2 regularization coefficient, and N is the batch size.

[0075] Use Stochastic Gradient Descent (SGD) with momentum for parameter update:

[0076]

[0077] where: v t is the momentum term, μ is the momentum coefficient, usually taken as 0.9, η is the learning rate, and t is the number of iterations.

[0078] The learning rate is dynamically adjusted according to the following rules:

[0079]

[0080] where: η0 is the initial learning rate, γ is the decay coefficient, usually taken as 0.95, and T is the learning rate adjustment period.

[0081] Define multiple convergence criteria:

[0082] Convergence of the loss function:

[0083] Convergence of the parameter gradient:

[0084] Convergence of the validation set performance: |NMSE val,t+1 -NMSE val,t | < ∈3

[0085] where: ∈1, ∈2, ∈3 are preset thresholds, and NMSE val is the normalized mean square error on the validation set.

[0086] Early stopping is triggered when any of the following conditions is met: the validation set performance has not improved for K consecutive epochs, the maximum number of iterations is reached, or all convergence criteria are satisfied simultaneously.

[0087] During the above training process, the Dropout mechanism is applied to dynamically reduce the computational complexity. The Dropout mechanism during training is to randomly disconnect some neuron connections to improve the generalization ability.

[0088] Finally, based on the trained predistortion model (i.e., the deep neural network with a multi-level cascaded structure), the input signal is predistorted. During the forward inference process, the Dropout mechanism is kept working, dynamically screening the neurons in each fully connected layer, only retaining the selected neurons to participate in the calculation, dynamically reducing the number of activated neurons, and at the same time dynamically adjusting the Dropout ratio according to actual needs, reducing the computational complexity of the fully connected layer through real-time Dropout.

[0089] The Dropout mechanism adopted by the present invention is dynamically adjusted based on the correlation between the input signal and the output feedback signal. For the k-th layer of the network, the Dropout decision at time n can be expressed as:

[0090] m k (n) = σ(αR xy (n) + βR yy (n) + γ)

[0091] where: m k (n) is the Dropout mask of the k-th layer at time n, σ(·) is the sigmoid function, R xy (n) is the cross-correlation function of the input signal and the output signal, R yy (n) is the autocorrelation function of the output signal, and α, β, γ are adjustable weight parameters;

[0092]

[0093] where: x(·) is the input signal, y(·) is the output signal, L is the observation window length of the input signal, and M is the observation window length of the output feedback signal.

[0094] The final neuron activation probability is determined in the following way

[0095] p k (n) = clip(m k (n), p min , p max )

[0096] where: p k (n) is the neuron retention probability of the k-th layer at time n, p min is the minimum retention probability (ensuring basic performance), p max is the maximum retention probability (guaranteeing computational simplicity), and clip(·) is the truncation function.

[0097] Please refer to Figure 2 As shown, when performing real-time Dropout control, the input signal first passes through a multi-path delay unit (z^-1). The amplitude squared value (|·|2) is calculated for each delayed signal. The delay unit and the amplitude calculation form a multi-stage cascaded structure for capturing the time-domain characteristics of the signal. The Dropout control module receives all the delay and amplitude information and dynamically controls the activation state of the neural network according to the characteristics of the input signal, realizing the adaptive mapping between the signal characteristics and the network complexity.

[0098] This mechanism has the following characteristics: when the input and output signals are strongly correlated, more neurons are retained; when the signal correlation is weak, the Dropout ratio is increased; the correlation analysis adopts a sliding window method, supporting real-time calculation; the Dropout strategy is adjusted in real time according to the signal characteristics to adapt to the dynamic characteristic changes of the power amplifier.

[0099] Adopt a real-time control mechanism to replace the traditional random Dropout. The control strategy is closely related to the characteristics of the input signal, forming a closed-loop adaptive control system to ensure the dynamic balance of performance and complexity. By reducing the scale of matrix operations through real-time Dropout, the number of memory accesses can be reduced, and the power consumption requirements can be lowered.

[0100] The following introduces a specific implementation scheme of the embodiments of the present invention:

[0101] System input part: Input signal sampling rate: 500 MHz; Signal quantization bits: 12 bit; Signal bandwidth: 100 MHz.

[0102] Multi-stage cascaded deep neural network structure: Number of cascaded layers: 4 levels; Number of delay units per level: 2; Number of neurons in the fully connected layer: 64; Activation function: ReLU.

[0103] Implementation of the feedback mechanism: Feedback signal delay: 1 sampling period; Initialization range of feedback weights: [-0.1, 0.1]; Signal fusion method: Weighted summation.

[0104] Real-time Dropout configuration: Dropout update frequency: Each sampling period

[0105] Computing resource optimization: Matrix operation optimization: Sparse matrix storage; Memory access optimization: Data caching mechanism.

[0106] Implementation on an embedded platform: Target platform: Xilinx Zynq xc7a200t; Processing delay: <1 us; Resource occupancy: DSP48: Usage rate <40%, BRAM: Usage rate <35%, LUT: Usage rate <45%.

[0107] Implementation of the training process:

[0108] 1. PA parameter estimation: The test signal is a broadband OFDM signal, with 10,000 sampling points. The least squares method is used as the parameter identification algorithm, and the estimation accuracy is that the mean square error < -45 dB;

[0109] 2. Network training process: Training data volume: >50,000,000 groups, batch size is 1, and the convergence condition is SNR > 30 dB.

[0110] The system performance of the above specific implementation scheme is as follows:

[0111] 1. Predistortion performance: ACLR improvement: > 25 dB, EVM improvement: > 20 dB, spectral regrowth suppression: > 45 dB;

[0112] 2. Computational efficiency: Single predistortion processing time: < 1 μs; Computational resource savings: > 50%; Power consumption reduction: > 40%.

[0113] The present invention realizes an efficient and lightweight digital predistortion method, which is particularly suitable for implementation in an embedded system. Experimental results show that while ensuring the predistortion performance, this method significantly reduces the computational complexity and hardware resource requirements, and has important engineering application value.

Claims

1. A feedback-based digital predistortion method for deep neural networks in embedded systems, characterized in that It includes the following steps: Construct a deep neural network with a multi-level cascade structure, where each level of the cascade structure includes a delay unit and a fully connected layer, and the fully connected layer has a feedback input mechanism; Estimate the parameters of the power amplifier to obtain the characteristic parameters of the power amplifier; Based on the characteristic parameters of the power amplifier, train the deep neural network to obtain a predistortion model; During the training and forward inference processes of the deep neural network, apply the Dropout mechanism to each fully connected layer in real time to dynamically reduce the computational complexity; Use the predistortion model to perform predistortion processing on the input signal.

2. The feedback-based deep neural network digital predistortion method for an embedded system according to claim 1, characterized in that, The output of each level of the cascade structure is expressed as y k (n) = f[W k ·x k (n) + F k ·y k (n - 1) + b k ​ Where: y k (n) is the output of the k-th cascaded structure at time n, x k (n) is the delayed input vector [x(n - k), x(n - k - 1),..., x(n - k - M)] T , W k is the forward weight matrix, F k is the feedback weight matrix, b k is the bias vector, f[·] is the activation function, and M is the memory depth of each cascaded structure.

3. The feedback type deep neural network digital predistortion method for an embedded system according to claim 1, wherein When estimating the parameters of the power amplifier, the input-output relationship of the power amplifier is expressed as where: y PA (n) is the output signal of the power amplifier at time n, x(n) is the input signal of the power amplifier, h p,m is the characteristic parameter of the power amplifier to be estimated, p is the order, and m is the time delay.

4. The feedback-type deep neural network digital predistortion method for an embedded system according to claim 1, wherein When estimating the parameters of the power amplifier, the least squares algorithm is used as the parameter estimation method.

5. The feedback-type deep neural network digital predistortion method for an embedded system according to claim 1, wherein During the training of the deep neural network, use the Dropout mechanism to randomly disconnect some neurons. During the forward inference process of the deep neural network, the Dropout mechanism is: For the k-th layer network, the Dropout decision at time n is denoted as m k (n) = σ(αR xy (n) + βR yy (n) + γ) where: m k (n) is the Dropout mask of the k-th layer at time n, σ(·) is the sigmoid function, R xy (n) is the cross-correlation function between the input signal and the output signal, R yy (n) is the autocorrelation function of the output signal, and α, β, γ are adjustable weight parameters; Where: x(·) is the input signal, y(·) is the output signal, L is the observation window length of the input signal, and M is the observation window length of the output feedback signal.

6. A feedback-type deep neural network digital predistortion system for an embedded system, characterized in that, It includes: A network construction module for constructing a deep neural network with a multi-level cascade structure, where each level of the cascade structure includes a delay unit and a fully connected layer, and the fully connected layer has a feedback input mechanism; A parameter estimation module for estimating the parameters of the power amplifier to obtain the characteristic parameters of the power amplifier; A network training module for training the deep neural network based on the characteristic parameters of the power amplifier to obtain a predistortion model; A real-time Dropout control module for applying the Dropout mechanism to each fully connected layer in real time during the training and forward inference processes of the deep neural network to dynamically reduce the computational complexity; A predistortion processing module for performing predistortion processing on the input signal using the predistortion model.

7. The feedback-type deep neural network digital predistortion system for an embedded system according to claim 6, characterized in that, The output of each level of the cascade structure is expressed as y k (n) = f[W k ·x k (n) + F k ·y k (n - 1) + b k ​ where: y k (n) is the output of the k-th cascaded structure at time n, x k (n) is the delayed input vector [x(n - k), x(n - k - 1),..., x(n - k - M)] T , W k is the forward weight matrix, F k is the feedback weight matrix, b k is the bias vector, f[·] is the activation function, and M is the memory depth of each cascaded structure.

8. The feedback-type deep neural network digital predistortion system for an embedded system according to claim 6, wherein When estimating the parameters of the power amplifier, the input-output relationship of the power amplifier is expressed as where: y PA (n) is the output signal of the power amplifier at time n, x(n) is the input signal of the power amplifier, h p,m is the characteristic parameter of the power amplifier to be estimated, p is the order, and m is the time delay.

9. The feedback-type deep neural network digital predistortion system for an embedded system according to claim 6, characterized in that, When estimating the parameters of the power amplifier, the least squares algorithm is used as the parameter estimation method.

10. The feedback-type deep neural network digital predistortion system for an embedded system according to claim 6, wherein During the training of the deep neural network, use the Dropout mechanism to randomly disconnect some neurons. During the forward inference process of the deep neural network, the Dropout mechanism is: For the k-th layer network, the Dropout decision at time n is denoted as m k (n) = σ(αR xy (n) + βR yy (n) + γ) where: m k (n) is the Dropout mask of the k-th layer at time n, σ(·) is the sigmoid function, R xy (n) is the cross-correlation function of the input signal and the output signal, R yy (n) is the autocorrelation function of the output signal, and α, β, γ are adjustable weight parameters; Where: x(·) is the input signal, y(·) is the output signal, L is the observation window length of the input signal, and M is the observation window length of the output feedback signal.

Images