A multi-dimensional shaping combined coherent end-to-end equalization system

Abstract

The present application belongs to the technical field of coherent optical access network, and specifically relates to a multi-dimensional shaping combined coherent end-to-end equalization system. The present application integrates the technologies of probability shaping, geometric shaping, pre-equalization, post-equalization and decoding in the coherent transmission system, uniformly allocates and jointly optimizes according to the actual channel, and strives to achieve the best channel capacity; through an E2E mechanism, the OLT and the ONU closely cooperate; the present application comprises establishing an artificial neural network channel model (ACM), deploying a corresponding transmitter artificial neural network (TANN) and a receiver artificial neural network (RANN) in an intelligent transmitter / receiver; these intelligent transmitters / receivers realize optimized signal generation from the bit to the waveform level and decoding from the symbol waveform to the bit level, and realize end-to-end optimized channel equalization; the present application can improve the sensitivity and dynamic range of the signal, improve the performance of the system, and realize a higher net data rate, so as to improve the channel capacity, improve the signal quality, extend the transmission distance and reduce the cost.

Description

Technical Field

[0001] This invention belongs to the field of coherent optical access network technology, and specifically relates to a coherent access network channel equalization system. Background Technology

[0002] Over the past decade, the throughput of optical access networks has increased a thousandfold. With the rise of emerging access network services such as cloud storage, high-definition video streaming, and VR 3D interactive services, the demand for optical access networks is expected to continue its rapid growth over the next decade. Unlike traditional direct detection solutions that represent signals only in one dimension—intensity or amplitude—coherent transmission can linearly transform the optical field and modulate and detect signals in amplitude, phase, and polarization. Therefore, multidimensional modulation can be performed to achieve high spectral efficiency, enabling high-speed data transmission based on low-bandwidth optoelectronic devices. Coherent detection offers greater power budgets and higher bit rates due to its superior receiver sensitivity and multidimensional modulation characteristics. Driven by the need for higher receiver sensitivity, higher-order modulation formats, advanced digital signal processing, and flexible deployment capabilities, coherent PON (Coherent Passive Optical Network, CPON) technology has become a solution for achieving access rates of 100 Gbits / s (Gbps) and above.

[0003] Coherent optical networks, due to their structural complexity, are also subject to more diverse noise. At the transmitter, there is electro-optic nonlinearity in the modulator, phase noise due to laser linewidth, and high-frequency losses in the frequency domain due to bandwidth limitations of the digital-to-analog converter. In the channel, dispersion loss occurs as the transmission wavelength shifts from 1310 nm to 1550 nm. Nonlinear effects in the fiber, such as self-phase modulation and four-wave mixing, cause signal distortion, spectral spread, and inter-channel crosstalk. These losses limit transmission distance, reduce data transmission rates, and affect overall system performance. To mitigate the impact of fiber nonlinear losses on signal transmission, effective compensation techniques, such as dispersion compensation and nonlinearity compensation, are needed to ensure reliable and high-quality communication. In coherent systems, optical and electrical distortions (such as dispersion) can be completely handled in the digital domain. This also places high demands on equalization algorithms in coherent optical systems. Summary of the Invention

[0004] The purpose of this invention is to propose an autoencoder equalization system based on end-to-end joint optimization—a coherent end-to-end equalization system based on multidimensional shaping joint optimization.

[0005] This invention integrates probabilistic shaping, geometric shaping, pre-equalization, post-equalization, and decoding techniques from coherent transmission system equalization technologies. It performs unified allocation and joint optimization based on the actual channel to achieve optimal channel capacity. Through an E2E mechanism, the Optical Line Terminal (OLT) and Optical Network Unit (ONU) collaborate closely. When a new user comes online, the OLT sends an estimation signal to the ONU and establishes a downlink neural network-based channel model (ACM). The downlink ACM simulates the channel, and the transmitted uplink channel estimation signal is transmitted to the OLT together. Upon receiving the signal, the OLT then establishes the uplink ACM. Through the corresponding ACM, a training base can create a suitable autoencoder model. Ultimately, the corresponding Transmitter Artificial Neural Network (TANN) and Receiver Artificial Neural Network (RANN) can be deployed in the intelligent transmitter / receiver. These intelligent transmitters / receivers can achieve optimized signal generation from bit to waveform level and decoding from symbol waveform to bit level, realizing end-to-end optimized channel equalization. Among them, TANN can perform constellation shaping, geometry shaping, and channel pre-equalization operations, while RANN can perform post-equalization and decoding.

[0006] The multidimensional shaping combined coherent end-to-end equalization system provided by this invention includes a probability shaping module, a coherent transmission demodulation module, an ACM training module, an autoencoder pre-training module, an end-to-end training module, and an autoencoder signal generation, transmission, and demodulation module; wherein:

[0007] The probability shaping module estimates the channel transmission conditions based on the transmitted test signal, and then performs probability shaping encoding on the signal strength to obtain the encoded bit stream.

[0008] The coherent transmission demodulation module modulates the signal processed by the probability shaping module onto the optical carrier through a coherent modulator. After transmission through optical fiber, the signal is received by a coherent receiver, converted into an electrical signal, and then processed in the backend to finally obtain a bit data stream.

[0009] The ACM training module uses the transmitted complex symbols and the complex symbols obtained after post-equalization as the teacher set and training set of the neural network, respectively, and obtains the ACM channel model through training.

[0010] The autoencoder pre-training module trains a constellation point encoder with probability shaping based on the principle of maximizing the minimum spacing between symbols, and inputs it into an AWGN channel (Additive White Gaussian Noise). The output sequence of the AWGN channel is then used to train a corresponding decoder.

[0011] The end-to-end training module puts the pre-trained autoencoder model into the ACM channel for end-to-end optimization to obtain an autoencoder for a specific ACM channel.

[0012] The autoencoder signal generation, transmission and demodulation module generates a series of coherent transmission signals from the TANN part of the trained autoencoder. The signals are modulated onto an optical carrier by a modulator and transmitted. After transmission through an actual optical fiber channel, the signals are coherently demodulated at the receiving end to recover complex symbols. The RANN part of the autoencoder then equalizes and decodes the signals into bits.

[0013] Furthermore, in the probability shaping module, the emission entropy of the signal can be adjusted by changing the constellation distribution to a Maxwell-Boltzmann distribution:

[0014]

[0015] Where v is the shaping factor controlling the entropy, typically ranging from 0 to 1. A value of 0 represents the standard constellation point distribution, and a larger value results in a smaller emission entropy. x represents a single symbol, and X represents the set of all symbols. For the same QAM order, a specified value of v corresponds to a specific emission entropy, as shown in the formula:

[0016]

[0017] During operation, a uniformly distributed 32QAM signal is first transmitted as a test signal. The generalized mutual information (GMI) of the received signal is then obtained through channel estimation and calculation, expressed as:

[0018]

[0019] Among them, b k,i Let represent the i-th transmitted bit of the k-th symbol, X represent the symbol set, Xtransmitted represent the transmitted symbol set, and Y represent the received symbol set. It defines the average number of information bits per symbol, where N and m represent the number of symbols and bits, respectively. GMI can be used to reflect the performance of the PS signal after transmission. Normalized GMI (NGMI) can accurately predict the bit error rate before forward error correction. The formula for calculating NGMI is:

[0020] NGMI = 1 - (H - GMI) / m, (4)

[0021] Next, by continuously adjusting the specific emission entropy value, the NGMI is increased by increasing the v value in the probabilistic shaping. After iteration, the NGMI value is modulated to a level slightly larger than the threshold, for example, if the threshold is 0.92, it is modulated to between 0.92 and 0.93. The v value at this time is recorded, and the signal is probabilistically shaped according to this value.

[0022] Furthermore, in the coherent transmission demodulation module, the signal first passes through a symbol mapping module, which merges the bit stream that underwent probability shaping in the previous step into a complex signal through a mapping relationship. Finally, a coherent transmitter modulates the symbol-mapped symbols onto an optical carrier. After transmission through the optical fiber channel, the received optical signal is received by a coherent receiver, converted into an electrical signal, and then undergoes backend coherent demodulation, including downsampling, clock recovery, polarization recovery, and carrier recovery. After being restored to two complex signals, it undergoes another post-equalization, and finally, through decoding, the received binary data stream is obtained.

[0023] Furthermore, the ACM training module is trained based on the transmitted complex symbols and the complex symbols obtained after post-equalization. By using the transmitted complex symbols as input, an artificial neural network encoding process is used to convert them into high-dimensional feature representations. Simultaneously, the post-equalized complex symbols are used as the target output, and the model is optimized by comparing the differences between the input and output symbols. Through repeated training and adjustments, this module can learn an optimized ACM channel model for accurately predicting and simulating influencing factors such as nonlinear distortion, inter-symbol interference, and noise in optical fiber communication.

[0024] Furthermore, the autoencoder pre-training module is trained according to the principle of maximizing the minimum spacing between symbols. First, a probabilistically shaped constellation-point encoder maps the input symbols to a high-dimensional feature space, and the encoder parameters are adjusted to maximize the minimum spacing between different symbols. Then, the encoded symbols are transmitted through an AWGN channel, introducing channel interference and noise. Next, a decoder decodes the received symbols, and the decoder parameters are adjusted using a backpropagation algorithm to ensure the decoder can accurately reconstruct the original symbols. Through multiple iterative training iterations, the autoencoder pre-training module learns a compact representation of symbols and improves anti-interference capabilities, thereby enhancing signal transmission quality and decoding accuracy.

[0025] Furthermore, in the end-to-end training module, the input symbols are encoded into high-dimensional feature representations using a pre-trained autoencoder model. The encoded symbols are then transmitted through an ACM channel, which simulates interference and distortion effects in a real communication environment. Next, an end-to-end optimization algorithm adjusts the parameters of the autoencoder model based on the received symbols and the desired output to minimize signal distortion and error. Through multiple iterative optimizations, an optimized autoencoder model for a specific ACM channel is obtained, better adapting to the channel's characteristics and improving signal transmission quality and decoding accuracy. This method effectively utilizes the characteristics of the autoencoder to achieve adaptive optimization for the ACM channel.

[0026] Furthermore, in the autoencoder signal generation, transmission, and demodulation module, a series of coherent transmit signals can be generated through the TANN part of the trained autoencoder model. These signals, after modulation by a modulator, are transmitted onto an optical carrier on the optical fiber. After the signal is transmitted through the actual optical fiber channel, the coherent demodulator at the receiving end recovers it into complex symbols. Then, the RANN part of the autoencoder model performs equalization processing on the received signal and decodes it into the corresponding bit sequence. Through this process, the recovery and decoding of the transmitted signal can be achieved, thereby realizing reliable data transmission. In this part, the invention further proposes a framework for efficiently establishing a smart transceiver when a new user comes online. Specifically, in the downlink, the smart transceiver system is established through three steps. First, the OLT transmits a test signal; then, the ONU receives the signal and establishes the ACM for the downlink channel. The ACM is sent back to the OLT; finally, the OLT obtains the trained network through the ACM, deploys the TANN to the OLT's transmitter, and sends the RANN to the ONU for deployment. In the uplink, the ONU first transmits a test signal; the OLT receives the signal and generates an ACM. After obtaining the trained network, the end OLT deploys the RANN and sends the TANN to the ONU for deployment. Then, signal transmission supported by the TANN and RANN can proceed.

[0027] This invention also provides the workflow of the above-mentioned multidimensional shaping combined coherent end-to-end equalization system, specifically as follows:

[0028] (1) The signal is probabilistically shaped and encoded using a probability shaping module;

[0029] (2) The signal is modulated onto an optical carrier, transmitted through optical fiber, demodulated to recover an electrical signal, and processed to obtain a bit data stream;

[0030] (3) Obtain the ACM channel model through the training module;

[0031] (4) Train the encoder and the corresponding decoder after probability shaping;

[0032] (5) The pre-trained autoencoder model is put into the ACM channel for optimization to obtain an autoencoder for a specific ACM channel;

[0033] (6) The transmitted signal is generated by the TANN part of the autoencoder, and after transmission and demodulation, it is equalized and decoded into bits by the RANN part.

[0034] The multidimensional shaping combined coherent end-to-end equalization system of this invention has the following advantages:

[0035] (1) End-to-end joint optimization: Through end-to-end joint optimization of self-encoder equalization technology, the entire transmission link is optimized in an integrated manner. The entire process from signal generation to decoding is uniformly coordinated and jointly optimized, which improves the overall performance and channel capacity.

[0036] (2) Integration of multiple equalization techniques: The present invention integrates multiple equalization techniques in coherent transmission systems, including probability shaping, geometric shaping, pre-equalization, post-equalization and decoding techniques. Through mutual cooperation and optimization, the advantages of each technique are fully utilized, thereby improving the reliability and performance of signal transmission.

[0037] (3) Adaptive performance optimization: By using a pre-trained autoencoder model and an ACM channel model, adaptive optimization for specific channel conditions is achieved, enabling the system to dynamically adjust and optimize according to the actual channel state, providing more accurate and reliable signal recovery and decoding.

[0038] (4) Efficient signal shaping and decoding: Constellation shaping, geometric shaping, pre-equalization, post-equalization and decoding techniques implemented by using TANN and RANN can effectively suppress signal distortion and interference, improve signal quality and decoding accuracy, thereby improving system performance and signal transmission quality.

[0039] (5) Close collaboration and adaptability: The E2E mechanism in this invention realizes close collaboration between the optical line terminal (OLT) and the optical network unit (ONU), which can be dynamically adjusted and optimized according to actual needs, providing flexibility and adaptability, and is suitable for different network environments and application scenarios.

[0040] In summary, the multi-dimensional shaping coherent end-to-end equalization technology provided by this invention has significant advantages over traditional channel equalization techniques. Through end-to-end joint optimization, it integrates multiple key technologies such as probabilistic shaping, geometric shaping, pre-equalization, post-equalization, and decoding, enabling the system to comprehensively optimize the signal transmission process and significantly improve performance and channel capacity. This multi-dimensional shaping coherent end-to-end equalization technology has broad application prospects and can play an important role in the field of optical access networks, improving system performance and transmission efficiency to meet the ever-increasing data transmission demands. Attached Figure Description

[0041] Figure 1 This is a block diagram of the coherent end-to-end equalization system of the multidimensional shaping combined according to the present invention.

[0042] Figure 2 This section describes the training methods, usage methods, and performance characteristics of the ACM channel.

[0043] Figure 3 This is the performance of the coherent end-to-end equalization technology of multidimensional shaping combined according to the present invention.

[0044] Figure 4 This example illustrates a specific implementation scheme that enables efficient establishment of a smart transceiver when a new user comes online.

[0045] In the diagram, the following modules are labeled: 101 is the probabilistic shaping module, 102 is the coherent transmission demodulation module, 103 is the ACM training module, 104 is the autoencoder pre-training module, 105 is the end-to-end training module, and 106 is the autoencoder signal generation, transmission, and demodulation module. Detailed Implementation

[0046] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0047] This invention proposes an end-to-end joint optimization-based autoencoder equalization system that integrates probabilistic shaping, geometric shaping, pre-equalization, post-equalization, and decoding techniques from coherent transmission system equalization technologies. It can uniformly allocate and jointly optimize based on the actual channel, striving to achieve optimal channel capacity. Through an E2E mechanism, the OLT and ONU can achieve close collaboration. When a new user comes online, the OLT sends an estimated signal to the ONU and establishes a downlink ACM based on an artificial neural network. The downlink ACM simulates the channel and transmits the uplink channel estimation signal together to the OLT, which then establishes the uplink ACM upon receiving the signal. Through the corresponding ACM, a training base can create a suitable autoencoder model. Ultimately, corresponding TANNs and RANNs can be deployed in intelligent transmitters / receivers. These intelligent transmitters / receivers can achieve optimized signal generation from bit to waveform level and decoding from symbol waveform to bit level, realizing end-to-end optimized channel equalization. The TANN can perform constellation shaping, geometric shaping, and channel pre-equalization operations, while the RANN can perform post-equalization and decoding techniques.

[0048] The multidimensional shaping combined coherent end-to-end equalization system proposed in this invention has the following structure: Figure 1 As shown, it includes a probability shaping module 101, a coherent transmission demodulation module 102, an ACM training module 103, an autoencoder pre-training module 104, an end-to-end training module 105, and an autoencoder signal generation, transmission, and demodulation module 106; wherein:

[0049] The probability shaping module 101 estimates the transmission conditions of the channel based on the transmitted test signal, and then performs probability shaping encoding on the signal strength to obtain the encoded bit stream.

[0050] The coherent transmission demodulation module 102 modulates the signal onto the optical carrier through a coherent modulator. After transmission through optical fiber, the signal is received by a coherent receiver, converted into an electrical signal, and processed by the back-end data processing to finally obtain a bit data stream.

[0051] The ACM training module 103 trains the ACM channel model based on the transmitted complex symbols and the complex symbols obtained after post-equalization.

[0052] The autoencoder pre-training module 104 trains a constellation point encoder that has undergone probability shaping according to the principle of maximizing the minimum spacing between symbols, and then trains a corresponding decoder after transmitting it through an AWGN channel.

[0053] The end-to-end training module 105 puts the pre-trained autoencoder model into the ACM channel for end-to-end optimization to obtain an autoencoder for a specific ACM channel.

[0054] The autoencoder signal generation, transmission and demodulation module 106 generates a series of coherent transmission signals from the TANN part of the trained autoencoder. The signals are modulated onto an optical carrier by a modulator and transmitted. After transmission through an actual optical fiber channel, the signals are coherently demodulated at the receiving end to recover complex symbols. The RANN part of the autoencoder then equalizes and decodes the signals into bits.

[0055] The specific steps for implementation are as follows:

[0056] Step 1: Use the probability shaping module to change the constellation distribution to a Maxwell-Boltzmann distribution to adjust the signal's emission entropy:

[0057]

[0058] Here, v is the shaping factor that controls the entropy. When v = 0, the constellation distribution of the transmitted signal is uniform; the larger the value, the smaller the transmission entropy. For the same QAM order, a specified value of v corresponds to a specific transmission entropy, as shown in the formula:

[0059]

[0060] During operation, a series of uniformly distributed 32QAM signals will first be transmitted as test signals, and the generalized mutual information (GMI) of the received signals will be obtained through channel estimation and calculation.

[0061] GMI can be used to reflect the performance of PS signal after transmission, and it can be expressed as:

[0062]

[0063] Among them, b k,i Let represent the i-th transmitted bit of the k-th symbol, X represent the symbol set, and X, Y represent the received symbol set and YY. It defines the average number of information bits per symbol. Normalized GMI (NGMI) can accurately predict the bit error rate before forward error correction. The formula for calculating NGMI is:

[0064] NGMI = 1 - (H - GMI) / m

[0065] Next, the emission entropy value is continuously adjusted iteratively. Specifically, the v value in probabilistic shaping is increased to improve the NGMI. After iteration, the NGMI value is modulated to a level slightly smaller than the threshold, and the v value at this point is recorded. The signal is then probabilistically shaped according to this value. In this example, the threshold can be set to NGMI = 0.82 for the test signal, and the v value at this point is recorded. If the value is already above 0.82 before adjustment, no further adjustment of the v value is necessary.

[0066] Step Two: Obtain the transmitted signal through a coherent transmission demodulation module. First, the signal passes through a symbol mapping module, which merges the probability-shaped bitstream from the previous step into a complex signal using a mapping relationship. Finally, a coherent transmitter modulates the symbol-mapped symbols onto an optical carrier. After transmission through the fiber optic channel, the received optical signal is received by a coherent receiver, converted into an electrical signal, and undergoes coherent demodulation, including downsampling, clock recovery, polarization recovery, and carrier recovery. Then, after being restored to two complex signals, it undergoes post-equalization, and finally, decoding yields the received binary data stream. In this embodiment, the transmission fiber length is 50km, and the signal before input to the fiber is... Figure 2 The training transmission signal shown is the signal received from the optical fiber and then demodulated. Figure 2 The training received signal is shown in the figure.

[0067] Step 3: Using the ACM training module, an ACM model of the real optical fiber channel is obtained by training based on the transmitted complex symbols and the complex symbols obtained after post-equalization. The transmitted complex symbols are used as input and converted into high-dimensional feature representations using an artificial neural network encoding process. Simultaneously, the post-equalized complex symbols are used as the target output, and the model is optimized by comparing the differences between the input and output symbols. Through repeated training and adjustments, this module can learn and optimize the ACM channel model, accurately predicting and simulating influencing factors such as nonlinear distortion, inter-symbol interference, and noise in optical fiber communication. In this embodiment, the ACM model consists of a fully connected deep neural network with two hidden layers. The input layer consists of 11 adjacent complex signals, and the output layer consists of a single complex signal. The activation functions of both the hidden and output layers are ReLU functions. The trained ACM model can effectively simulate the performance of a real channel. The reception results of a signal through a real optical fiber channel and an ACM channel are shown below. Figure 2 The arrows below indicate the actual channel and the ACM channel.

[0068] Step 4: Train the initial states of the TANN and RANN in the autoencoder using the autoencoder pre-training module. Training is performed based on the principle of maximizing the minimum spacing between symbols. First, the input symbols are mapped to a high-dimensional feature space using a probabilistically shaped constellation point encoder, and the encoder parameters are adjusted to maximize the minimum spacing between different symbols. Then, the encoded symbols are transmitted through an AWGN channel, introducing channel interference and noise. Next, the received symbols are decoded using a decoder, and the decoder parameters are adjusted using a backpropagation algorithm so that the decoder can accurately reconstruct the original symbols. Through multiple iterations of training, the autoencoder pre-training module can learn a compact representation of symbols and anti-interference capabilities, improving signal transmission quality and decoding accuracy. In this embodiment, the actual utility of the pre-training module is as follows: Figure 3As shown in (a), the loss function exhibits a rapid decreasing trend after pre-training. The initial transmit signal constellation diagram and the transmit signal constellation diagram after 20 iterations are shown in the inset. It can be seen that the autoencoder is now able to quickly learn the channel information and adjust to it.

[0069] Step 5: Train the autoencoder using an end-to-end training module. The pre-trained autoencoder model encodes the input symbols into high-dimensional feature representations. Then, the encoded symbols are transmitted through an ACM channel, which simulates interference and distortion effects in a real communication environment. Next, an end-to-end optimization algorithm adjusts the parameters of the autoencoder model based on the received symbols and the desired output to minimize transmission signal distortion and error. Through multiple iterative optimizations, an optimized autoencoder model for a specific ACM channel is obtained, better adapting to the channel's characteristics and improving signal transmission quality and decoding accuracy. This method effectively utilizes the characteristics of the autoencoder to achieve adaptive optimization for the ACM channel. In this embodiment, the slicing process of the training process is as follows: Figure 2 The transmitted and received signals are shown in the diagram. The signal generated by the autoencoder TANN is the transmitted signal, while the signal transmitted through the ACM channel is the received signal. The E2E optimizer performs backpropagation based on the cross-entropy loss function of the received signal, ultimately optimizing both TANN and RANN simultaneously.

[0070] Step Six: The signal is self-encoded, transmitted, and self-decoded using an autoencoder signal generation, transmission, and demodulation module. The TANN part of the trained autoencoder model generates a coherent transmit signal. These signals, after modulation by a modulator, are transmitted onto an optical carrier on the optical fiber. After transmission through the actual optical fiber channel, the coherent demodulator at the receiving end recovers the signal as complex symbols. Then, the RANN part of the autoencoder model equalizes the received signal and decodes it into the corresponding bit sequence. This process enables the recovery and decoding of the transmitted signal, thus achieving reliable data transmission. In a specific embodiment between the OLT and ONU... Figure 3 (b) Demonstrates the performance of 32-QAM with autoencoder assistance at different received optical power (ROP). Compared to using only the receiver post-equalizer, the autoencoder improves the sensitivity of the 32-QAM signal by 2.9 dB. Figure 3 (c) shows the net data rate (NDR) gain of the autoencoder-assisted transmission signal as optical path loss (OPL) increases. Figure 3(d) shows the 32-QAM constellation diagrams using conventional Gray coding (i-iii) and E2E-optimized autoencoders (iv-vi) under different probability shaping factors.

[0071] Figure 4 This document demonstrates a specific implementation scheme for efficiently establishing a smart transceiver system when a new user comes online. In the downlink, the smart transceiver system is established through three steps. First, the OLT transmits a test signal. Then, the ONU receives the signal and establishes the ACM for the downlink channel. The ACM is sent back to the OLT. A training center is deployed at the OLT, allowing the OLT to obtain the TANN and RANN trained for the downlink channel ACM. Finally, the TANN is deployed to the OLT's transmitter, and the RANN is sent from the OLT to the ONU for deployment. In the uplink, the ONU first transmits a test signal. The OLT receives the signal and generates the ACM. After obtaining the trained network from the training center, the OLT deploys the RANN and sends the TANN to the ONU for deployment. Signal transmission supported by the TANN and RANN can then proceed. As shown in the diagram, this process requires three signal transmission steps between the OLT and ONU in the downlink and two signal transmission steps in the uplink. Because smart transceivers can be established simultaneously in both the uplink and downlink, the entire process requires only three signal transmissions.

[0072] This concludes the implementation method of a multidimensional shaping combined coherent end-to-end equalization technology.

[0073] In summary, the multidimensional shaping combined coherent end-to-end equalization system proposed in this invention can improve signal sensitivity and dynamic range, enhance system performance, and achieve higher net data rates, thereby increasing channel capacity, improving signal quality, extending transmission distance, and reducing costs.

Claims

1. A coherent end-to-end equalization system with multidimensional shaping, characterized in that, By integrating probabilistic shaping, geometric shaping, pre-equalization, post-equalization, and decoding technologies in coherent transmission systems, and by uniformly allocating and jointly optimizing them according to the actual channel, the aim is to achieve the best channel capacity. Through the E2E mechanism, the Optical Line Terminal (OLT) and the Optical Network Unit (ONU) collaborate closely. When a new user comes online, the OLT sends an estimated signal to the ONU and establishes a downlink neural network channel model (ACM) based on an artificial neural network. The downlink ACM simulates the channel and transmits the uplink channel estimation signal together to the OLT. After receiving the signal, the OLT establishes the uplink ACM. Through the corresponding ACM, a suitable autoencoder model is created in the training environment. Finally, the corresponding transmitter neural network (TANN) and receiver neural network (RANN) are deployed in the smart transmitter / receiver. These intelligent transmitters / receivers achieve optimized signal generation from bit to waveform level and decoding from symbol waveform to bit level, realizing end-to-end optimized channel equalization; among them, TANN can perform constellation shaping, geometry shaping and channel pre-equalization operations, while RANN can perform post-equalization and decoding. The system specifically includes a probability shaping module, a coherent transmission demodulation module, an ACM training module, an autoencoder pre-training module, an end-to-end training module, and an autoencoder signal generation, transmission, and demodulation module; among which: The probability shaping module estimates the channel transmission conditions based on the transmitted test signal, and then performs probability shaping encoding on the signal strength to obtain the encoded bit stream. The coherent transmission demodulation module modulates the signal processed by the probability shaping module onto the optical carrier through a coherent modulator. After transmission through optical fiber, the signal is received by a coherent receiver, converted into an electrical signal, and then processed in the backend to finally obtain a bit data stream. The ACM training module uses the transmitted complex symbols and the complex symbols obtained after post-equalization as the teacher set and training set of the neural network, respectively, and obtains the ACM channel model through training. The autoencoder pre-training module trains a constellation point encoder that has undergone probability shaping according to the principle of maximizing the minimum spacing between symbols, and inputs it into an AWGN channel. Then, it uses the output sequence of the AWGN channel to train a corresponding decoder. The end-to-end training module puts the pre-trained autoencoder model into the ACM channel for end-to-end optimization to obtain an autoencoder for a specific ACM channel; The autoencoder signal generation, transmission and demodulation module generates a series of coherent transmission signals from the TANN part of the trained autoencoder. The signals are modulated onto an optical carrier by a modulator and transmitted. After transmission through an actual optical fiber channel, the signals are coherently demodulated at the receiving end to recover complex symbols. The RANN part of the autoencoder then equalizes and decodes the signals into bits.

2. The coherent end-to-end equalization system according to claim 1, characterized in that, In the probability shaping module, the emission entropy of the signal is adjusted by changing the constellation distribution to a Maxwell-Boltzmann distribution. ， （1） Where v is the shaping factor controlling the entropy, distributed between 0 and 1, x is a single symbol, and X is the set of all symbols; for the same QAM order, a specified value of v corresponds to a specific emission entropy, as shown in the formula: ， （2） During operation, a uniformly distributed 32QAM signal is first transmitted as a test signal. The generalized mutual information (GMI) of the received signal is then obtained through channel estimation and calculation, expressed as: ， （3） Among them, b k,i Let X represent the transmitted symbol set and Y represent the received symbol set; it defines the average number of information bits per symbol, and N and m represent the number of symbols and bits, respectively; GMI is used to reflect the performance of the PS signal after transmission; the normalized generalized mutual information (NGMI) accurately predicts the bit error rate before forward error correction, and the formula for calculating NGMI is: ， （4） By continuously adjusting the specific emission entropy value, the specific operation steps are to increase the v value in the probability shaping to improve NGMI. After iteration, the NGMI value is modulated to a level slightly larger than the threshold, the v value at this time is recorded, and the signal is probabilistically shaped according to this value.

3. The coherent end-to-end equalization system according to claim 2, characterized in that, In the coherent transmission demodulation module, the signal first passes through a symbol mapping module, which merges the bit stream that underwent probability shaping in the previous step into a complex signal through a mapping relationship. Finally, a coherent transmitter modulates the symbol-mapped signal onto an optical carrier. After transmission through the optical fiber channel, the received optical signal is received by a coherent receiver, converted into an electrical signal, and then undergoes coherent demodulation at the back end, including downsampling, clock recovery, polarization recovery, and carrier recovery. After being restored to two complex signals, it undergoes a post-equalization process, and finally, the received binary data stream is obtained through decoding.

4. The coherent end-to-end equalization system according to claim 3, characterized in that, In the ACM training module, training is performed based on the transmitted complex symbols and the complex symbols obtained after post-equalization. By taking the transmitted complex symbols as input, an artificial neural network encoding process is used to convert them into high-dimensional feature representations. At the same time, the post-equalized complex symbols are used as the target output. The model is optimized by comparing the differences between the input and output symbols. Through repeated training and adjustment, an optimized ACM channel model is learned. This ACM channel model is used to accurately predict and simulate the nonlinear distortion, inter-symbol interference, and noise factors in optical fiber communication.

5. The coherent end-to-end equalization system according to claim 4, characterized in that, In the autoencoder pre-training module, training is performed based on the principle of maximizing the minimum spacing between symbols. The specific process is as follows: First, the input symbols are mapped to a high-dimensional feature space using a probabilistically shaped constellation encoder, and the encoder parameters are adjusted to maximize the minimum spacing between different symbols. Then, the encoded symbols are transmitted through an AWGN channel, introducing channel interference and noise. Next, the received symbols are decoded using a decoder, and the decoder parameters are adjusted using a backpropagation algorithm to ensure the decoder accurately reconstructs the original symbols. Through multiple iterative training iterations, the autoencoder pre-training module learns the compact representation of symbols and its anti-interference capability, improving signal transmission quality and decoding accuracy.

6. The coherent end-to-end equalization system according to claim 5, characterized in that, In the end-to-end training module, the input symbols are encoded into high-dimensional feature representations using a pre-trained autoencoder model. Then, the encoded symbols are transmitted through an ACM channel, which simulates interference and distortion effects in a real communication environment. Next, an end-to-end optimization algorithm adjusts the parameters of the autoencoder model based on the received symbols and the desired output to minimize signal distortion and error. Through multiple iterative optimizations, an optimized autoencoder model for a specific ACM channel is obtained, which better adapts to the characteristics of the channel and improves signal transmission quality and decoding accuracy.

7. The coherent end-to-end equalization system according to claim 6, characterized in that, In the autoencoder signal generation, transmission and demodulation module, a series of coherent transmission signals are generated through the TANN part of the trained autoencoder model. These coherent transmitted signals are modulated by a modulator and then transmitted onto an optical carrier on an optical fiber. After the signal is transmitted through the actual optical fiber channel, the coherent demodulator at the receiving end recovers it into complex symbols. Then, the received signal is equalized by the RANN part of the autoencoder model and decoded into the corresponding bit sequence. Through this process, the transmission signal is recovered and decoded, thereby achieving reliable data transmission.

8. The coherent end-to-end equalization system according to claim 7, characterized in that, In realizing the recovery and decoding of transmitted signals, a framework capable of efficiently establishing intelligent transceivers when new users come online is adopted, specifically: In the downlink, the intelligent transceiver system is established through three steps: First, the OLT transmits a test signal; then, the ONU receives the signal and establishes the ACM for the downlink channel; the ACM is sent back to the OLT; finally, the OLT obtains the trained network through the ACM, deploys the TANN to the OLT's transmitter, and sends the RANN to the ONU for deployment. In the uplink, the ONU first transmits a test signal; the OLT receives the signal and generates an ACM. After obtaining the trained network, the end OLT deploys the RANN and sends the TANN to the ONU for deployment; then the TANN and RANN-supported signal transmission are performed.

9. The coherent end-to-end equalization system according to claim 8, characterized in that, The system workflow is as follows: (1) The signal is probabilistically shaped and encoded using a probabilistic shaping module; (2) The signal is modulated onto an optical carrier, transmitted through an optical fiber, demodulated to recover an electrical signal, and then processed to obtain a bit data stream; (3) Obtain the ACM channel model through the training module; (4) Train the encoder and the corresponding decoder after probability shaping; (5) The pre-trained autoencoder model is put into the ACM channel for optimization to obtain the autoencoder for the specific ACM channel; (6) The transmitted signal is generated by the TANN part of the autoencoder, and after transmission and demodulation, it is equalized and decoded into bits by the RANN part.

Images