Data-driven end-to-end modeling method and system for optical fiber communication experimental system

Abstract

The present invention provides a data-driven end-to-end modeling method and system for an optical fiber communication experimental system, comprising: selectively compensating for optical fiber communication experimental system characteristics based on a digital signal processing algorithm (DSP) at the transceiver end, collecting input data and compensated output data to construct a training data set; constructing a deterministic feature model and establishing an input layer data structure corresponding to a neural network; constructing a random feature model and establishing an input layer data structure corresponding to the neural network; constructing a loss function, sequentially training the deterministic feature model and the random feature model in two stages; summing the output results of the deterministic feature model and the random feature model to obtain a complete system modeling output. The present invention combines DSP design to directly model the experimental system, does not rely on theoretical models, does not require assumptions and system simplification, and avoids iterative operations, thereby achieving rapid and accurate modeling of optical fiber communication experimental systems.

Description

Technical Field

[0001] The present invention relates to the fields of optical fiber communication system modeling, optical fiber channel modeling and neural network application technology, and in particular to a data-driven end-to-end modeling method and system for an optical fiber communication experimental system. Background Art

[0002] Fiber-optic communication system modeling is crucial for communication system design and performance prediction. Traditional modeling methods primarily focus on modeling fiber channels, including the split-step Fourier transform method and the Gaussian noise model. The former, used to represent signal transmission in optical fibers, is computationally complex and has limited applicability. The latter, using a Gaussian assumption, only captures the statistical representation of the channel, failing to accurately characterize signal characteristics and failing to account for the impact of the transmitting and receiving DSPs on the system. Both methods derive from theoretical models, employing approximations and assumptions that can lead to significant discrepancies with experimental data.

[0003] Patent document CN114499723A (application number: CN202210083438.5) discloses a method for rapid modeling of optical fiber channels based on a Fourier neural operator. Specifically, the method comprises the following steps: first, a long-distance optical signal transmission simulation system is established based on the traditional split-step Fourier algorithm to obtain training pairs consisting of input signals and output signals of each span; then, a Fourier neural operator is constructed using a neural network; and finally, the weight parameters of the Fourier neural operator are iteratively updated using the training data to complete the mapping approximation. However, the technical effects achieved by this patent are essentially different from those of the present invention. Summary of the Invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide a data-driven end-to-end modeling method and system for an optical fiber communication experimental system.

[0005] The data-driven end-to-end modeling method for an optical fiber communication experimental system provided by the present invention includes:

[0006] Step S1: selectively compensate the characteristics of the optical fiber communication experimental system based on the digital signal processing algorithm DSP at the transceiver end, collect input data and compensated output data to construct a training data set D;

[0007] Step S2: Construct a deterministic feature model F and establish the input layer data structure f of the corresponding neural network;

[0008] Step S3: Construct a random feature model G and establish the input layer data structure g of the corresponding neural network;

[0009] Step S4: Construct a loss function and train the deterministic feature model and the random feature model in two stages;

[0010] Step S5: summing the output results of the deterministic characteristic model and the random characteristic model to obtain a complete system characteristic model.

[0011] Preferably, the step S1 includes:

[0012] Step S1.1: Taking a specific modeling task as the target, the DSP algorithm is used to compensate for the system characteristics to obtain data that meets the modeling target characteristics;

[0013] Step S1.2: Set the acquired data to single or multiple sampling rates, flexibly select two nodes as the collection points for input and output data according to the modeling objectives, and construct the required data set D;

[0014] Step S1.3: Adjust the digital signal parameters of the transmitting and receiving ends and the parameters of the optical fiber communication experimental system to enrich the diversity of the data set and enable the model to have the ability to model multi-conditional features.

[0015] Preferably, the step S2 includes:

[0016] Step S2.1: Construct a deterministic feature model F, including a bidirectional long short-term memory network and a forward fully connected network, to extract deterministic features;

[0017] Step S2.2: Establish the input layer data structure f of the corresponding neural network, including input data and control data. For the modeling of the i-th output symbol, the i-th input symbol and the N adjacent input symbols before and after form a vector x, which constitutes part of the input data structure. The number N of adjacent symbols is related to the memory length of the data features; the control data includes distance, power, and symbol probability, which is another part of the input data, and together with the vector x, constitutes a complete input layer data structure for determining the feature model.

[0018] Preferably, step S3 includes:

[0019] Step S3.1: Construct a random feature model G, including a generative adversarial network and a variational autoencoder network, to extract random features;

[0020] Step S3.2: Establish the input layer data structure g of the corresponding neural network, including hidden codes and control data. The hidden codes are Gaussian random sampling, which is a necessary input for generating adversarial models or variational autoencoding models; the control data includes distance, power, symbol probability, and signal-to-noise ratio. If the distribution of random features is related to the input, the input data is added.

[0021] Preferably, the step S4 includes:

[0022] Step S4.1: Construct a training loss function for the feature determination model, including the mean square error and absolute value error between the model output and the system output data, and train the parameters of the feature determination model to minimize the loss function;

[0023] Step S4.2: After the feature model is trained, a random feature dataset is constructed using the difference between the original dataset D and the output F(f) of the feature model.

[0024] Step S4.3: Construct a training loss function for the random feature model, including the cross entropy and the original loss function of the generative adversarial network, and train the parameters of the random feature model so that the distribution of the random features generated by the model is consistent with the distribution characteristics of the random feature dataset;

[0025] Step S4.4: Sum the output results of the deterministic feature model and the random feature model to obtain a complete modeling model that is aligned with the experimental system characteristics.

[0026] The data-driven end-to-end modeling system for optical fiber communication experimental systems provided by the present invention includes:

[0027] Module M1: Based on the digital signal processing algorithm DSP at the transceiver end, the characteristics of the optical fiber communication experimental system are selectively compensated, and the input data and compensated output data are collected to construct the training data set D;

[0028] Module M2: Construct a deterministic feature model F and establish the input layer data structure f of the corresponding neural network;

[0029] Module M3: Construct random feature model G and establish the input layer data structure g of the corresponding neural network;

[0030] Module M4: Construct the loss function and train the deterministic feature model and the random feature model in two stages;

[0031] Module M5: Sum the output results of the deterministic characteristic model and the random characteristic model to obtain the complete system characteristic model.

[0032] Preferably, the module M1 includes:

[0033] Module M1.1: Targeting a specific modeling task, use DSP algorithms to compensate for system characteristics and obtain data that meets the modeling target characteristics;

[0034] Module M1.2: Set the acquired data to single or multiple sampling rates, flexibly select two nodes as the collection points for input and output data according to the modeling objectives, and construct the required data set D;

[0035] Module M1.3: Adjust the digital signal parameters of the transmitting and receiving ends and the parameters of the optical fiber communication experimental system to enrich the diversity of the data set and enable the model to have the ability to model multi-conditional features.

[0036] Preferably, the module M2 includes:

[0037] Module M2.1: Construct a deterministic feature model F, including a bidirectional long short-term memory network and a forward fully connected network, to extract deterministic features;

[0038] Module M2.2: Establish the input layer data structure f of the corresponding neural network, including input data and control data. For the modeling of the i-th output symbol, the i-th input symbol and the N adjacent input symbols before and after it form a vector x, which constitutes part of the input data structure. The number N of adjacent symbols is related to the memory length of the data features; the control data includes distance, power, and symbol probability, which is another part of the input data, and together with the vector x, constitutes a complete input layer data structure for determining the feature model.

[0039] Preferably, the module M3 includes:

[0040] Module M3.1: Construct a random feature model G, including a generative adversarial network and a variational autoencoder network, to extract random features;

[0041] Module M3.2: Establish the input layer data structure g of the corresponding neural network, including hidden codes and control data. The hidden codes are Gaussian random sampling, which is a necessary input for generating adversarial models or variational autoencoding models; the control data includes distance, power, symbol probability, and signal-to-noise ratio. If the distribution of random features is related to the input, then add the input data.

[0042] Preferably, the module M4 includes:

[0043] Module M4.1: Construct a training loss function for the deterministic feature model, including the mean square error and absolute error between the model output and the system output data, and train the parameters of the deterministic feature model to minimize the loss function.

[0044] Module M4.2: After the deterministic feature model is trained, a random feature dataset is constructed using the difference between the original dataset D and the output F(f) of the deterministic feature model.

[0045] Module M4.3: Construct training loss functions for random feature models, including cross entropy and the original loss functions of generative adversarial networks, and train the parameters of random feature models so that the distribution of random features generated by the model is consistent with the distribution characteristics of the random feature dataset;

[0046] Module M4.4: Sum the outputs of the deterministic feature model and the stochastic feature model to obtain a complete model that aligns with the experimental system characteristics.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention combines DSP design to directly model experimental data, does not rely on theoretical models, does not require assumptions and system simplifications, and avoids iterative calculations. The model architecture is simple, and can achieve fast and accurate modeling of optical fiber communication experimental systems. The Q factor error is within 0.1dB and the running time is extremely short. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0050] Figure 1 This is an illustration of a data-driven end-to-end fast and accurate modeling method for an optical fiber communication experimental system according to an embodiment of the present invention;

[0051] Figure 2 This is a dual-model training flow chart in a data-driven end-to-end fast and accurate modeling method for an optical fiber communication experimental system according to an embodiment of the present invention;

[0052] Figure 3 This is a result diagram aligned with the experiment of a data-driven end-to-end fast and accurate modeling method for an optical fiber communication experimental system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0054] Example 1:

[0055] like Figure 1-Figure 3 As shown, the present invention proposes a data-driven end-to-end fast and accurate modeling method for an optical fiber communication experimental system, taking the nonlinear and noise feature modeling task as an example, including the following steps:

[0056] Step S1: Combining the digital signal processing algorithm (DSP) at the transmitting and receiving ends to selectively compensate for the system characteristics, collecting input data and compensated output data to construct a training data set D;

[0057] Step S2: Design a deterministic feature model F and design the input layer data structure f of the neural network F;

[0058] Step S3: Design a random feature model G and design the input layer data structure g of the neural network G;

[0059] Step S4: Design a loss function and train the deterministic feature model F and the random feature model G in two stages.

[0060] Step S5: The output results of the deterministic characteristic model F and the random characteristic model G are summed to obtain the complete system characteristic model.

[0061] Step S1 includes: Targeting the nonlinear and noise feature modeling task, using DSP to compensate for the system's linear characteristics, obtaining data that meets the target modeling characteristics. This data can be set at a single sampling rate to construct the required data set. Based on this, the transmit power is adjusted from 1.0dBm to 7.0dBm to construct a multi-power data set, enabling the model to generalize across multiple power levels.

[0062] The step S2 comprises:

[0063] Step S2.1: Design a deterministic feature model F and use a bidirectional long short-term memory network to extract deterministic features;

[0064] Step S2.2: Design the input layer data structure f of the neural network F. The deterministic feature model structure consists of two parts: input data and control data. For a 40-channel, 2000 km transmission scenario, 51 input symbols are selected to form a vector x, which, together with the conditional control information power, forms the complete input layer data structure f of the deterministic feature model.

[0065] The step S3 comprises:

[0066] Step S3.1: Design a random feature model G and use a variational autoencoder network to extract random features;

[0067] Step S3.2: Design the input layer data structure g of the neural network G. The random feature model structure consists of two parts: the hidden code and the control data. The hidden code is a Gaussian random sample with a dimension of 10, and the control data is the power value.

[0068] The step S4 comprises:

[0069] Step S4.1: Design a training loss function for the feature model to be the mean square error between the model output and the system output data, and train the parameters of the feature model F to minimize the loss function.

[0070] Step S4.2: After determining that the feature model F is trained, construct a random feature dataset D1 using the difference between the original dataset D and the output of the F model, i.e., DF(f);

[0071] Step S4.3: Design the training loss function of the random feature model G, select the original loss function of the generative adversarial network to train the parameters of the random feature model G, obtain the mean and variance of the hidden layer of the random feature model, and then sample the random distribution data of the Gaussian distribution so that the distribution of the random features generated by the model is the same as the distribution characteristics between the random feature data set D1.

[0072] The step S5 includes, after obtaining the deterministic feature model F and the random feature model G, summing the output results of the two models F(f)+G(g) to obtain a complete modeling model aligned with the experimental system characteristics.

[0073] This example also presents generalized modeling results for different power levels using this system modeling method. The system Q factor and signal-to-noise ratio results are essentially aligned with the experimental system, with an error of less than 0.1 dB. While maintaining accuracy, the time required is significantly less than with traditional modeling methods.

[0074] Example 2:

[0075] The present invention also provides a data-driven end-to-end rapid and accurate modeling system for an optical fiber communication experimental system. The data-driven end-to-end rapid and accurate modeling system for an optical fiber communication experimental system can be implemented by executing the process steps of the data-driven end-to-end rapid and accurate modeling method for an optical fiber communication experimental system. That is, those skilled in the art can understand the data-driven end-to-end rapid and accurate modeling method for an optical fiber communication experimental system as a preferred embodiment of the data-driven end-to-end rapid and accurate modeling system for an optical fiber communication experimental system. The system specifically includes the following modules:

[0076] Module M1: Based on the digital signal processing algorithm DSP at the transceiver end, the characteristics of the optical fiber communication experimental system are selectively compensated, and the input data and compensated output data are collected to construct a training data set; Module M2: Construct a deterministic feature model and establish the input layer data structure of the corresponding neural network; Module M3: Construct a random feature model and establish the input layer data structure of the corresponding neural network; Module M4: Construct a loss function and train the deterministic feature model and the random feature model in two stages; Module M5: Summing the output results of the deterministic feature model and the random feature model to obtain the complete system feature model.

[0077] The module M1 includes: module M1.1: with a specific modeling task as the goal, using the DSP algorithm to compensate for the system characteristics to obtain data that meets the modeling target characteristics; module M1.2: setting the obtained data to data at a single or multiple sampling rate, flexibly selecting two nodes as the collection points for input and output data according to the modeling target, and constructing the required data set; module M1.3: adjusting the digital signal parameters of the transmitting and receiving ends and the parameters of the optical fiber communication experimental system to enrich the diversity of the data set, so that the model has the ability to model multi-conditional characteristics.

[0078] The module M2 includes: module M2.1: constructing a deterministic feature model, including a bidirectional long short-term memory network and a forward fully connected network, to extract deterministic features; module M2.2: establishing the input layer data structure of the corresponding neural network, including input data and control data. For the modeling of the i-th output symbol, the i-th input symbol and the N adjacent input symbols before and after form a vector x, which constitutes a part of the input data structure. The number N of adjacent symbols is related to the memory length of the data feature; the control data includes distance, power, and symbol probability, which is another part of the input data, and together with the vector x, constitutes the input layer data structure of a complete deterministic feature model.

[0079] The module M3 includes: Module M3.1: constructing a random feature model, including a generative adversarial network and a variational autoencoding network, to extract random features; Module M3.2: establishing the input layer data structure of the corresponding neural network, including hidden codes and control data. The hidden codes are Gaussian random sampling and are necessary inputs for generating adversarial models or variational autoencoding models; the control data include distance, power, symbol probability, and signal-to-noise ratio. If the distribution of random features is related to the input, the input data is added.

[0080] The module M4 includes: module M4.1: constructing a training loss function for a deterministic feature model, including the mean square error and absolute value error between the model output and the system output data, and training the parameters of the deterministic feature model to minimize the loss function; module M4.2: after the deterministic feature model is trained, constructing a random feature data set through the difference between the original data set and the output of the deterministic feature model; module M4.3: constructing a training loss function for the random feature model, including the original loss function of the cross entropy and the generative adversarial network, and training the parameters of the random feature model to make the distribution of the random features generated by the model the same as the distribution characteristics between the random feature data sets; module M4.4: summing the output results of the deterministic feature model and the random feature model to obtain a complete modeling model aligned with the characteristics of the experimental system.

[0081] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0082] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A data-driven end-to-end modeling method for an optical fiber communication experimental system, characterized in that: include: Step S1: selectively compensate the characteristics of the optical fiber communication experimental system based on the digital signal processing algorithm DSP at the transmitting and receiving end, collect input data and compensated output data to construct a training data set D; Step S2: Construct a deterministic feature model F and establish the input layer data structure f of the corresponding neural network; Step S3: Construct a random feature model G and establish the input layer data structure g of the corresponding neural network; Step S4: Construct a loss function and train the deterministic feature model and the random feature model in two stages; Step S5: summing the output results of the deterministic characteristic model and the random characteristic model to obtain a complete system characteristic model; The step S2 comprises: Step S2.1: Construct a deterministic feature model F, including a bidirectional long short-term memory network and a forward fully connected network, to extract deterministic features; Step S2.2: Establish the input layer data structure f of the corresponding neural network, including input data and control data. For the modeling of the i-th output symbol, the i-th input symbol and the N adjacent input symbols before and after it form a vector x, which constitutes part of the input data structure. The number N of adjacent symbols is related to the memory length of the data features. The control data, including distance, power, and symbol probability, is another part of the input data. Together with the vector x, they form the complete input layer data structure for the determined feature model. The step S3 comprises: Step S3.1: Construct a random feature model G, including a generative adversarial network and a variational autoencoder network, to extract random features; Step S3.2: Establish the input layer data structure g of the corresponding neural network, including hidden codes and control data. The hidden codes are Gaussian random sampling, which is a necessary input for generating adversarial models or variational autoencoding models; the control data includes distance, power, symbol probability, and signal-to-noise ratio. If the distribution of random features is related to the input, the input data is added.

2. The data-driven end-to-end modeling method for an optical fiber communication experimental system according to claim 1, characterized in that: The step S1 comprises: Step S1.1: Taking a specific modeling task as the target, the DSP algorithm is used to compensate for the system characteristics to obtain data that meets the modeling target characteristics; Step S1.2: Set the acquired data to single or multiple sampling rates, flexibly select two nodes as the collection points for input and output data according to the modeling objectives, and construct the required data set D; Step S1.3: Adjust the digital signal parameters of the transmitting and receiving ends and the parameters of the optical fiber communication experimental system to enrich the diversity of the data set and enable the model to have the ability to model multi-conditional features.

3. The data-driven end-to-end modeling method for optical fiber communication experimental system according to claim 1 is characterized in that: The step S4 comprises: Step S4.1: Construct a training loss function for the feature determination model, including the mean square error and absolute value error between the model output and the system output data, and train the parameters of the feature determination model to minimize the loss function; Step S4.2: After the feature model is trained, a random feature dataset is constructed using the difference between the original dataset D and the output F(f) of the feature model. Step S4.3: Construct a training loss function for the random feature model, including the cross entropy and the original loss function of the generative adversarial network, and train the parameters of the random feature model so that the distribution of the random features generated by the model is consistent with the distribution characteristics of the random feature dataset; Step S4.4: Sum the output results of the deterministic feature model and the random feature model to obtain a complete model that is aligned with the experimental system characteristics.

4. A data-driven end-to-end modeling system for optical fiber communication experimental systems, characterized in that: include: Module M1: Based on the digital signal processing algorithm DSP at the transceiver end, the characteristics of the optical fiber communication experimental system are selectively compensated, and the input data and compensated output data are collected to construct the training data set D; Module M2: Construct a deterministic feature model F and establish the input layer data structure f of the corresponding neural network; Module M3: Construct random feature model G and establish the input layer data structure g of the corresponding neural network; Module M4: Construct the loss function and train the deterministic feature model and the random feature model in two stages; Module M5: Sum the output results of the deterministic characteristic model and the random characteristic model to obtain the complete system characteristic model; The module M2 includes: Module M2.1: Construct a deterministic feature model F, including a bidirectional long short-term memory network and a forward fully connected network, to extract deterministic features; Module M2.2: Establish the input layer data structure f of the corresponding neural network, including input data and control data. For the modeling of the i-th output symbol, the i-th input symbol and the N adjacent input symbols before and after it form a vector x, which constitutes part of the input data structure. The number of adjacent symbols N is related to the memory length of the data features. The control data, including distance, power, and symbol probability, is another part of the input data. Together with vector x, they form the complete input layer data structure of the deterministic feature model. The module M3 includes: Module M3.1: Construct a random feature model G, including a generative adversarial network and a variational autoencoder network, to extract random features; Module M3.2: Establish the input layer data structure g of the corresponding neural network, including hidden codes and control data. The hidden codes are Gaussian random sampling, which is a necessary input for generating adversarial models or variational autoencoding models; the control data includes distance, power, symbol probability, and signal-to-noise ratio. If the distribution of random features is related to the input, then add the input data.

5. The data-driven end-to-end modeling system for optical fiber communication experimental system according to claim 4 is characterized in that: The module M1 includes: Module M1.1: Targeting a specific modeling task, use DSP algorithms to compensate for system characteristics and obtain data that meets the modeling target characteristics; Module M1.2: Set the acquired data to single or multiple sampling rates, flexibly select two nodes as the collection points for input and output data according to the modeling objectives, and construct the required data set D; Module M1.3: Adjust the digital signal parameters of the transmitting and receiving ends and the parameters of the optical fiber communication experimental system to enrich the diversity of the data set and enable the model to have the ability to model multi-conditional features.

6. The data-driven end-to-end modeling system for optical fiber communication experimental system according to claim 4 is characterized in that: The module M4 includes: Module M4.1: Construct a training loss function for the deterministic feature model, including the mean square error and absolute error between the model output and the system output data, and train the parameters of the deterministic feature model to minimize the loss function. Module M4.2: After the deterministic feature model is trained, a random feature dataset is constructed using the difference between the original dataset D and the output F(f) of the deterministic feature model. Module M4.3: Construct training loss functions for random feature models, including cross entropy and the original loss functions of generative adversarial networks, and train the parameters of random feature models so that the distribution of random features generated by the model is consistent with the distribution characteristics of the random feature dataset; Module M4.4: Sum the outputs of the deterministic feature model and the stochastic feature model to obtain a complete model that aligns with the experimental system characteristics.

Images