Optical cable monitoring and data analysis method, device, equipment and medium

Abstract

The invention relates to an optical cable monitoring and data analysis method and device, equipment and a medium. The method comprises the following steps: acquiring current state data of an optical fiber link, and dynamically generating adaptive pulse coding parameters and matched filter parameters based on the data by using a deep reinforcement learning agent, so as to generate and inject coded pulses into the optical fiber link; and then receiving an echo signal, performing matched filtering processing on the signal by using the generated matched filter parameter to obtain a filtering output signal, further calculating a performance index vector and a reward signal, and finally updating a strategy network of the intelligent agent by using the reward signal, thereby realizing continuous optimization of pulse coding and matched filtering parameters. Therefore, the system can automatically adapt to the dynamic change of an optical fiber link, the spatial resolution, the dynamic range and the signal-to-noise ratio are effectively improved, and the adaptive capability and the robustness of the system are enhanced at the same time.

Description

TECHNICAL FIELD

[0001] The application belongs to the cross technical field of optical communication and adaptive control, and particularly relates to an optical cable monitoring and data analysis method, device, equipment and medium. BACKGROUND

[0002] Optical cable monitoring and data analysis is a core link in modern communication network operation and maintenance, and its performance is directly related to network reliability and fault positioning accuracy. As the mainstream technology in this field, optical time domain reflectometry realizes link diagnosis by injecting optical pulses into optical fibers and analyzing their backscattered signals. Traditional OTDR systems generally use fixed waveform coded pulses and corresponding matched filters to balance the key indicators of spatial resolution and dynamic range through pulse compression technology. However, this static coding scheme faces fundamental challenges in actual engineering applications. Optical fiber links are in a continuously changing physical environment, and factors such as temperature fluctuations and mechanical stress disturbances will cause dynamic changes in the dispersion characteristics and attenuation coefficients of optical fibers, resulting in a mismatch between the pre-designed fixed coded pulses and the actual link response. This mismatch can significantly reduce the effectiveness of pulse compression, resulting in signal-to-noise ratio degradation, event blind zone enlargement, and spatial resolution reduction, ultimately affecting the accuracy and sensitivity of fault detection.

[0003] The industry has tried various ways to alleviate this problem, such as using a pre-defined library of multiple coding schemes and switching according to simple link indicators, or optimizing a limited set of pulse parameters through offline calculation. However, these improvements are still essentially static or semi-static adjustments and cannot achieve true real-time adaptation. They lack the ability to perceive subtle changes in link state and have no intelligence to continuously learn and optimize coding strategies based on historical monitoring data. When the link experiences rapid or unknown types of disturbances, these methods will still fail, causing the system performance to drop back to the traditional level. The fundamental reason is that they have failed to establish an intelligent decision-making mechanism that can optimize the closed-loop linkage of link state, transmitted pulse, and received signal processing.

[0004] Therefore, to break through this technical bottleneck, there is an urgent need for an optical cable monitoring method that can autonomously perceive, decide, and evolve. SUMMARY

[0005] Therefore, it is necessary to provide an optical cable monitoring and data analysis method, device, equipment and medium to solve the above technical problems.

[0006] In a first aspect, the application provides an optical cable monitoring and data analysis method, comprising:

[0007] S1. Collect the current state data of the fiber optic link. Based on the current state data, generate pulse coding parameters and matched filter parameters through a deep reinforcement learning agent. The current state data includes signal features extracted from the echo signal, environmental parameters provided by environmental sensors, and historical performance indicators. The pulse coding parameters are used to define the time-frequency characteristics of the pulse sequence.

[0008] S2. Generate coded pulses according to the pulse coding parameters and inject the coded pulses into the optical fiber link;

[0009] S3. Receive the echo signal from the optical fiber link, and use the matched filter parameters to perform matched filtering on the echo signal to obtain the filtered output signal.

[0010] S4. Calculate the performance index vector based on the filtered output signal, and calculate the reward signal based on the performance index vector.

[0011] S5. Update the policy network of the deep reinforcement learning agent using reward signals.

[0012] Secondly, this application also provides an optical cable monitoring and data analysis device for implementing the method described in the first aspect, the device comprising:

[0013] The intelligent state perception module is used to collect the current state data of the fiber optic link. Based on the current state data, it generates pulse coding parameters and matched filter parameters through a deep reinforcement learning agent. The current state data includes signal features extracted from the echo signal, environmental parameters provided by environmental sensors, and historical performance indicators. The pulse coding parameters are used to define the time-frequency characteristics of the pulse sequence.

[0014] The pulse injection execution module is used to generate coded pulses according to pulse coding parameters and inject the coded pulses into the optical fiber link;

[0015] The echo signal filtering module is used to receive echo signals from the fiber optic link, and to perform matched filtering on the echo signals using matched filter parameters to obtain the filtered output signal.

[0016] The performance evaluation and reward generation module is used to calculate the performance index vector based on the filtered output signal and the reward signal based on the performance index vector.

[0017] The policy adaptive optimization module is used to update the policy network of the deep reinforcement learning agent using reward signals.

[0018] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement an optical cable monitoring and data analysis method as described in the first aspect.

[0019] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an optical cable monitoring and data analysis method as described in the first aspect.

[0020] The aforementioned optical fiber monitoring and data analysis method, apparatus, equipment, and medium collect current status data of the optical fiber link and utilize a deep reinforcement learning agent to dynamically generate adaptive pulse coding parameters and matched filter parameters based on this data, thereby generating and injecting coded pulses into the optical fiber link. Subsequently, echo signals are received, and the generated matched filter parameters are used to perform matched filtering on the signals to obtain filtered output signals. Then, performance index vectors and reward signals are calculated, and finally, the reward signal is used to update the agent's policy network. This achieves continuous optimization of pulse coding and matched filtering parameters, enabling the system to autonomously adapt to the dynamic changes of the optical fiber link, effectively improving spatial resolution, dynamic range, and signal-to-noise ratio, while enhancing the system's adaptability and robustness. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for monitoring and analyzing optical cables provided by this invention;

[0023] Figure 2 This is a schematic diagram of the process for generating pulse coding parameters and matched filter parameters in an optional embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of an optical cable monitoring and data analysis device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] refer to Figure 1 The document presents a flowchart illustrating a method for monitoring and analyzing optical cables provided in this application. This method includes the following steps:

[0027] S1. Collect the current state data of the optical fiber link, and generate pulse coding parameters and matched filter parameters based on the current state data through a deep reinforcement learning agent; wherein, the current state data includes signal features extracted from the echo signal, environmental parameters provided by environmental sensors, and historical performance indicators; the pulse coding parameters are used to define the time-frequency characteristics of the pulse sequence.

[0028] Specifically, when collecting current status data of fiber optic links, three core data categories are covered: signal features extracted from echo signals, environmental parameters provided by environmental sensors, and historical performance indicators. When extracting signal features from echo signals, the time domain dimension obtains peak amplitude, pulse width, rising edge slope, falling edge slope, and event point interval. Peak amplitude is extracted using a sliding window maximum value detection algorithm, pulse width is the time span during which the signal amplitude drops to a specific proportion of the peak value, and rising and falling edge slopes are obtained by fitting the linear equation of the steep segment of the signal using the least squares method. In the frequency domain dimension, the time domain signal is converted to the frequency domain using a fast Fourier transform to extract the center frequency, bandwidth, number of spectral peaks, and spectral density distribution. In the statistical domain dimension, the mean, variance, skewness, and kurtosis of the signal are calculated using a sliding window to eliminate random noise interference.

[0029] The environmental sensors provide environmental parameters including temperature, strain, humidity, and vibration. Temperature parameters are acquired based on the Raman scattering effect and calculated by measuring the intensity ratio of anti-Stokes scattered light to Stokes scattered light; the corresponding calculation formula is as follows: ,in For the calculated temperature value, To calibrate the reference temperature, These are calibration coefficients determined experimentally. The intensity of anti-Stokes scattered light. The intensity of Stokes scattered light; the strain parameter is calculated by monitoring the center wavelength shift of the fiber optic grating, using the formula: ,in The strain value, This is the wavelength shift at the center of the grating. The strain sensitivity coefficient is used; the humidity parameter is directly acquired by a humidity sensor integrated into the optical cable junction box; the vibration parameter is obtained by detecting the triaxial acceleration value through a microelectromechanical system accelerometer.

[0030] Historical performance metrics include the signal-to-noise ratio, spatial resolution, event blind zone, dynamic range, and fault detection probability from the previous monitoring round. Historical data is stored in a time-series database. When collecting current status data, multiple recent sets of historical data are called as input to reflect the temporal correlation of link status.

[0031] All current state data needs to undergo a preprocessing step. First, an improved wavelet thresholding denoising algorithm is used to suppress noise; the threshold calculation formula is as follows: ,in For wavelet threshold, Let be the standard deviation of the wavelet detail coefficients. The data length is given; then, min-max normalization is used to map the data to a specific interval, as shown in the formula. ,in For the normalized data, The original data, It is the minimum value in historical data. The maximum value in the historical data is used as the basis for dimensionality reduction. Finally, principal component analysis is used to reduce the dimensionality and retain the principal components whose cumulative contribution rate meets the requirements, forming a state feature vector as the input of the deep reinforcement learning agent.

[0032] The deep reinforcement learning agent is constructed using a proximal policy optimization algorithm, with a core consisting of a policy network and a value network. The policy network is a fully connected neural network. The input layer receives a preprocessed state feature vector, and the hidden layers contain multiple layers of neurons. Each neuron in each layer uses the ReLU activation function, and a batch normalization layer and a dropout layer are added after each layer to suppress overfitting. The output layer has two branches, outputting pulse coding parameters and matched filter parameters, respectively. The pulse coding parameters define the time-frequency characteristics of the pulse sequence, including code pattern selection, code length, modulation frequency, and pulse amplitude. The matched filter parameters include the filter order, cutoff frequency, and window function type. The activation function of the output layer is selected according to the parameter type: continuous parameters are mapped using the Sigmoid function and then inversely normalized to obtain the actual value; discrete parameters are determined by sampling after outputting the probability distribution using the Softmax function. The value network has the same structure as the policy network, except that the output layer is a single neuron that outputs the state value to assist in the optimization of the policy network. The agent's initialization parameters are randomly generated using a specific distribution to ensure that the variance of each layer's output is consistent during network initialization.

[0033] S2. Generate an encoded pulse according to the pulse encoding parameters and inject the encoded pulse into the optical fiber link.

[0034] Specifically, when generating coded pulses based on pulse coding parameters, a three-level architecture of "baseband signal modulation - optical signal conversion - power control" is adopted. In the baseband signal generation stage, the corresponding modulation scheme is selected according to the code pattern parameters: if it is a binary phase shift keying (BPSK) code pattern, a binary baseband signal is generated based on a pseudo-random sequence generator, and different binary values ​​are mapped to different phases through a phase modulator; if it is a quadrature phase shift keying (QPSK) code pattern, the binary baseband signal is grouped and mapped to multiple phase states, and dual-channel signal synchronous modulation and synthesis are achieved through a quadrature modulator; if it is other multi-level phase shift keying (MPS) code patterns, the phase mapping is similar, only the phase mapping table is different. The pseudo-random sequence generator is implemented based on a linear feedback shift register, and the number of register bits is configured according to the code length parameter to ensure that the randomness and periodicity of the generated sequence meet the monitoring requirements.

[0035] After the baseband signal is generated, it is converted from a digital signal to an analog electrical signal by a digital-to-analog converter (DAC). The sampling rate of the DAC must satisfy the Nyquist sampling theorem to ensure that the sampling accuracy and bandwidth of the time-domain signal meet the requirements. The output amplitude of the DAC is adjusted by a programmable gain amplifier. The gain value is set according to the pulse amplitude in the pulse coding parameters. The gain value is calculated using the following formula: ,in This is the gain value. For the target pulse amplitude, The reference output power of the digital-to-analog converter is configured in real time through the digital control interface of the programmable gain amplifier.

[0036] The analog electrical signal is then input to a Mach-Zehnder modulator, which modulates the signal onto an optical carrier. The optical carrier is provided by a distributed feedback laser, with its center wavelength locked within a low-loss communication window. The linewidth and output power stability must meet requirements to ensure both transmission loss and stability of the optical signal. The bias voltage of the Mach-Zehnder modulator is dynamically adjusted by an automatic bias control module. Based on monitoring the DC component of the modulator's output optical power, a proportional-integral-derivative (PI-DE) control algorithm is used to lock the bias voltage at the optimal operating point to suppress modulation distortion and improve modulation efficiency.

[0037] After the optical signal is modulated, it undergoes pulse shaping and power calibration. The pulse shaping module uses a microstrip line filter, and the amplitude-frequency characteristics of the filter are designed according to the modulation frequency and bandwidth in the pulse coding parameters to ensure the spectral purity of the output pulse and suppress spurious frequency components. The power calibration module integrates an optical power meter and a variable optical attenuator. The optical power meter monitors the output optical power in real time and compares it with the target pulse amplitude in the pulse coding parameters. If there is a deviation, the attenuation of the variable optical attenuator is adjusted through a proportional-integral-derivative algorithm to control the deviation between the actual output optical power and the target value within a reasonable range.

[0038] The generated coded pulses are injected into the fiber optic link via an optical switch and fiber optic coupler. The optical switch employs a multi-link switching type, and the switching time must meet the flexibility requirements of the monitoring system. The coupling ratio of the fiber optic coupler must balance the injected link optical power with local monitoring needs, and the insertion loss and return loss must be controlled within a reasonable range to avoid interference from reflected light to the transmitting module. During the injection process, compatibility with different types of optical cables is achieved through a link interface module. The interface module supports standard fiber optic connectors and features a dustproof and waterproof sealed design to adapt to outdoor operation and maintenance environments.

[0039] S3. Receive the echo signal from the optical fiber link, and use the matched filter parameters to perform matched filtering on the echo signal to obtain a filtered output signal.

[0040] Specifically, when receiving echo signals from fiber optic links, an integrated architecture of "optical signal detection - electrical signal conversion - signal preprocessing - matched filtering" is adopted. First, the echo signal from the link is extracted using fiber optic couplers symmetrically configured with the transmitter. The echo signal is then input to the photodetector via an optical isolator. The photodetector is selected based on the link loss characteristics: PIN photodiodes are used for short-distance, low-loss links, while avalanche photodiodes are used for long-distance, high-loss links. The reverse bias voltage of the avalanche photodiode is dynamically adjusted by a high-voltage power supply module, and the gain is optimized according to the link attenuation to avoid signal saturation or excessive noise.

[0041] After the photodetector converts the optical signal into a weak electrical signal, it is amplified by a preamplifier. The preamplifier must have low-noise characteristics to minimize noise introduction during signal amplification. The amplified electrical signal is then input to an anti-aliasing filter. The cutoff frequency of the filter is set to a specific multiple of the cutoff frequency in the matched filter parameters to suppress high-frequency noise and aliasing interference. The filtered electrical signal is then converted into a digital signal by an analog-to-digital converter (ADC). The sampling rate of the ADC must meet the Nyquist sampling condition. The sampling triggering method uses external synchronization triggering, synchronized with the start time of the coded pulse at the transmitting end to ensure time alignment of the time domain signal.

[0042] The digital signal preprocessing stage includes baseline calibration and noise suppression. Baseline calibration uses a moving average method, constructing a sliding window and calculating the mean of the signal within the window as the baseline value. The original signal is then subtracted from the baseline value to eliminate DC drift. Noise suppression employs an adaptive Wiener filtering algorithm, dynamically adjusting the filter coefficients based on the signal's local variance, as shown in the formula: ,in The filtered signal The smoothing coefficient (calculated as follows) , For the local variance of the signal, (for noise variance) The original digital signal, This algorithm uses the local mean of the signal to suppress random noise while preserving signal details.

[0043] Matched filtering is performed in real-time using a field-programmable gate array (FPGA), dynamically generating filter coefficients based on the matched filter parameters. If the window function type is a Hanning window, the coefficient calculation formula is as follows: ,in These are the filter coefficients. Coefficient index (value range from 0 to...) ), Where is the filter order; if it is a Blackman window, the coefficient formula is... The parameter definitions are the same as for the Hanning window; if it is a Kaiser window, the coefficient formula is... ,in It is a zero-order modified Bessel function of the first kind. These are shape parameters that are dynamically adjusted based on the cutoff frequency.

[0044] The filter employs a finite-length unit impulse response structure, and convolution operations are implemented using a field-programmable gate array (FPGA) multiplier-accumulator array. The convolution formula is as follows: ,in The output signal of the convolution. For the generated filter coefficients, This is the preprocessed input signal. To improve computational efficiency, a pipelined architecture is adopted, decomposing the convolution operation into three parallel stages: data reading, multiplication, and accumulation. The data buffer uses dual-port RAM to implement ping-pong operation, ensuring a continuous supply of input data to meet the monitoring frame rate requirements.

[0045] After filtering, signal post-optimization is performed, including peak enhancement and distortion correction. Peak enhancement employs an adaptive threshold peak detection algorithm, which sets a dynamic threshold based on the ratio of the local maximum value to the global mean of the filtered output signal to locate event point peaks in the signal. Distortion correction addresses the pulse broadening problem caused by dispersion by using a dispersion compensation algorithm based on fractional Fourier transform. The algorithm searches for the optimal transform order to minimize the compensated pulse width. The optimal order is determined using an ergodic method combined with a mean square error criterion. The mean square error is calculated using the following formula: ,in This is the mean square error value. The number of signal sampling points. To compensate for the signal, The ideal pulse signal template is used. The final filtered output signal, after post-optimization processing, is stored in the on-chip FIFO buffer of the field-programmable gate array (FPGA) for subsequent performance index calculations. At the same time, it is synchronized to the main control unit via a high-speed serial interface for data backup.

[0046] S4. Calculate the performance index vector based on the filtered output signal, and calculate the reward signal based on the performance index vector.

[0047] Specifically, when calculating the performance index vector based on the filtered output signal, the core evaluation dimensions of optical cable monitoring are discussed, including five key indicators: signal-to-noise ratio, spatial resolution, event blind zone, dynamic range, and fault detection probability.

[0048] The signal-to-noise ratio (SNR) is calculated using the peak-to-noise ratio (PNR) method. First, a peak detection algorithm is used to extract the peak amplitudes of all event points in the filtered output signal. The average of multiple maximum peak values ​​is then taken as the effective signal amplitude. The noise amplitude is represented by the standard deviation of the event-free segments in the statistical signal. The calculation formula is as follows: ,in The standard deviation of noise. The number of sampling points for the event-free segment. For the first event segment The amplitude of each sampling point The mean of the signal during the event-free period is given; the signal-to-noise ratio is ultimately calculated as follows: ,in For signal-to-noise ratio, This represents the effective signal amplitude.

[0049] Spatial resolution is calculated based on the relationship between pulse width and fiber transmission characteristics. First, the full width at half maximum (FWHM) of the pulse corresponding to the standard reflection event in the filtered output signal is determined. Then, the time interval at which the amplitude drops to a specific proportion of the peak value is accurately calculated using linear interpolation. This is combined with the fiber's group velocity (calculated using the formula...). ,in For group velocity, The speed of light in a vacuum (where the group refractive index is the fiber group index) and the formula for calculating spatial resolution is: ,in For spatial resolution, The pulse half-width is 0.5.

[0050] Event blind spot The calculation targets the resolution of adjacent event points. It selects the peak positions of two consecutive reflection events in the filtered output signal, calculates the time difference and converts it into spatial distance; at the same time, it calculates the spatial distance when the tail of the previous event pulse decays to a certain multiple of the noise level. The event blind zone is defined as the larger of the two spatial distances.

[0051] The dynamic range is calculated using the integral method. First, the total integral energy of the filtered output signal is calculated, as shown in the formula: ,in The total integral energy, This refers to the number of sampling points in a single frame of signal. For the first The amplitude of each sampling point is then used to calculate the noise energy, using the formula: ,in For noise energy, the parameter definitions are consistent with those used in signal-to-noise ratio calculations; the dynamic range calculation formula is... ,in For dynamic range.

[0052] The calculation of fault detection probability is based on pre-set fault template matching. A signal feature template library is constructed for different types of faults (such as fiber breakage, loose connectors, and bending loss). Each fault type in the template library corresponds to a set of feature vectors, including characteristic parameters such as pulse peak shape, amplitude change rate, spectral distribution, and duration. During calculation, the feature vector of the filtered output signal is first extracted and its similarity to the feature vectors of each fault type in the template library is calculated using a cosine similarity algorithm. The formula is as follows: ,in The cosine similarity value is... The feature vector of the filtered output signal. The feature vector of the fault template, The dot product of two vectors. and These are the magnitudes of the two vectors, respectively.

[0053] When the similarity value is greater than a preset threshold, it is determined that a fault of the corresponding type has been detected; if the similarity values ​​are all less than the threshold, it is determined that there is no fault or an unknown fault. To ensure detection accuracy, the preset threshold is determined through a large number of fault simulation experiments and is set separately for different fault types (e.g., the threshold for fiber optic breakage faults is 0.85, and the threshold for loose connector faults is 0.75). The statistical method for fault detection probability is as follows: select multiple frames (e.g., 100 frames) of continuous monitoring data, and calculate the ratio of the number of frames in which the fault was successfully detected to the total number of monitoring frames, using the formula: ,in This represents the probability of fault detection. The number of frames in which a fault was successfully detected. This represents the total number of monitored frames.

[0054] Performance metric vectors are constructed as This includes the calculation results of the five key indicators mentioned above. When calculating the reward signal based on this vector, each indicator is first normalized to balance the differences in the dimensions of different indicators. For indicators that need to be maximized, such as signal-to-noise ratio, dynamic range, and fault detection probability, forward normalization is used, and the formula is: ,in The normalized index value, These are the original indicator values. This is the minimum acceptable value for the indicator (determined through statistical analysis of historical monitoring data). The optimal target value for the indicator is defined as follows; for indicators that need to be minimized, such as spatial resolution and event blind zone, inverse normalization is used, and the formula is as follows: The parameter definition is consistent with that of forward normalization.

[0055] Weight coefficients are set for each indicator, determined using the analytic hierarchy process (AHP), taking into account the impact of each indicator on the optical cable monitoring performance (e.g., signal-to-noise ratio weight is 0.3, spatial resolution weight is 0.2, event blind zone weight is 0.15, dynamic range weight is 0.2, and fault detection probability weight is 0.15). The basic reward signal calculation formula is as follows: ,in Based on the base reward value, For the first The weighting coefficients of each indicator For the first The normalized values ​​of each indicator.

[0056] Dynamic penalty and exploration incentive terms are introduced to optimize the reward signal. If the fault detection probability is less than the preset minimum acceptable value, a penalty coefficient is added. The corrected reward signal is If, during monitoring across multiple consecutive frames (e.g., 3 frames), the rate of change for all indicators is less than a preset threshold, it indicates that the agent's strategy is stagnating, and an exploration incentive term should be added. The reward signal at this time is Finally, the reward signal is mapped to the [-1, 1] interval to ensure compatibility with deep reinforcement learning algorithms. The mapping formula is as follows: ,in As the final reward signal, The minimum value of the reward signal, This represents the maximum value of the reward signal.

[0057] S5. Update the policy network of the deep reinforcement learning agent using the reward signal.

[0058] Specifically, the policy network update is based on the proximal policy optimization algorithm to achieve stable iteration of the agent's policy. The update process includes four core steps: experience storage, advantage estimation, policy optimization, and value network update, ensuring the convergence and generalization ability of the update process.

[0059] The experience storage stage constructs an experience replay pool to store experience data generated by the agent's interaction with the environment. Each set of experience data includes a current state feature vector. , generated action vectors (A vector combining pulse code parameters and matched filter parameters), reward signal Next state feature vector and termination mark (When the link is under normal monitoring) When monitoring is terminated due to link interruption or failure. The experience replay pool has a fixed capacity and uses a first-in, first-out (FIFO) strategy to manage data. When the storage capacity reaches its limit, the oldest stored experience data is deleted to ensure the timeliness of the data in the experience pool. A priority-based experience replay mechanism is also introduced, assigning priority to each group of experiences based on the timing difference error. The timing difference error is calculated using the following formula: ,in For timing difference error, This is a discount factor (used to weigh the importance of current rewards against future rewards). Output the value function for the next state. Output the value function for the current state. The priority calculation formula is as follows: ,in Prioritizing experience To minimize this value, a priority of 0 is used to prevent experience from being unsampled. Experience batches are drawn according to a priority probability distribution during sampling to improve the utilization efficiency of high-value experiences. The batch size is determined based on computing resources to ensure the stability of each update.

[0060] The dominance estimation stage employs a generalized dominance estimation algorithm to balance the bias and variance of the estimation results. The calculation formula is as follows: ,in This is the advantage estimate. The attenuation coefficient is... This is the trajectory length (i.e., the number of steps in one continuous interaction). For the first The time-series difference error is calculated step by step. During calculation, the value function values ​​of the current state and the next state are first output through the value network, and then substituted into the time-series difference error formula to obtain the error value for each step. Then, the cumulative advantage value is calculated recursively to achieve an unbiased estimate of the true advantage function. To improve the stability of policy updates, the advantage estimate is normalized using the standard deviation, as shown in the formula: ,in This is the normalized advantage estimate. The mean of all advantage estimates within the batch. This represents the standard deviation of all odds estimates within the batch.

[0061] The core of the policy optimization phase is to minimize the clipped surrogate objective function of the near-end policy optimization algorithm, as shown in the formula:

[0062]

[0063] in, The objective function value, These are the parameters of the current policy network. The old strategy network parameters are synchronized once after each preset round of updates. For the current policy network in state Select action The probability, This represents the probability that the old policy network will choose the same action under the same conditions. The clipping coefficients (used to limit the step size of policy updates to avoid parameter mutations that could lead to training instability) This represents the expectation of the advantage estimate.

[0064] For discrete action parameters (such as code selection and window function type), the policy network output is a probability distribution. The parameters are calculated using the Softmax function; for continuous action parameters (such as code length, modulation frequency, and filter order), the policy network is modeled using a Gaussian distribution. Let be the probability density function of the Gaussian distribution, and the formula is: ,in The mean of the Gaussian distribution is calculated from the output layer neurons of the policy network. The standard deviation of the Gaussian distribution is calculated by another set of output layer neurons in the policy network, ensuring that the standard deviation is positive.

[0065] The objective function is minimized using stochastic gradient descent, with the Adam algorithm chosen as the optimizer. A cosine annealing scheduling strategy is employed for the learning rate. The initial learning rate is determined based on the network size and decays to a specific percentage of the initial value after each preset number of updates. The minimum learning rate can be set to 1% of the initial value to avoid convergence stagnation due to an excessively low learning rate. During each parameter update, the gradient of the objective function with respect to the policy network parameters is calculated, and the optimizer adjusts the parameters until the objective function converges or the preset number of update cycles is reached.

[0066] The value network and policy network are updated simultaneously. The loss function of the value network uses the mean squared error loss, and the formula is as follows: ,in The loss value of the value network. For experience batch size, For value networks in state The output value below, For the first The reward signal for the step, For the value network in the next state The output value is calculated. The same optimizer and learning rate scheduling strategy as the policy network are used to update the value network parameters, minimizing the loss function to ensure the value network can accurately estimate state values, thus assisting in policy network optimization.

[0067] After the policy network is updated, the update effect is verified by calculating the performance index vector and reward signal corresponding to the action generated by the updated policy network and comparing them with the index before the update. If the performance improvement exceeds the preset threshold, the current updated parameters are retained; if the performance does not improve or decreases, the parameters before the update are reverted, the learning rate or batch size is readjusted, and the update is performed again to ensure that the policy network update always moves in the direction of improving monitoring performance.

[0068] The aforementioned optical fiber monitoring and data analysis method collects current status data of the optical fiber link and uses a deep reinforcement learning agent to dynamically generate adaptive pulse coding parameters and matched filter parameters based on this data, thereby generating and injecting coded pulses into the optical fiber link. Subsequently, echo signals are received, and the generated matched filter parameters are used to perform matched filtering on the signals to obtain filtered output signals. Then, performance index vectors and reward signals are calculated, and finally, the reward signal is used to update the agent's policy network. This achieves continuous optimization of pulse coding and matched filtering parameters, enabling the system to autonomously adapt to the dynamic changes of the optical fiber link, effectively improving spatial resolution, dynamic range, and signal-to-noise ratio, while enhancing the system's adaptability and robustness.

[0069] refer to Figure 2 In one optional embodiment, based on the current state data, pulse coding parameters and matched filter parameters are generated by a deep reinforcement learning agent, including the following steps:

[0070] S11. Normalize the current state data to obtain a normalized state vector; input the normalized state vector into the policy network of the deep reinforcement learning agent, and calculate the action probability distribution through forward propagation.

[0071] Specifically, an improved min-max normalization algorithm is used for the normalization of the current state data to address the input bias problem of the policy network caused by differences in data dimensions. First, historical extreme values ​​are calculated for each data category in the current state dataset. Each data category corresponds to a set of historical minimum and maximum values. These historical extreme values ​​are obtained by statistically analyzing recent sets of historical monitoring data from a time-series database. The extreme values ​​are updated after each fixed round of monitoring to ensure adaptation to long-term changes in link states. The normalization calculation uses the following formula:

[0072]

[0073] in, This represents the original value of the i-th type of data in the current state data. For the normalized value of the i-th type of data, For the historical minimum value of the i-th data type, U represents the historical maximum value of the i-th class of data, and U and L are the upper and lower limits of the normalization target interval, with U set to 1 and L set to 0. This interval matches the effective input range of the policy network activation function to avoid neuron output saturation.

[0074] After normalization, validity verification is required. Calculate the mean and standard deviation of the normalized state vector. The normalized state vector consists of the normalized values ​​of all classes, and the dimension is determined based on the number of classes in the current state data. The formula for calculating the mean is:

[0075]

[0076] in, Let M be the mean of the normalized state vector, and M be the dimension of the normalized state vector. It is the i-th element in the normalized state vector.

[0077] The formula for calculating the standard deviation is:

[0078]

[0079] in, This represents the standard deviation of the normalized state vector. Other parameters are defined in the same way as those in the mean calculation formula. If both the mean and standard deviation are within a preset reasonable range, the normalization is considered effective. If they exceed the range, the historical extreme value statistics need to be re-examined, and the extreme historical data generated during link failures should be excluded before recalculating the normalized vector.

[0080] The normalized state vector is input into the policy network for forward propagation to calculate the action probability distribution. The policy network adopts a fully connected architecture of "input layer-hidden layer-output layer". The number of neurons in the input layer is consistent with the dimension of the normalized state vector, and it is responsible for receiving the normalized state vector and converting it into a vector form that the network can compute. The hidden layer contains multiple layers, and the number of neurons in each layer is designed according to the network size. Each layer uses the ReLU activation function, and the formula for calculating the ReLU activation function is: , where x is the input value of the hidden layer neuron. This represents the output value of the neuron. Each hidden layer is followed by a batch normalization layer, and the batch normalization is calculated using the following formula:

[0081]

[0082] Where y is the batch-normalized output value, and x is the input value of the batch-normalized layer. This represents the average of the input data within the batch. The variance of the input data within the batch. and For learnable parameters, To prevent local minima where the denominator is zero, batch normalization accelerates network convergence and suppresses overfitting. A dropout layer is placed between the hidden and output layers to further enhance the network's generalization ability by randomly masking some neuron connections.

[0083] The output layer is designed with a two-branch structure based on the action type. The first branch outputs the mean of each consecutive action parameter, and the second branch outputs the standard deviation of each consecutive action parameter. Both branches use linear activation functions to output parameters within the true numerical range. The action probability distribution is modeled using a Gaussian distribution. For each action parameter, its probability density function is:

[0084]

[0085] in, For the normalized state vector Next action parameters The probability density, Let j be the value of the action parameter. The first branch of the output layer outputs the mean value of the j-th action parameter. This outputs the standard deviation of the j-th action parameter in the second branch of the output layer. To ensure the standard deviation is positive, it is calculated from the original output of the second branch of the output layer using the following formula: ,in This is the original output of the second branch of the output layer. The probability distributions of all action parameters are independent of each other, and the overall action probability distribution is the product of the probability density functions of each parameter.

[0086] S12. Sample according to the action probability distribution to generate an initial action vector; wherein, the initial action vector includes the pulse start frequency, pulse bandwidth, pulse width and frequency modulation slope.

[0087] Specifically, a reparameterization technique is used for sampling based on the action probability distribution to avoid the vanishing gradient problem in the policy network caused by the non-differentiability of the sampling process. For each continuous action parameter, including pulse start frequency, pulse bandwidth, pulse width, and frequency modulation slope, the sampling calculation uses the following formula:

[0088]

[0089] in, For the j-th action parameter value obtained from sampling, Output the mean of the j-th action parameter. Output the standard deviation of the j-th action parameter. Let be a random variable that follows a standard normal distribution. This formula transforms the random sampling process into a process of sampling the random variable. The sampling ensures that the gradient can be backpropagated to the policy network parameters through the mean and standard deviation.

[0090] The sampling process controls the range of randomness of random variables to avoid generating extreme outliers. For random variables... A cutoff interval is set. When the sampled random variable value is less than the lower limit of the interval, it is forced to equal the lower limit; when the sampled random variable value is greater than the upper limit of the interval, it is forced to equal the upper limit. This interval corresponds to the high-confidence interval of the standard normal distribution, which can reduce extreme sampling results while preserving randomness.

[0091] The generated initial motion vector consists of four parameters: pulse start frequency, pulse bandwidth, pulse width, and frequency modulation slope. The pulse start frequency refers to the initial frequency at which the laser emits the coded pulse, corresponding to the initial wavelength of the optical carrier, and is obtained by converting the speed of light formula. The speed of light formula is: ,in where λ is the wavelength and c is the speed of light in a vacuum. The pulse start frequency determines the initial transmission frequency band of the coded pulse in the optical fiber. The pulse bandwidth refers to the frequency range of the coded pulse, that is, the difference between the highest and lowest pulse frequencies, calculated using the following formula: Where B is the pulse bandwidth, The highest frequency of the pulse. The pulse bandwidth, being the lowest frequency of the pulse, directly affects the pulse's frequency domain resolution and anti-interference capability. Pulse width refers to the duration of the coded pulse in the time domain; that is, the time interval between the start of the rising edge where the pulse amplitude rises from a specific percentage of its peak to another specific percentage, and the end of the falling edge where the amplitude falls from that other specific percentage to the previous specific percentage. The pulse width determines the pulse's time domain resolution. The frequency modulation slope refers to the rate at which the coded pulse frequency changes linearly with time. For a linearly modulated pulse, its calculation formula is... ,in B is the frequency modulation slope, and B is the pulse bandwidth. The pulse width and the frequency modulation slope determine the steepness of the frequency change, which affects the impulse response design of the matched filter.

[0092] After sampling, the initial action vector is format-converted, converting each parameter into a numerical format recognizable by the physical device. The pulse start frequency and pulse bandwidth are converted to corresponding high-frequency units, the pulse width to corresponding short-time units, and the frequency modulation slope to corresponding frequency change rate units, ensuring consistency with device parameter units during subsequent constraint verification.

[0093] S13. Adjust the over-limit parameters according to the laser's tuning range and system safety threshold, and perform constraint verification on the initial action vector to obtain a compliant action vector.

[0094] Specifically, the core of constraint verification is to ensure that the parameters in the initial action vector are adapted to the performance limits of the physical equipment and the safety requirements of system operation. It consists of four steps: constraint definition, out-of-limit parameter detection, parameter adjustment, and secondary verification.

[0095] First, constraints are defined, including two categories: laser tuning range and system safety thresholds. Laser tuning range constraints are determined based on the hardware parameters of the selected laser. The laser's wavelength tuning range corresponds to a specific frequency range, derived from the speed of light formula; therefore, the pulse start frequency must be within this range. Simultaneously, the highest pulse frequency is the sum of the pulse start frequency and the pulse bandwidth, and this highest frequency must not exceed the laser's maximum tuning frequency; that is, the pulse bandwidth must not exceed the difference between the laser's maximum tuning frequency and the pulse start frequency. System safety threshold constraints are determined based on the hardware capacity and measurement accuracy requirements of the optical cable monitoring system. The pulse width must be no lower than a certain lower limit; if it is lower, the laser's output power density will be too high, damaging the fiber end-face coating. The pulse width must also be no higher than a certain upper limit; if it is higher, the spatial resolution will not meet the fault location accuracy requirements. The frequency modulation slope must be no higher than a certain upper limit; if it is higher, the frequency synthesizer will not be able to track the frequency modulation speed in real time, resulting in frequency distortion. Similarly, the frequency modulation slope must be no lower than a certain lower limit; if it is lower, the pulse frequency domain characteristics will be too narrow, reducing anti-dispersion capability.

[0096] The out-of-limit parameter detection employs a parameter-by-parameter traversal check method. Each parameter in the initial action vector is compared with its corresponding constraint condition, and parameters exceeding the constraint range are marked. For example, if the pulse start frequency obtained from the initial sampling is lower than the laser's minimum tuning frequency, the pulse start frequency is marked as an out-of-limit parameter; if the sum of the pulse start frequency and the pulse bandwidth exceeds the laser's maximum tuning frequency, the pulse bandwidth is marked as an out-of-limit parameter.

[0097] The adjustment of out-of-limit parameters adopts a "priority adaptation method," adjusting parameters according to their priority of impact on system performance from high to low. First, the pulse start frequency is adjusted. If the pulse start frequency is lower than the laser's minimum tuning frequency, it is adjusted to the laser's minimum tuning frequency; if the pulse start frequency is higher than the laser's maximum tuning frequency, it is adjusted to the laser's maximum tuning frequency. Next, the pulse bandwidth is adjusted. The maximum allowable bandwidth is calculated based on the adjusted pulse start frequency, which is the difference between the laser's maximum tuning frequency and the adjusted pulse start frequency. If the current pulse bandwidth is greater than the maximum allowable bandwidth, it is adjusted to the maximum allowable bandwidth; if the pulse bandwidth is less than zero (an abnormal situation), it is adjusted to the minimum bandwidth supported by the system. Then, the pulse width is adjusted. If the pulse width is lower than the lower limit of the system's safety threshold, it is adjusted to that lower limit; if the pulse width is higher than the upper limit of the system's safety threshold, it is adjusted to that upper limit. Finally, the frequency modulation slope is adjusted. First, the theoretical frequency modulation slope is calculated based on the adjusted pulse bandwidth and pulse width. The theoretical frequency modulation slope is the ratio of pulse bandwidth to pulse width. If the theoretical frequency modulation slope exceeds the upper limit of the system safety threshold, the pulse bandwidth is appropriately reduced and the theoretical frequency modulation slope is recalculated until the theoretical frequency modulation slope is within the upper limit range. If the theoretical frequency modulation slope is lower than the lower limit of the system safety threshold, the pulse width is appropriately reduced and the theoretical frequency modulation slope is recalculated until the theoretical frequency modulation slope is within the lower limit range.

[0098] After parameter adjustment, a secondary verification is performed. All adjusted parameters are compared with the constraints again to check for any out-of-limit parameters. If all parameters meet the constraints, the adjusted action vector is determined as a compliant action vector. If out-of-limit parameters still exist, the sampling process is returned to the previous step to resample and generate an initial action vector. A maximum number of resampling attempts is set. If a compliant action vector cannot be obtained after reaching the maximum number of attempts, the system parameter self-check process is triggered to check whether the constraint definition matches the hardware parameters.

[0099] S14. Extract impulse code parameters and matched filter parameters from the compliant action vector; wherein, the matched filter parameters include matched filter coefficients used to define the impulse response of the matched filter.

[0100] Specifically, the compliant action vector consists of four core parameters after constraint verification: pulse start frequency, pulse bandwidth, pulse width, and frequency modulation slope. These parameters directly determine the time-frequency characteristics of the coded pulse and the design basis of the matched filter. The extraction process needs to be based on the physical meaning and functional relationship of the parameters, and classify them into pulse coding parameters and matched filter parameters respectively to ensure that the two types of parameters can work together accurately in subsequent system operation.

[0101] When extracting pulse coding parameters, the focus is on the time-frequency control parameters required for generating the coded pulse. The pulse start frequency, pulse bandwidth, pulse width, and frequency modulation slope are directly extracted from the compliant action vector as core pulse coding parameters. Specifically, the pulse start frequency serves as the frequency domain reference for the coded pulse, determining the initial oscillation frequency of the optical carrier and serving as the starting point for subsequent frequency modulation. The pulse bandwidth defines the frequency variation range of the coded pulse, directly affecting its frequency domain resolution and anti-interference capability. Its value must be adapted to the transmission bandwidth of the fiber optic link to avoid excessive signal attenuation due to excessive bandwidth or insufficient information carrying capacity due to insufficient bandwidth. The pulse width defines the duration of the coded pulse in the time domain and is a key indicator for balancing spatial resolution and dynamic range. The duration setting must consider both fault location accuracy and signal detection capability over long distances. The frequency modulation slope describes the rate of frequency change of the coded pulse over time and is a core parameter for generating linear frequency modulated pulses. Its value is indirectly determined by the ratio of pulse bandwidth to pulse width, ensuring a linear frequency change and guaranteeing the pulse compression effect of matched filtering. The extracted pulse coding parameters need to be stored in a preset format, and each parameter needs to be associated with a corresponding unit identifier so that the subsequent encoded pulse generation module can accurately read and parse them.

[0102] The core of extracting matched filter parameters is calculating the matched filter coefficients used to define the impulse response. This process requires signal transformation and discretization based on the pulse coding parameters and the design principles of the matched filter. First, a time-domain mathematical model of the coded pulse is constructed based on the pulse coding parameters. If the coded pulse is a linear frequency modulated pulse, its continuous time-domain expression is:

[0103]

[0104] In the formula, A is a continuous time-domain complex signal of the encoded pulse, where A is the pulse amplitude (determined by the system's preset transmit power to ensure that the signal strength meets the link detection requirements). For rectangular window functions (when hour, (When the value exceeds this range, it is 0, used to limit the duration of the pulse in the time domain). The pulse width extracted from the compliant action vector. The pulse start frequency extracted from the compliant action vector. The frequency modulation slope extracted from the compliant action vector, where j is the imaginary unit (satisfying... (used to describe the phase change of a signal). For the initial phase term, This is the linear frequency modulation phase term.

[0105] According to the design principles of matched filters, the continuous-time impulse response of a matched filter must have a conjugate inverse relationship with the time-domain signal of the encoded pulse, expressed as:

[0106]

[0107] In the formula, The continuous time-domain impulse response of the matched filter is given by k, which is the gain coefficient (used to adjust the amplitude of the filtered output signal to avoid signal saturation or excessively low amplitude; it is set to a constant according to the dynamic range requirements). Represents the encoded pulse complex signal The conjugate is taken (i.e., the real part remains unchanged, and the imaginary part is inverted, to cancel phase distortion during signal transmission), and T is the pulse delay time (usually set as the duration of the encoded pulse). (to ensure that the impulse response is consistent with the time domain length of the encoded pulse). This means reversing the time axis so that the impulse response can be precisely aligned with the received echo signal in the time domain, achieving a pulse compression effect.

[0108] Since the actual system uses digital signal processing, the continuous-time impulse response is discretized into matched filter coefficients. First, the sampling period for discretization is determined. Its value is determined by the system's sampling rate (the sampling rate must satisfy the Nyquist sampling theorem, i.e., the sampling rate is not less than twice the highest frequency of the encoded pulse, ensuring that the signal is free of aliasing distortion after discretization). Replace the continuous time t with discrete time. (n is the discrete-time index, with values ​​ranging from 1 to 1000) (where N is the length of the discretized coefficients), then the expression for the discretized matched filter coefficients is:

[0109]

[0110] In the formula, Let N be the coefficient of the nth discretized matched filter, and N be the total length of the coefficients (given by the pulse width). With sampling period Determined, that is , This indicates rounding up to the nearest integer to ensure that the coefficients fully cover the time domain range of the impulse response.

[0111] To avoid numerical overflow or signal amplitude distortion during filtering, the discretized matched filter coefficients are normalized using an energy normalization formula:

[0112]

[0113] In the formula, These are the normalized matched filter coefficients. The sum of energy for all discrete coefficients ( Indicates the coefficient Modulo operation (i.e., calculating the amplitude of complex coefficients) is used to normalize the total energy of the matched filter coefficients to 1, ensuring that the energy of the filtered output signal matches the energy of the input echo signal. This prevents system saturation due to excessive energy or signal detection sensitivity due to insufficient energy.

[0114] Finally, the extracted pulse coding parameters (including pulse start frequency, pulse bandwidth, pulse width, and frequency modulation slope) and matched filter parameters (including normalized matched filter coefficients) need to be stored in the system's parameter cache area, and a parameter association index should be established to ensure that the corresponding pulse coding parameters can be called when generating coded pulses and the corresponding matched filter coefficients can be called when performing matched filtering, so as to achieve accurate parameter calling and coordinated operation.

[0115] In one optional embodiment, the echo signal is subjected to matched filtering using matched filter parameters to obtain a filtered output signal, including the following steps:

[0116] S21. Construct the impulse response function of the matched filter based on the matched filter coefficients in the matched filter parameters.

[0117] Specifically, the matched filter coefficients are discrete, normalized coefficients, denoted as... ( (where N represents the total length of the coefficients). In the discrete-time domain, the impulse response function of the matched filter directly corresponds to this coefficient, and its expression is: ,in The discrete-time impulse response is represented by n, which is the discrete-time index used to characterize the discrete position of the impulse response on the time axis. N is the same as the length of the matched filter coefficients and is determined by the duration of the encoded pulse and the system sampling period. This discrete impulse response can be directly used for subsequent digital domain convolution operations without additional conversion.

[0118] If a continuous-time-domain impulse response function needs to be constructed (for theoretical analysis or analog signal processing scenarios), the discrete coefficients are converted into a continuous signal using linear interpolation, expressed as follows: In the formula The response is a continuous-time impulse response, where t is a continuous-time variable. The system sampling period is determined by the sampling rate, and the sampling rate must satisfy the Nyquist sampling theorem. For rectangular interpolation window functions (when hour, ;when hour, Its function is to expand each discrete coefficient to a duration of The rectangular pulses are superimposed to form a continuous impulse response curve.

[0119] After the construction is completed, the validity is verified. The frequency domain characteristics of the impulse response are calculated by Fourier transform to confirm that its amplitude frequency characteristics are mirror images of the coded pulse amplitude frequency characteristics and that its phase frequency characteristics are linearly compensated. If these conditions are met, the construction is considered valid; otherwise, the extraction process of the matched filter coefficients needs to be checked again.

[0120] S22. Preprocess the echo signal to obtain an aligned echo signal; the preprocessing includes noise reduction and time alignment.

[0121] Specifically, when preprocessing the echo signal to obtain the aligned echo signal, denoising is first performed using an improved wavelet threshold denoising algorithm. First, a wavelet basis suitable for the characteristics of the echo signal is selected (a db-series wavelet basis can be used, which combines good temporal locality with the ability to capture abrupt changes). The number of wavelet decomposition levels is then set (determined based on the echo signal bandwidth and noise intensity, balancing noise suppression and detail preservation). Wavelet decomposition is then performed on the echo signal to obtain wavelet approximation coefficients representing low-frequency trends and wavelet detail coefficients representing high-frequency noise and abrupt change components.

[0122] An adaptive thresholding process is applied to the wavelet detail coefficients. The threshold calculation formula is as follows: ,in For wavelet threshold, Let M be the noise standard deviation of the wavelet detail coefficients (obtained by dividing the absolute value of the median of the first-scale wavelet detail coefficients by 0.6745, which is fast and unaffected by outliers), and M be the number of echo signal sampling points; then, the detail coefficients are processed using a soft thresholding function, expressed as follows: In the formula For the post-processing detail coefficients, These are the original detail coefficients. The sign function (output 1 for positive input, -1 for negative input, and 0 for 0 input) sets coefficients with absolute values ​​less than the threshold to 0 and subtracts the threshold from coefficients with absolute values ​​greater than the threshold while retaining their sign, thus achieving noise suppression.

[0123] Wavelet reconstruction is performed on the processed approximation coefficients and detail coefficients to obtain the denoised echo signal; the signal-to-noise ratio improvement of the signal before and after denoising is calculated. If the improvement reaches the preset threshold, the denoising is effective; otherwise, the wavelet basis and decomposition level are readjusted.

[0124] Subsequently, time alignment processing is performed using a cross-correlation-based synchronization algorithm. First, the coded pulse reference signal (the time-domain signal of the coded pulse generated at the transmitter, which ideally matches the shape of the echo signal) is extracted. Then, a cross-correlation operation is performed between the denoised echo signal and the reference signal, expressed as follows: In the formula The cross-correlation function value, For the denoised echo signal, For reference signal, This represents the time offset; the peak position of the cross-correlation function corresponds to the optimal synchronization point, i.e. Maximum This represents the time offset of the echo signal relative to the reference signal.

[0125] according to Adjust the denoised echo signal. Time forward translation , negative translation The aligned echo signal is obtained; the alignment effect is verified by mean square error, using the following formula: In the formula Mean square error, To align the i-th sampling point of the echo signal, The i-th sampling point of the reference signal is M, and the number of sampling points is M. If the mean square error is less than the preset threshold, the alignment is valid.

[0126] S23. Perform convolution operation between the aligned echo signal and the impulse response function to obtain the initial filtered output.

[0127] Specifically, when convolving the aligned echo signal with the impulse response function to obtain the initial filtered output, the aligned echo signal is a discrete-time domain signal. ( (where M is the number of sampling points), the impulse response function is the discrete-time impulse response. ( N is the impulse response length, typically To avoid computational redundancy.

[0128] The expression for discrete convolution is: In the formula This represents the m-th sample point of the initial filtered output, where m is the output discrete index (range of values). The output length is the sum of the two input lengths minus 1, and k is the discrete index of the impulse response. To align the echo signal sampling points (when or hour, (Distortion is avoided by using zero-filling to handle boundaries).

[0129] Convolution operations are implemented using a pipelined architecture (such as a multiplier-accumulator array in an FPGA), which converts the impulse response... Pre-stored in on-chip RAM as convolution kernels to align echo signals The input is buffered point by point through a FIFO buffer. For each input sampling point, a multiplication operation is performed simultaneously with all sampling points of the impulse response. The results are then accumulated to obtain the current m. The system adopts a ping-pong operation mode, which caches the next set of sampling points while performing calculations to ensure that the real-time monitoring frame rate requirements are met.

[0130] After the operation is completed, verify the initial filtered output: observe the time domain waveform to confirm the pulse compression effect (the pulse width at the event point is reduced and the peak amplitude is increased), calculate the energy comparison between the output and the input signal to verify the energy conservation (the fluctuation is controlled within the preset percentage). If the standard is not met, re-check the accuracy of the impulse response construction and time alignment, eliminate the error, and re-operate.

[0131] S24. Calculate the signal-to-noise ratio estimate by performing peak detection and background noise estimation on the initial filtered output.

[0132] Specifically, when calculating the signal-to-noise ratio estimate by performing peak detection and background noise estimation on the initial filtered output, peak detection is performed first, using an adaptive threshold peak detection algorithm. The initial filtered output... Perform a local maximum search, set a local search window (the length is determined by the width of the compressed encoded pulse, covering the pulse waveform of a single event point), traverse all sampling points, and mark the sampling point with the largest amplitude within the window as the candidate peak point.

[0133] A dynamic threshold is used to filter candidate peak points, and the formula is as follows: In the formula For dynamic thresholds, The mean signal value of the local area surrounding the candidate peak point is (L sampling points are taken before and after the candidate point, and L is determined according to the compressed pulse width). This is the threshold adjustment coefficient (with a value of 2 to 3, balancing the false negative rate and the false positive rate). The standard deviation of the signal in the local area (calculation formula) (where m is the index of the candidate peak point).

[0134] Filter out values ​​with amplitude greater than The candidate points are selected as effective peak points, and their arithmetic mean amplitude is taken as the effective signal amplitude, as shown in the formula: In the formula K represents the average amplitude of the effective signal, and K is the number of effective peak points (required). To ensure statistical significance, the search window should be expanded if necessary. Let be the amplitude of the i-th valid peak point.

[0135] Then, background noise estimation is performed. In the initial filtered output, a stable region without event interference is selected. The influence range of the event point is determined according to the position of the effective peak point (M sampling points are taken before and after each effective point, where M is 1 / 2 of the pulse tail length). The remaining region after excluding this range is the background noise statistical region. If the region length is insufficient, M is reduced or non-reflection sections at both ends of the link are selected to supplement it.

[0136] Calculate the mean and standard deviation of the noise region using the mean formula. In the formula The noise mean. The number of sampling points in the noise area. The amplitude of the p-th sampling point in the noise region; standard deviation formula. In the formula This represents the standard deviation of noise (characterizing noise intensity).

[0137] right To smooth the sliding window, the formula is: In the formula The noise intensity after smoothing is represented by W, which is the length of the sliding window (50~100 sampling points, determined according to the noise fluctuation frequency). is the standard deviation of the q-th noise sub-region (the noise region is divided into sub-regions according to W).

[0138] The formula for calculating the signal-to-noise ratio estimate is as follows: In the formula Here is the signal-to-noise ratio estimate, and 20 is the coefficient for converting the amplitude ratio to dB. The effective signal average amplitude, This is the standard deviation of the smoothed noise; if If the value is greater than the preset minimum threshold, the estimation is valid; otherwise, the noise region is readjusted or Then it was tested again.

[0139] S25. Adjust the stride and window function of the convolution operation based on the signal-to-noise ratio estimate to obtain the adjusted convolution operation parameters; based on the adjusted convolution operation parameters, re-perform the convolution operation on the aligned echo signal and the impulse response function to obtain the optimized filtered output signal.

[0140] Specifically, the step size S is defined as the number of offset steps of the input signal when calculating adjacent output sampling points (the initial step size is usually 1), and a low signal-to-noise ratio threshold is set. (Typically 20dB) and high signal-to-noise ratio threshold (Typically 35dB).

[0141] like Strong noise interference, adjust step size as follows (Pointwise convolution) ensures that details of weak event points are not lost; if The signal quality is moderate, and the adjustment step size is... Balancing details and efficiency; if The signal quality is excellent, and the adjustment step size is [missing information]. This improves computational efficiency. The adjusted verification sampling interval... ( (The sampling period is used to ensure that it is less than 1 / 2 of the width of the compressed signal to avoid aliasing.)

[0142] Next, adjust the window function until the signal-to-noise ratio is lower than... When using the Blackman window (strong sidelobe suppression), the formula is as follows: In the formula The window function weights are given by N, where N is the window function length (consistent with the impulse response). When the signal-to-noise ratio is between the two thresholds, the Hanning window is maintained, as shown in the formula. The signal-to-noise ratio is higher than that of other parts of the country. When selecting a rectangular window, the formula is: ( The main lobe is narrow and the operation is simple.

[0143] After adjusting the window function, normalization is performed using the following formula: In the formula The weights are normalized to ensure that the signal energy remains unchanged after weighting.

[0144] Based on the adjusted parameters ( and normalized window function When re-performing the convolution operation on the aligned echo signal and the impulse response function, the weights of the normalized window function are first fused with the impulse response coefficients to form a weighted impulse response. This achieves the suppression of signal spectral leakage and boundary effects by the window function. The corrected discrete convolution operation formula is as follows:

[0145]

[0146] In the formula, To optimize the filtered output signal's first... One sampling point,

[0147] It is the weighted impulse response coefficient ( This represents the original discrete impulse response. The normalized window function weights are multiplied together to achieve a weighted adjustment of the impulse response. It is based on the adjusted step size Selected aligned echo signal sampling points (when or When the value is 0, zero-padding is used to handle the boundaries to avoid signal amplitude distortion at the boundaries. m is the discrete index of the optimized filtered output, with a value range of... ( This indicates rounding up to the nearest integer to ensure coverage of all valid signal segments.

[0148] The reconvolution operation is still implemented based on a pipelined architecture, requiring only the addition of a multiplication step for the window function weights to the original computational logic. The pre-stored impulse response coefficients... With normalized window function weights The weighting coefficients are obtained by pre-multiplication and stored in on-chip RAM. The subsequent multiplication and accumulation operations with the aligned echo signal sampling points are consistent with the initial convolution, ensuring that the computational efficiency is not significantly affected.

[0149] In one optional embodiment, calculating the performance index vector based on the filtered output signal includes the following steps:

[0150] S31. Extract the peak power and noise variance from the filtered output signal, and calculate the signal-to-noise ratio (SNR) based on the peak power and noise variance. The formula for calculating the SNR is:

[0151]

[0152] SNR represents the signal-to-noise ratio. Let be the filtered output signal at time t. The peak power of the filtered output signal is represented by t, where t is time. This represents the noise variance, calculated from the signal background region.

[0153] Specifically, when extracting peak power from the filtered output signal, the effective event pulses in the signal are first located. These pulses correspond to characteristic locations such as joints and fault points in the fiber optic link. By traversing all time points of the filtered output signal, pulse segments with significantly higher amplitudes than surrounding signals are identified, and the square of the signal amplitude at all time points within each pulse segment is calculated. (i.e., instantaneous power), and select the maximum value among them, which is the value in the formula. (Peak power).

[0154] Extracting noise variance First, determine the background signal region. This region should avoid the time periods corresponding to all valid event pulses. Select a continuous time period in the filtered output signal where the amplitude is stable and without significant fluctuations to ensure that this region contains only background noise. Calculate the variance for all signal sampling points within the background region. The specific process is as follows: first, calculate the mean of the signal amplitude in this region; then, calculate the square of the difference between the amplitude and the mean for each sampling point; finally, calculate the average of these squared values. The result is the noise variance. .

[0155] Substituting the extracted peak power and noise variance into the signal-to-noise ratio (SNR) calculation formula, the resulting SNR value directly reflects the intensity ratio of the effective signal to the background noise in the filtered output signal. The larger the SNR value, the more prominent the effective signal, and the higher the accuracy of subsequent fault detection or event localization.

[0156] S32. Analyze the main lobe width after pulse compression from the filtered output signal; based on the main lobe width, combined with the speed of light and the refractive index of the fiber, calculate the spatial resolution; the formula for calculating the spatial resolution is:

[0157]

[0158] in, denoted by spatial resolution, c represents the speed of light, n represents the refractive index of the fiber, and B represents the pulse bandwidth.

[0159] Specifically, when analyzing the main lobe width from the filtered output signal, focus on the time-domain waveform of the effective event pulse. The main lobe width refers to the time range covered by the pulse amplitude rising from 50% of the peak value to the peak value and then falling back to 50% of the peak value (i.e., the pulse half-width at half maximum). The start and end times of this time range are accurately calculated using linear interpolation, and the difference between the two is the main lobe width. Since the main lobe width is related to the pulse bandwidth... There is an inverse relationship (the wider the pulse bandwidth, the narrower the main lobe width), and the pulse bandwidth determined in the pulse coding stage can be directly used. Used for spatial resolution calculations.

[0160] In the formula for calculating spatial resolution, c is the speed of light in a vacuum (a physical constant). To monitor the refractive index of the optical fiber used (which is determined according to the type of optical fiber; for example, the refractive index of single-mode optical fiber is usually a fixed value). This represents the pulse bandwidth. In the formula... The term represents the speed at which light signals propagate in an optical fiber (since the speed of light in an optical fiber is...). Dividing by 2 is to convert the time difference into a one-way spatial distance, multiplying by After that, the final result was This refers to spatial resolution. This metric reflects the system's ability to distinguish between two adjacent event points in a fiber optic link. The smaller the value, the closer the distance between adjacent event points the system can distinguish, and the higher the positioning accuracy.

[0161] S33. Standardize the signal-to-noise ratio and spatial resolution to obtain a standardized performance index vector.

[0162] Specifically, the signal-to-noise ratio (SNR) and spatial resolution are standardized. The core purpose is to eliminate the difference in their dimensions (SNR is a dimensionless value, while spatial resolution is measured in units of length), so that the weights of the two indicators can be fairly reflected in subsequent analyses. The standardization process uses the min-max normalization method, which achieves a unified dimension by mapping the indicator values ​​to the [0,1] interval.

[0163] For positive normalization of the signal-to-noise ratio (SNR), since a higher SNR generally indicates better signal quality and monitoring performance, the normalization formula is as follows:

[0164]

[0165] in, The signal-to-noise ratio after standardization. This represents the minimum signal-to-noise ratio among historical monitoring data. The maximum signal-to-noise ratio (SNR) in historical monitoring data is represented by this formula. This formula maps different ranges of SNR to [0,1]. The closer the value is to 1, the better the SNR performance.

[0166] Inverse normalization is performed for spatial resolution. Since smaller spatial resolution results in better positioning accuracy and monitoring performance, the normalization formula is as follows:

[0167]

[0168] in, For the standardized spatial resolution, This represents the minimum spatial resolution value in the historical monitoring data. This represents the maximum spatial resolution in historical monitoring data. The spatial resolution is also mapped to [0,1] using this formula. The closer the value is to 1, the better the spatial resolution performance.

[0169] Standardized signal-to-noise ratio With standardized spatial resolution By combining them in sequence, a two-dimensional standardized performance index vector is formed. This vector can be directly used to calculate subsequent reward signals, or as a core input for deep reinforcement learning agents to judge current monitoring performance and adjust strategies.

[0170] In an optional embodiment, before constructing the impulse response function of the matched filter based on the matched filter coefficients in the matched filter parameters, the following steps are further included:

[0171] S41. Based on the latest status data and performance index vector of the fiber optic link, calculate the residual dispersion error, and calculate the error signal based on the residual dispersion error; wherein, the error signal represents the difference between the expected output and the actual filtered output.

[0172] Specifically, let the link length be in the latest status data of the fiber optic link. Reference wavelength The dispersion coefficient of the fiber is as follows The actual transmission wavelength is relative to The offset is The currently deployed dispersion compensation amount is In the performance metric vector, the correction coefficient related to dispersion is determined by the spatial resolution. With signal-to-noise ratio Jointly determined, denoted as ( , (This is a preset scaling factor used to quantify the weight of the performance index on the dispersion error).

[0173] Residual dispersion error The calculation needs to take into account the dispersion changes caused by the actual wavelength shift and the performance index correction. The formula is:

[0174]

[0175] In the formula, It is the wavelength derivative of the dispersion coefficient (characterizing the rate of change of dispersion with wavelength, which is determined by the type of optical fiber). The actual cumulative dispersion of the link, minus the compensated amount. With performance index correction factor Then, the residual dispersion error is obtained. This error reflects the part that is not covered by the current dispersion compensation and is also the core basis for subsequent adjustment of the filter coefficients.

[0176] Error signal The difference between the expected filter output and the actual filter output is quantized, where the expected filter output... Determined by the ideal signal model when the residual dispersion error is 0, the expression is: ( For the ideal peak amplitude, The sampling period is For the ideal event time, (The ideal pulse width is used); the actual filtered output is the value under the current iteration. Therefore, the formula for the error signal is: In the formula The larger the absolute value, the more significant the deviation between the actual output and the ideal state, requiring further adjustments to the filter coefficients through subsequent algorithms.

[0177] S42. Update the filter coefficients using the least mean square algorithm based on the error signal and the echo signal; the expression for the least mean square algorithm is:

[0178]

[0179] in, This represents the updated filter coefficient vector at the (n+1)th iteration. This represents the current filter coefficient vector at the nth iteration. This represents the adaptive step size parameter used to control the update magnitude. This represents the error signal at the nth iteration. This represents the input echo signal vector at the nth iteration.

[0180] Specifically, in the nth iteration, the current filter coefficient vector is ( The superscript indicates the filter order. (Indicates transpose), the input echo signal vector is (Arranged in reverse chronological order to ensure compatibility with the convolution of the coefficient vectors), the error signal is a scalar. Adaptive step size parameter This is a preset constant.

[0181] During the update process, each coefficient component ( The adjustment amount is That is, through error signals The positive and negative judgment adjustment direction ( Increase the coefficient at time, (Time reduction coefficient), through echo signal components The amplitude is used to determine the adjustment range (the larger the signal amplitude, the more significant the corresponding coefficient adjustment, thus achieving targeted optimization for strong signal components).

[0182] After updating the filter coefficient vector To constrain the amplitude and prevent the filter output from saturating due to excessively large coefficients, the constraint formula is as follows:

[0183]

[0184] In the formula, It represents the 2-norm of a vector (i.e., the square root of the sum of the squares of the magnitudes of the coefficients). , These are the preset maximum and minimum amplitude limits for the coefficient vector, determined by the system's linear operating range. Once the constraints are in place, the system can be... These updated matched filter coefficients are used to construct the subsequent impulse response function.

[0185] The aforementioned optical fiber monitoring and data analysis method collects current status data of the optical fiber link and uses a deep reinforcement learning agent to dynamically generate adaptive pulse coding parameters and matched filter parameters based on this data, thereby generating and injecting coded pulses into the optical fiber link. Subsequently, echo signals are received, and the generated matched filter parameters are used to perform matched filtering on the signals to obtain filtered output signals. Then, performance index vectors and reward signals are calculated, and finally, the reward signal is used to update the agent's policy network. This achieves continuous optimization of pulse coding and matched filtering parameters, enabling the system to autonomously adapt to the dynamic changes of the optical fiber link, effectively improving spatial resolution, dynamic range, and signal-to-noise ratio, while enhancing the system's adaptability and robustness.

[0186] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0187] Based on the same inventive concept, this application also provides an apparatus for implementing the optical cable monitoring and data analysis method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the optical cable monitoring and data analysis apparatus provided below can be found in the limitations of the optical cable monitoring and data analysis method described above, and will not be repeated here.

[0188] In one exemplary embodiment, such as Figure 3 As shown, an optical cable monitoring and data analysis device 30 is provided to implement the methods in the above-described method embodiments. The device includes:

[0189] The intelligent state perception module 31 is used to collect the current state data of the optical fiber link. Based on the current state data, it generates pulse coding parameters and matched filter parameters through a deep reinforcement learning agent. The current state data includes signal features extracted from the echo signal, environmental parameters provided by environmental sensors, and historical performance indicators. The pulse coding parameters are used to define the time-frequency characteristics of the pulse sequence.

[0190] The pulse injection execution module 32 is used to generate coded pulses according to pulse coding parameters and inject the coded pulses into the optical fiber link.

[0191] The echo signal filtering module 33 is used to receive the echo signal from the optical fiber link, and to perform matched filtering on the echo signal using matched filter parameters to obtain the filtered output signal.

[0192] The performance evaluation and reward generation module 34 is used to calculate the performance index vector based on the filtered output signal and to calculate the reward signal based on the performance index vector.

[0193] The policy adaptive optimization module 35 is used to update the policy network of the deep reinforcement learning agent using reward signals.

[0194] Embodiments of this application also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the aforementioned method embodiments.

[0195] Embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0196] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0197] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for monitoring and analyzing optical cables, characterized in that, The method includes: S1. Collect the current state data of the fiber optic link, and based on the current state data, generate pulse coding parameters and matched filter parameters through a deep reinforcement learning agent; wherein, the current state data includes signal features extracted from the echo signal, environmental parameters provided by environmental sensors, and historical performance indicators; the pulse coding parameters are used to define the time-frequency characteristics of the pulse sequence; S2. Generate an encoded pulse according to the pulse coding parameters and inject the encoded pulse into the optical fiber link; S3. Receive the echo signal from the optical fiber link, and perform matched filtering on the echo signal using the matched filter parameters to obtain a filtered output signal; S4. Calculate the performance index vector based on the filtered output signal, and calculate the reward signal based on the performance index vector; S5. Update the policy network of the deep reinforcement learning agent using the reward signal.

2. The method according to claim 1, characterized in that, The step of generating pulse coding parameters and matched filter parameters through a deep reinforcement learning agent based on the current state data includes: S11. Normalize the current state data to obtain a normalized state vector; input the normalized state vector into the policy network of the deep reinforcement learning agent, and calculate the action probability distribution through forward propagation; S12. Sample according to the action probability distribution to generate an initial action vector; wherein, the initial action vector includes pulse start frequency, pulse bandwidth, pulse width and frequency modulation slope; S13. Adjust the over-limit parameters according to the laser's tuning range and the system safety threshold, and perform constraint verification on the initial action vector to obtain a compliant action vector; S14. Extract the pulse coding parameters and the matched filter parameters from the compliant action vector; wherein, the matched filter parameters include matched filter coefficients used to define the impulse response of the matched filter.

3. The method according to claim 2, characterized in that, The step of performing matched filtering on the echo signal using the matched filter parameters to obtain a filtered output signal includes: S21. Construct the impulse response function of the matched filter based on the matched filter coefficients in the matched filter parameters; S22. The echo signal is preprocessed to obtain an aligned echo signal; wherein, the preprocessing includes noise reduction and time alignment; S23. Perform a convolution operation between the aligned echo signal and the impulse response function to obtain the initial filtered output; S24. Calculate the signal-to-noise ratio estimate by performing peak detection and background noise estimation on the initial filtered output; S25. Adjust the stride and window function of the convolution operation based on the signal-to-noise ratio estimate to obtain the adjusted convolution operation parameters; based on the adjusted convolution operation parameters, re-perform the convolution operation on the aligned echo signal and the impulse response function to obtain the optimized filtered output signal.

4. The method according to any one of claims 1 to 3, characterized in that, The calculation of the performance index vector based on the filtered output signal includes: S31. Extract the peak power and noise variance from the filtered output signal, and calculate the signal-to-noise ratio (SNR) based on the peak power and the noise variance; the formula for calculating the SNR is: Wherein, SNR represents the signal-to-noise ratio. Let be the filtered output signal at time t. The peak power of the filtered output signal is represented by t, where t is time. The noise variance is represented and calculated from the signal background region; S32. Analyze the pulse-compressed main lobe width from the filtered output signal; based on the main lobe width, combined with the speed of light and the refractive index of the fiber, calculate the spatial resolution; the formula for calculating the spatial resolution is: in, The spatial resolution is represented by c, the speed of light is represented by n, the refractive index of the optical fiber is represented by B, and the pulse bandwidth is represented by B. S33. The signal-to-noise ratio and the spatial resolution are standardized to obtain the standardized performance index vector.

5. The method according to claim 3, characterized in that, Before constructing the impulse response function of the matched filter based on the matched filter coefficients in the matched filter parameters, the method further includes: S41. Based on the latest status data of the optical fiber link and the performance index vector, calculate the residual dispersion error, and calculate the error signal according to the residual dispersion error; wherein, the error signal represents the difference between the expected output and the actual filtered output; S42. Update the filter coefficients using the least mean square algorithm based on the error signal and the echo signal; the expression for the least mean square algorithm is: in, This represents the updated filter coefficient vector at the (n+1)th iteration. This represents the current filter coefficient vector at the nth iteration. This represents the adaptive step size parameter used to control the update magnitude. This represents the error signal at the nth iteration. This represents the input echo signal vector at the nth iteration.

6. An optical cable monitoring and data analysis device, used to implement the method according to any one of claims 1 to 5, characterized in that, The device includes: The intelligent state perception module is used to collect the current state data of the optical fiber link, and based on the current state data, generates pulse coding parameters and matched filter parameters through a deep reinforcement learning agent; wherein, the current state data includes signal features extracted from the echo signal, environmental parameters provided by environmental sensors, and historical performance indicators; the pulse coding parameters are used to define the time-frequency characteristics of the pulse sequence; A pulse injection execution module is used to generate an encoded pulse according to the pulse encoding parameters and inject the encoded pulse into the optical fiber link; An echo signal filtering module is used to receive echo signals from the optical fiber link, and to perform matched filtering on the echo signals using the matched filter parameters to obtain a filtered output signal. The performance evaluation and reward generation module is used to calculate a performance index vector based on the filtered output signal and to calculate a reward signal based on the performance index vector. The policy adaptive optimization module is used to update the policy network of the deep reinforcement learning agent using the reward signal.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images