New energy power supply actual measurement modeling system based on reinforcement learning actual measurement data preprocessing

Abstract

The invention provides a new energy power supply actual measurement modeling system based on reinforcement learning actual measurement data preprocessing, which comprises an actual measurement data preprocessing subsystem, a parameter identification subsystem, a simulation verification subsystem, a deviation calculation and model verification subsystem and a new energy power supply actual measurement model output subsystem, the processing subsystem based on reinforcement learning eliminates abnormal data points by using a Q-learning algorithm, and performs preliminary parameter extraction in combination with a recursive least square module; the parameter identification subsystem based on the BP neural network can accurately identify fault ride-through control parameters, an actual measurement model output after simulation verification and deviation correction can cover complex working conditions, and a high-precision and high-reliability model support is provided for safe and stable grid connection of a new energy power supply, so that the fault ride-through control parameters can be accurately identified. The method has the advantages that abnormal data can be efficiently removed, the model precision and the multi-working-condition adaptability are improved, and then the accuracy, reliability and adaptability of new energy power supply actual measurement modeling are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of new energy power source modeling and parameter identification, and particularly relates to a new energy power source measured modeling system based on reinforcement learning measured data preprocessing. BACKGROUND

[0002] In the field of new energy power source modeling and parameter identification, the accuracy of measured data and the precision of parameter identification directly affect the reliability of the model. During the fault ride-through process of the new energy power source, the measured data is easily disturbed by various factors, and data quality problems such as deviation control model curve abnormal points and reactive current out-of-limit value points often occur. The traditional data preprocessing method mainly relies on manual experience or simple threshold judgment, and it is difficult to efficiently identify and eliminate such abnormal data, resulting in a large error in the subsequent parameter identification result.

[0003] In the existing parameter identification technology, the recursive least squares method can realize the preliminary extraction of parameters, but the identification precision is insufficient when dealing with nonlinear parameters under complex working conditions. Although the BP neural network has strong nonlinear fitting capability, its performance is highly dependent on the quality of input data, and when the original data contains noise or outliers, it will significantly affect the identification effect of the neural network. In addition, the traditional modeling process usually adopts an open-loop verification method, and lacks effective closed-loop simulation verification and deviation correction mechanism, resulting in poor adaptability of the finally established model under multiple working conditions, and it is difficult to meet the demand of high-precision modeling for the safe and stable grid connection of new energy power sources.

[0004] Therefore, how to improve the accuracy, reliability and adaptability of the measured modeling of the new energy power source is a technical problem to be solved by those skilled in the art. SUMMARY

[0005] The present application provides a new energy power source measured modeling system based on reinforcement learning measured data preprocessing, to solve the defects of low accuracy, poor reliability and poor adaptability of the measured modeling of the new energy power source in the prior art.

[0006] In one aspect, the present application provides a new energy power source measured modeling system based on reinforcement learning measured data preprocessing, which comprises: a measured data preprocessing subsystem for processing and preliminary parameter identification of the measured data of the fault ride-through of the new energy power source, to obtain sample data; a parameter identification subsystem based on BP neural network for receiving the sample data and identifying the control parameters of the fault ride-through of the new energy power source; a simulation verification subsystem for receiving the control parameters, performing simulation experiments by using the built simulation loop of the fault ride-through of the new energy power source, and collecting simulation data; a deviation calculation and model verification subsystem, configured to calculate deviation of the measured data from the simulation data and verify whether the deviation is less than a preset standard deviation; a new energy power supply measured model output subsystem, configured to output a new energy power supply measured model when the deviation is less than the preset standard deviation; The deviation calculation and model verification subsystem is further configured to feed back a verification result to the BP neural network-based parameter identification subsystem for parameter optimization when the deviation is greater than or equal to the preset standard deviation, and re-perform simulation verification until the deviation is less than the preset standard deviation.

[0007] According to the new energy power supply measured modeling system based on reinforcement learning measured data preprocessing provided by the application, the measured data preprocessing subsystem comprises: a reinforcement learning-based measured data processing subsystem, configured to eliminate abnormal value points of a deviation control model curve and out-of-limit value points of a reactive current in the measured data, to obtain processed data; a recursive least square-based parameter identification module, configured to perform preliminary parameter identification on the processed data, to obtain the sample data.

[0008] According to the new energy power supply measured modeling system based on reinforcement learning measured data preprocessing provided by the application, the reinforcement learning-based measured data processing subsystem comprises: an initialized Q table module, configured to initialize a Q table and set Q values of all state-action pairs as initial values; a selected processed measured data action module, configured to select an action by using an epsilon-greedy strategy, including randomly selecting an action with a probability of epsilon and selecting an action with a maximum current Q value with a probability of 1-epsilon; an executed action recorded result module, configured to execute the selected action and observe instant rewards and a new state fed back by an environment; a Q value updated module, configured to update Q values according to a preset calculation formula; a repeated iteration module, configured to repeatedly execute action selection and Q value updating until the Q table converges or a termination condition is reached; a measured data processing optimal strategy formed module, configured to output an optimal data processing strategy based on the converged Q table, to eliminate abnormal points of a deviation control model curve and out-of-limit value points of a reactive current in the measured data.

[0009] According to the new energy power supply measured modeling system based on reinforcement learning measured data preprocessing provided by the application, the preset calculation formula is:

[0010] wherein, is a learning rate, controlling an influence degree of new information; is a discount factor that controls the current value of future rewards; is the maximum Q-value of all possible actions in the new state , representing the maximum expectation of future rewards; is the value actually experienced, is the immediate reward obtained from the environment after performing action at time step .

[0011] The application provides a new energy power source actual measurement modeling system based on reinforcement learning actual measurement data preprocessing. The initial value of the recursive least square method is set, and initial response values and initial independent variable vectors corresponding to the processing data are input. The response value at the current moment and the independent variable vector at the current moment are sampled. The parameter estimation value vector is updated by using a pre-recursion calculation formula. The sampling and parameter updating are cyclically performed until a stop identification condition is met, and the sample data are obtained.

[0012] The application provides a new energy power source actual measurement modeling system based on reinforcement learning actual measurement data preprocessing. In the reinforcement learning process, if the iteration number is greater than or equal to a preset number, the initial value of the covariance matrix of the recursive least square method is set as a first preset value; if the iteration number is less than the preset number, the initial value of the covariance matrix of the recursive least square method is set as a second preset value; the first preset value is greater than the second preset value. In the reinforcement learning process, if the maximum Q-value change rate in the Q table of the actual measurement data processing subsystem is less than or equal to a preset convergence threshold, the parameter estimation value vector of the recursive least square method is set as the final output vector of the least square at the last recursion; if the maximum Q-value change rate is greater than the preset convergence threshold, the parameter estimation value vector of the recursive least square method is set as a zero vector or a sufficiently small positive real vector.

[0013] The application provides a new energy power source actual measurement modeling system based on reinforcement learning actual measurement data preprocessing.

[0014] wherein, is a gain matrix, is the parameter estimation value vector at the kth iteration, is the parameter estimation value vector at the (k-1) th iteration, is a covariance matrix, is the independent variable vector corresponding to the kth measured data, is the response value corresponding to the kth measured data.

[0015] According to the new energy power supply measured modeling system based on the measured data preprocessing of the reinforcement learning provided by the application, the parameter identification subsystem based on the BP neural network comprises: A forward propagation data input module is used for inputting the sample data to the neural network input layer. An input data weighting and nonlinear transformation module is used for weighting and nonlinear transformation of the input layer data and transmission to the next layer. An output result calculation module is used for calculating the prediction result of the neural network output layer. An output result and actual value error calculation module is used for calculating the error value of the prediction result and the actual value of the measured data. A backward propagation gradient calculation module is used for calculating the gradient of the weight and bias layer by layer according to the error value through the chain rule. A weight and bias adjustment module is used for adjusting the weight and bias according to the gradient to reduce the error. A parameter identification result optimization module is used for optimizing the control parameters when the error value is greater than the allowable error, and outputting to the simulation verification subsystem when the error value is less than or equal to the allowable error.

[0016] According to the new energy power supply measured modeling system based on the measured data preprocessing of the reinforcement learning provided by the application, the simulation verification subsystem comprises: A simulation working condition configuration module is used for configuring the same working condition as the measured experiment, including symmetric fault, asymmetric fault, low penetration working condition and high penetration working condition of high power, medium power and small power. A new energy power supply fault ride-through simulation loop module is used for building a simulation loop corresponding to the working condition. A simulation data acquisition module is used for executing simulation experiments and collecting simulation data.

[0017] According to the new energy power supply measured modeling system based on the measured data preprocessing of the reinforcement learning provided by the application, the deviation calculation and model verification subsystem comprises: A deviation calculation module is used for calculating the deviation of the measured data and the simulation data. A parameter identification result verification module is used for verifying whether the deviation is less than the preset standard deviation. If the deviation is less than the preset standard deviation, triggering the new energy power supply measured model output subsystem to output, if the deviation is greater than or equal to the preset standard deviation, triggering parameter optimization.

[0018] The new energy power supply test modeling system based on reinforcement learning-based measured data preprocessing provided by this invention utilizes reinforcement learning to preprocess measured data, BP neural network parameter identification, and closed-loop simulation verification mechanisms. The reinforcement learning-based processing subsystem employs the Q-learning algorithm to remove outlier data points, and combines this with a recursive least squares module for preliminary parameter extraction. The BP neural network-based parameter identification subsystem accurately identifies fault ride-through control parameters. Combined with simulation verification and deviation correction, the output measured model can cover complex operating conditions, providing high-precision and high-reliability model support for the safe and stable grid connection of new energy power supplies. This system offers advantages such as efficient removal of outlier data, improved model accuracy, and multi-condition adaptability, thereby enhancing the accuracy, reliability, and adaptability of new energy power supply test modeling. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of the new energy power supply test modeling system based on reinforcement learning test data preprocessing provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] In existing technologies, the modeling and parameter identification of new energy power sources has long faced the problem of model reliability due to poor quality of measured data. Traditional data preprocessing methods are insufficient to efficiently remove outliers in the deviation control model curves and reactive current exceeding limits. The recursive least squares method lacks accuracy in identifying nonlinear parameters under complex operating conditions. Backpropagation neural networks are susceptible to noisy data and lack a closed-loop simulation verification mechanism, resulting in poor model adaptability under multiple operating conditions. For example, during fault ride-through of new energy power sources, measured data is easily disturbed and generates outliers. Traditional methods cannot effectively identify and handle these outliers, leading to the accumulation of subsequent parameter identification errors, and ultimately the model cannot accurately reflect the actual operating characteristics.

[0023] To solve the above problems, it is found through research that improving data preprocessing efficiency and parameter identification accuracy is the key to breaking through the bottleneck of existing technology. First, the traditional abnormal data elimination method relies on manual experience, which is inefficient and difficult to deal with dynamic data characteristics, so an adaptive mechanism needs to be introduced. Reinforcement learning algorithm can dynamically identify and process abnormal data points through the interaction between agent and environment. Second, although recursive least squares method can achieve preliminary parameter extraction, it is not accurate enough when facing nonlinear relationship, so it needs to combine the nonlinear fitting ability of BP neural network. Finally, the traditional modeling process lacks a closed-loop verification link, which leads to insufficient model adaptability, so a simulation verification and deviation correction mechanism needs to be established to form an iterative optimization process. Based on this, the reinforcement learning and recursive least squares method are combined for data preprocessing, the BP neural network is used for parameter identification, and the closed-loop verification is used to optimize the model.

[0024] Specifically, Figure 1 is a structural schematic diagram of a new energy power source measured modeling system based on reinforcement learning measured data preprocessing provided by the embodiment of the present application. As Figure 1 shown, the new energy power source measured modeling system based on reinforcement learning measured data preprocessing can include a measured data preprocessing subsystem, a parameter identification subsystem based on a BP neural network, a simulation verification subsystem, a deviation calculation and model verification subsystem, a new energy power source measured model output subsystem 20, and each subsystem is connected in turn to form a closed-loop optimization process. The measured data is input into the parameter identification module after preprocessing, and the simulation verification result triggers parameter optimization iteration through deviation calculation until the output model meets the accuracy requirement.

[0025] Among them, the measured data preprocessing subsystem refers to a module that processes raw data through algorithms to improve data quality, which can use reinforcement learning algorithm to dynamically identify and eliminate abnormal data points, for example, through Q-learning algorithm to train agent to select optimal data processing action. The parameter identification subsystem based on BP neural network refers to a module that uses neural network for nonlinear parameter identification, which can adjust network weights through forward propagation and back propagation to achieve parameter optimization. The simulation verification subsystem refers to a module that builds a simulation environment to verify the accuracy of the model, which can configure various power levels and fault types of experimental conditions. The deviation calculation and model verification subsystem refers to a module that calculates the difference between measured and simulated data, which can determine whether to trigger parameter optimization through a preset threshold. The new energy power source measured model output subsystem 20 refers to a module that outputs the final qualified model, which can generate a usable model when the deviation meets the requirements, that is, output a usable model when the model verification meets the requirements, and trigger the parameter optimization process again for identification and simulation when the model verification does not meet the requirements, until the final model is output after the deviation meets the requirements.

[0026] Specifically, the measured data first enters the preprocessing subsystem, and the abnormal points are dynamically removed through the reinforcement learning algorithm, and the preliminary parameter extraction is performed in combination with the recursive least square method. The processed data is input into the BP neural network for deep parameter identification, and the identification result is transmitted to the simulation verification subsystem for multi-condition simulation. The simulation data and the original measured data enter the deviation calculation module 18, and if the deviation exceeds the preset standard, the parameter optimization process is triggered to re-identify and simulate until the deviation meets the standard and the final model is output. Through the closed-loop iteration of data cleaning, parameter identification and simulation verification, the model accuracy is gradually improved.

[0027] Compared with the prior art, the existing method uses fixed rules to process abnormal data, which cannot adapt to dynamically changing data characteristics, while the present scheme realizes adaptive abnormal point removal through reinforcement learning. The existing parameter identification technology separately uses recursive least square method or BP neural network, which is difficult to consider both linear and nonlinear relationships, and the present scheme realizes complementary advantages through two-stage processing. The traditional modeling process lacks a verification feedback mechanism, and the present scheme continuously optimizes model parameters through closed-loop simulation verification to improve multi-condition adaptability.

[0028] Through the above technical scheme, the present application effectively solves the problem of large parameter identification error caused by abnormal data, and the reinforcement learning preprocessing improves the data quality, the recursive least square and BP neural network combination improves the identification accuracy, and the closed-loop verification mechanism ensures the accuracy of the model under various conditions, and finally outputs a high-reliability new energy power supply measured model.

[0029] It should be noted that the data transmission between the subsystems can adopt: triple verification mechanism: CRC verification, hash value verification, data type assertion; when the verification fails for 3 times in a row, the reinforcement learning module of the current processing stage is reinitialized.

[0030] Among them, the CRC verification is used for cyclic redundancy check to detect bit errors in data transmission. The hash value verification is used to compare the SHA-256 digest of the sending end and the receiving end to ensure that the data is not tampered with. The data type assertion is used to verify that the data format meets the preset specification (such as matrix dimension, floating point range). Reinitialization resets the Q table, state buffer, etc. of reinforcement learning, and clears the potential contaminated data.

[0031] In this embodiment, the triple verification mechanism can cover bit errors, tampering, and format errors, and the error detection rate is greatly improved. When the verification fails for 3 times in a row, the reinforcement learning module of the current processing stage is reinitialized, which can quickly isolate the fault node.

[0032] Continuing to refer to Figure 1, the measured data preprocessing subsystem can include a measured data processing subsystem based on reinforcement learning and a measured data preprocessing subsystem of a recursive least squares parameter identification module 7. The measured data processing subsystem based on reinforcement learning eliminates the deviation control model curve abnormal value points and reactive current out-of-limit value points in the measured data to obtain processed data; the recursive least squares parameter identification module 7 performs preliminary parameter identification on the processed data to obtain sample data.

[0033] The measured data processing subsystem based on reinforcement learning refers to a module that uses a reinforcement learning algorithm to filter and process measured data, and can specifically use a Q-learning algorithm to achieve this. Through the interaction between the agent and the environment, the subsystem forms an optimal data processing strategy and automatically identifies and eliminates abnormal data points. The recursive least squares parameter identification module 7 refers to a module that uses a recursive least squares method to estimate parameters, and can specifically use an online updating covariance matrix and parameter vector to achieve this. The module can dynamically adjust the parameter estimate value according to the real-time input processed data and gradually approximate the true parameter.

[0034] Specifically, the working process of the measured data preprocessing subsystem is divided into two stages. In the first stage, the measured data processing subsystem based on reinforcement learning dynamically generates a data processing strategy through a reinforcement learning algorithm to filter and eliminate abnormal points in the measured data. For example, during the fault ride-through process of a new energy power source, when a sudden change in the voltage or current signal is detected, the reinforcement learning agent determines whether the data point is an abnormal value based on historical experience and performs an elimination action. In the second stage, the recursive least squares parameter identification module 7 performs preliminary parameter estimation on the cleaned processed data, and gradually corrects the parameter estimate value by updating the covariance matrix and parameter vector online, and finally outputs sample data that meets the accuracy requirements.

[0035] Compared with the prior art, the traditional method usually uses a fixed threshold or a statistical method to eliminate abnormal values, which is difficult to adapt to the dynamic data characteristics under complex working conditions. However, the present scheme introduces a reinforcement learning algorithm, which can automatically adjust the data processing strategy according to real-time data characteristics, effectively solving the limitations of traditional methods in abnormal value identification. At the same time, compared with the batch least squares method, the recursive least squares method can realize online updating of parameters, avoiding the waste of computing resources caused by repeated data processing.

[0036] Through the above technical scheme, the present application can effectively improve the quality of measured data preprocessing, accurately eliminate abnormal data points generated during the fault ride-through process, and provide high-precision sample data for subsequent parameter identification. Through the combination of reinforcement learning and recursive least squares, the intelligence of abnormal value elimination is guaranteed, and the real-time performance of parameter estimation is realized, significantly improving the accuracy and efficiency of new energy power source modeling.

[0037] In some embodiments, the present application further proposes a measured data processing subsystem based on reinforcement learning, comprising an initialization Q table module 1 for initializing the Q table and setting the Q value of all state-action pairs to an initial value; a selected processing measured data action module 2 for selecting an action using an ε-greedy strategy, including randomly selecting an action with a probability of ε and selecting an action with the largest current Q value with a probability of 1-ε; an executed action record result module 3 for executing the selected action and observing the immediate reward and new state of the environmental feedback; a Q value update module 4 for updating the Q value according to a pre-designed formula; a repeated iteration module 5 for repeatedly executing action selection and Q value updating until the Q table converges or a termination condition is reached; and a measured data processing optimal strategy forming module 6 for outputting an optimal data processing strategy based on the converged Q table to eliminate deviation control model curve abnormal points and reactive current out-of-limit value points in the measured data.

[0038] The initialization Q table module 1 refers to establishing a state-action value mapping table for the reinforcement learning agent, which can be implemented by a two-dimensional array or a hash table data structure, and by initializing the Q value to zero or a specific value, a benchmark is provided for subsequent action selection. The selected processing measured data action module 2 refers to balancing exploration and utilization through the ε-greedy strategy, for example, when ε is set to 0.1, the system has a 10% probability of randomly trying a new action to avoid local optimization, and a 90% probability of selecting the current optimal action. The executed action record result module 3 refers to obtaining environmental feedback after performing data point elimination operations, for example, when an abnormal point is correctly eliminated, a positive reward is obtained, and when a valid data point is mistakenly deleted, a negative reward is obtained. The Q value update module 4 updates the Q table using the time difference algorithm, for example, by adjusting the learning rate and the discount factor to control the weight of short-term and long-term returns. The repeated iteration module 5 terminates training by setting the maximum number of iterations or the Q value change threshold, for example, when the Q value change amount of 10 consecutive iterations is less than 1×10 -5 The measured data processing optimal strategy forming module 6 converts the converged Q table into a data processing rule, for example, when a voltage drop condition is detected, the reactive current threshold filtering action is preferentially executed.

[0039] Specifically, the measured data processing process first establishes the association matrix of the state space and the action space through the initialization Q table module 1. The state can be defined as the voltage deviation rate, the over-limit degree of reactive current and other characteristic combinations of the data points, and the action includes retention, rejection or marking for verification and the like. The processing measured data action module 2 selects the action corresponding to the ε-greedy strategy from the Q table according to the current state at each data processing decision, for example, in the voltage surge working condition, if the current state corresponds to a high probability of over-limit of reactive current, the rejection action is preferentially selected. The action execution record result module 3 obtains the environmental feedback after the action is executed, for example, when the successful rejection causes the abnormal point to reduce the parameter identification error, the system returns a +5 reward value; when the normal data point is mistakenly deleted, causing the subsequent identification error to increase, a-3 penalty value is returned. The Q value updating module 4 adjusts the current Q value according to the immediate reward and the maximum Q value of the next state, for example, when the learning rate is set to 0.2, the new Q value will retain 80% of the historical experience and absorb 20% of the new experience. The repeated iteration module 5 makes the Q table gradually approach the optimal strategy through cyclic training, for example, after 500 times of complete data set training, the Q value distribution of each state-action pair in the Q table tends to be stable. The finally formed optimal strategy for measured data processing can automatically identify and process various abnormal modes, for example, in the frequency fluctuation scene, the burr noise point in the control curve can be accurately identified.

[0040] Compared with the prior art, the traditional data preprocessing method relies on fixed threshold or manual rule for abnormal point rejection, which is difficult to adapt to the dynamically changing abnormal mode in the fault ride-through process of new energy power supply. The scheme realizes the dynamic optimization of the strategy through reinforcement learning, for example, automatically adjusts the reactive current filtering threshold in the low voltage ride-through working condition, and optimizes the control curve smoothness judgment standard in the high voltage ride-through. The static data processing rule in the prior art is easy to cause effective data loss or abnormal point residue, and the scheme balances the data integrity and accuracy through the reward mechanism.

[0041] Through the above technical scheme, the adaptive processing of the measured data of the fault ride-through of the new energy power supply is realized, and the problem of insufficient abnormal mode recognition ability of the traditional method is effectively solved. The dynamic strategy generated through reinforcement learning can accurately distinguish the real fault characteristics and measurement noise, for example, in the symmetric fault working condition, the key transient response data is retained while the sensor false alarm point is rejected. The scheme reduces the demand for manual intervention and improves the input data quality of the subsequent parameter identification subsystem, providing a reliable data basis for establishing a high-precision new energy power supply model.

[0042] In some embodiments, the pre-designed formula described above can refer to formula (1): (1) wherein, is the learning rate, controlling the influence degree of new information; is a discount factor that controls the current value of future rewards; is the maximum Q-value of all possible actions in the new state , representing the maximum expectation of future rewards; is the actual experienced value, is the immediate reward obtained from the environment after performing action at time step .

[0043] In a specific implementation, the learning rate refers to the proportion of new information in the current Q-value update, which can be implemented by a fixed value or a dynamically decaying value, for example, set to 0.8 in the initial stage to accelerate learning, and adjusted to 0.2 in the later stage to stabilize convergence. The discount factor refers to the discounting proportion of future rewards, used to balance current and long-term rewards. The immediate reward refers to the quantitative evaluation value of the environment feedback after performing action at time step , i.e., the immediate reward obtained from the environment, which can be calculated by a pre-set reward function, for example, assigning a positive reward when successfully removing an outlier point, and assigning a negative reward when mistakenly deleting normal data.

[0044] Specifically, the pre-designed formula is applied to the Q-value update process of the Q-learning algorithm. In the actual data processing stage, the agent selects action (e.g., retaining or removing data points) according to the current state , obtains the immediate reward after performing the action, and enters a new state . By calculating the maximum expected Q-value in the new state, combined with the current reward , the Q-value of the original state-action pair is dynamically adjusted. The learning rate α controls the influence strength of new experience on the Q table, for example, when α = 0.5, the new Q-value is composed of 50% of the new calculation result and 50% of the original Q-value. The discount factor γ determines the discounting proportion of future rewards, for example, γ = 0.9 means that the reward value at the next moment is equivalent to 90% of the current moment. Through iterative updating, the Q table gradually converges to the optimal strategy, and finally forms a processing rule that can accurately identify abnormal data points.

[0045] Compared with the prior art, the traditional data preprocessing method mainly uses fixed threshold or static rule to remove abnormal points, which cannot dynamically adjust the processing strategy according to the data distribution. The present scheme uses the Q-value update mechanism of reinforcement learning to automatically optimize the abnormal point identification strategy according to the data processing effect. For example, when the reactive current out-of-limit value point presents new distribution characteristics, the agent can quickly adjust the action selection strategy through immediate reward feedback, while the traditional method needs to manually reset the threshold.

[0046] Through the technical solution, the application realizes adaptive optimization of the abnormal data processing strategy, solves the problem of missing detection or false deletion of abnormal points caused by rule solidification in the traditional method. The Q value update formula quantifies the immediate benefits and long-term value of the action, so that the data processing strategy can ensure the current data quality while considering the subsequent parameter identification requirements, and finally improve the effectiveness of the measured data preprocessing and the parameter identification accuracy.

[0047] In some embodiments, the application further proposes an implementation of a parameter identification module 7 based on recursive least squares, which sets the initial value of the recursive least squares method and inputs the initial response value and independent variable vector corresponding to the processed data, samples the response value and independent variable vector at the current time, updates the parameter estimation value vector using a pre-recursion calculation formula, and repeatedly executes sampling and parameter updating until the stop condition is met, and finally outputs the sample data.

[0048] Among them, the initial value setting of the recursive least squares method refers to assigning initial values to the covariance matrix and the parameter estimation value vector when the algorithm starts, which can be dynamically adjusted through the iteration number or Q table convergence state in the reinforcement learning process. For example, when the iteration number exceeds the preset threshold, the initial value of the covariance matrix can be set to a large value to speed up the convergence; when the maximum Q value change rate of the Q table is lower than the preset threshold, the parameter estimation value vector can inherit the last recursive result to maintain continuity.

[0049] Among them, sampling the response value and independent variable vector at the current time refers to extracting the input-output data pair at the current time in chronological order from the preprocessed measured data. For example, in the new energy power fault ride-through experiment, the independent variable vector can include electrical quantities such as voltage and current, and the response value can correspond to the measured results of the control parameters.

[0050] Among them, the pre-recursion calculation formula updates the parameter estimation value vector, which dynamically adjusts the parameter estimation value through the gain matrix. Specifically, the standard calculation formula of the recursive least squares method can be used to combine the estimation value at the previous time with the current data to generate a new parameter estimation result. For example, the gain matrix is dynamically calculated through the product of the covariance matrix and the independent variable vector, which is used to correct the deviation of the parameter estimation value.

[0051] Specifically, the implementation process of the module includes the following steps: firstly, setting the covariance matrix initial value according to the reinforcement learning state, for example, when the number of iterations reaches a preset number, the covariance matrix initial value is set to a first preset value to speed up the convergence; then input the initial response value and the independent variable vector, for example, the initial response value can be taken from the first record of the preprocessed measured data; then sample the response value and the independent variable vector of the current time in chronological order, for example, in the new energy power fault ride-through experiment, voltage drop data is collected every millisecond; then calculate the gain matrix using the recursive formula, update the covariance matrix and the parameter estimation value vector, for example, by matrix operation, the new data is included in the parameter estimation process; finally, the above steps are executed in a loop until the change of the parameter estimation value is less than a preset threshold or the maximum number of iterations is reached, at which time the stable sample data is output for subsequent neural network training.

[0052] Compared with the prior art, the traditional recursive least squares method usually uses a fixed initial value, which leads to insufficient convergence speed or poor stability when the data quality fluctuates or the working condition is complex. However, the present scheme dynamically adjusts the initial value by combining reinforcement learning state, for example, selects the covariance matrix initial value size according to the convergence of the Q table, so that the parameter identification process can adapt to different data states and improve the algorithm robustness. In addition, by inheriting the historical recursive results as the initial value, the problem of parameter jump caused by data segmentation processing can be avoided.

[0053] Through the above technical scheme, the present application realizes efficient parameter identification of new energy power fault ride-through measured data, solves the problem of slow convergence speed and low identification accuracy caused by improper initial value setting of traditional methods. By dynamically adjusting the initial value and the recursive update mechanism, the stability of parameter estimation can be maintained when the data quality is unstable or the working condition is complex, providing high-quality sample data for subsequent neural network training, thereby improving the reliability and accuracy of the overall modeling system.

[0054] In some embodiments, the present application further sets the initial value of the recursive least squares method, including in the reinforcement learning process, if the number of iterations is greater than or equal to a preset number, setting the initial value of the covariance matrix of the recursive least squares method to a first preset value; if the number of iterations is less than the preset number, setting the initial value of the covariance matrix of the recursive least squares method to a second preset value; the first preset value is greater than the second preset value; in the reinforcement learning process, if the maximum Q value change rate in the Q table of the measured data processing subsystem is less than or equal to a preset convergence threshold, setting the parameter estimation value vector of the recursive least squares method to the final output vector of the least squares at the last recursion; if the maximum Q value change rate is greater than the preset convergence threshold, setting the parameter estimation value vector of the recursive least squares method to a zero vector or a sufficiently small positive real vector.

[0055] The initial value of the covariance matrix refers to a matrix used to adjust the update step of the parameter estimation in the recursive least squares method, and can be initialized by different preset values to balance the convergence speed and stability of the algorithm. When the number of iterations reaches a preset number, a larger initial value can accelerate the convergence of the parameters; when the number of iterations is insufficient, a smaller initial value can prevent the parameter estimation from falling into a local optimum too early. The setting of the parameter estimation value vector refers to the initial estimation value of the parameter to be identified in the recursive least squares method, and can be dynamically adjusted according to the convergence state of the Q table in the reinforcement learning process. If the Q table has converged, the historical optimal parameter estimation value (the final output vector of the least squares at the last recursion) is used as the initial value to inherit the optimization results of the previous period; if the Q table has not converged, a zero vector or a small vector is used as the initial value to avoid the transmission of incorrect parameters.

[0056] Specifically, before the recursive least squares method is started, it is first determined whether the number of iterations of the reinforcement learning process reaches a preset number. If it does, the initial value of the covariance matrix is set to a larger first preset value, for example, 1000, to enhance the adaptability of the algorithm to the current data; if it does not, it is set to a smaller second preset value, for example, 10, to maintain the stability of the parameter update. At the same time, the rate of change of the maximum Q value in the Q table is monitored in real time. When the rate of change is lower than a preset convergence threshold, it indicates that the reinforcement learning has stabilized, and at this time the parameter estimation value vector of the recursive least squares method is assigned the final result of the last recursion, thereby inheriting the optimized parameter information; if the rate of change is higher than the threshold, the parameter estimation value vector is reset to a zero vector or a small vector to avoid interference of residual errors on the subsequent recursion process.

[0057] Compared with the prior art, the traditional recursive least squares method usually uses a fixed initial value, which cannot be dynamically adjusted according to the data processing stage, resulting in a decrease in parameter identification accuracy in the early stage of reinforcement learning or when the data is unstable. The present scheme dynamically adjusts the initial values of the covariance matrix and the parameter estimation value by correlating the number of iterations of reinforcement learning and the convergence state of the Q table, so that the recursive least squares method adaptively matches the optimal initialization strategy at different learning stages, avoiding convergence problems caused by improper initial value setting, and improving the continuity and accuracy of parameter identification.

[0058] Through the above technical scheme, the present application effectively solves the problem of rigid initial value setting of the traditional recursive least squares method in the dynamic data processing scene, and realizes intelligent adjustment of the initial value configuration in the parameter identification process through the cooperative optimization of the reinforcement learning state and the recursive algorithm, thereby significantly improving the stability and reliability of parameter estimation in the data preprocessing stage, and providing a high-quality sample data basis for subsequent modeling.

[0059] In some embodiments, the calculation formula of the recursive least squares method can refer to formula (2): (2) wherein, is a gain matrix, is a parameter estimation value vector at the kth iteration, is a parameter estimation value vector at the (k-1)th iteration, is a covariance matrix, is an independent variable vector corresponding to the kth measured data, is a response value corresponding to the kth measured data.

[0060] In one specific implementation process, the gain matrix refers to a matrix used to adjust the parameter update step, which can be specifically calculated by the product of the covariance matrix at the previous moment and the current independent variable vector, and its function is to dynamically adjust the parameter correction amount according to new data. The parameter estimation value vector refers to the current moment parameter estimation result obtained by recursive calculation, which is specifically updated by the product of the estimation value at the previous moment and the gain matrix on the prediction error, and its function is to gradually approach the true parameter value. The covariance matrix refers to a matrix reflecting the uncertainty of parameter estimation, which is specifically updated by the product of the covariance matrix at the previous moment, the gain matrix and the independent variable vector, and its function is to quantify the confidence of parameter estimation. The independent variable vector refers to a vector containing the input variable at the current moment, which can be specifically composed of voltage, current or power signals extracted from measured data, and its function is to establish the correlation between parameters and system response. The response value refers to the measured system output at the current moment, which can specifically include the fault ride-through dynamic response index of new energy power supply, and its function is to provide the target value for parameter identification.

[0061] Specifically, the calculation process of recursive least squares method is divided into three steps: first, the gain matrix is calculated according to the covariance matrix at the previous moment and the current independent variable vector, then the parameter estimation value at the previous moment is corrected by using the gain matrix, and finally the covariance matrix is updated for the next moment calculation. When receiving new measured data every time, the system inputs the current independent variable vector and the response value into the calculation formula, and updates the parameter estimation value vector in real time through matrix operation. This process gradually optimizes the parameter estimation result through continuous iteration until the preset convergence condition is met or the maximum iteration number is reached.

[0062] Compared with the prior art, the traditional recursive least square method adopts fixed values in initial value setting, cannot dynamically adjust the covariance matrix initial value according to the data processing state, and leads to the problems that the parameter identification process may have slow convergence speed or fall into local optimum. The scheme dynamically selects the covariance matrix initial value setting strategy through the iteration number and the Q table convergence state in the reinforcement learning process: when the iteration number is high or the Q table tends to be stable, a larger initial value is used to speed up the convergence; when the iteration number is low or the Q table has not converged, a smaller initial value is used to maintain the calculation stability. The dynamic adjustment mechanism overcomes the adaptability defect caused by the fixed initial value setting in the traditional method.

[0063] Through the above technical scheme, the present application realizes intelligent optimization configuration of the recursive least square method initial value, solves the parameter identification precision fluctuation problem caused by the fixed initial value setting in the traditional method. By associating the covariance matrix initial value with the state of the reinforcement learning process, noise interference can be suppressed in the early stage of data processing, and the convergence speed can be accelerated in the later stage of data processing, so that more stable parameter identification results can be obtained under the new energy power supply fault ride-through working condition. At the same time, the calculation formula avoids the defect that the traditional batch least square method needs to store all historical data through matrix recursion, and significantly reduces the consumption of computing resources.

[0064] In some embodiments, the present application further proposes a parameter identification subsystem based on BP neural network, comprising a forward propagation data input module 8 for inputting sample data to the neural network input layer; an input data weighting and nonlinear transformation module 9 for weighting and nonlinear transformation of the input layer data and passing to the next layer; an output result calculation module 10 for calculating the prediction result of the neural network output layer; an output result and actual value error calculation module 11 for calculating the error value of the prediction result and the actual value of the measured data; a back propagation gradient calculation module 12 for calculating the gradient of weight and bias layer by layer according to the error value through chain rule; a weight bias adjustment module 13 for adjusting the weight and bias according to the gradient to reduce the error; a parameter identification result optimization module 14 for optimizing the control parameters when the error value is greater than the allowed error, and outputting to the simulation verification subsystem when the error value is less than or equal to the allowed error.

[0065] The forward propagation data input module 8 refers to the process of inputting the preprocessed sample data into the input layer of the neural network. Specifically, matrix operation can be used to realize data transmission, ensuring the matching of input data and network structure. The input data weighting and nonlinear transformation module 9 refers to the operation of applying an activation function after linear weighting of the input data, such as using the Sigmoid function to realize nonlinear mapping, enhancing the fitting ability of the network to complex relationships. The output result calculation module 10 refers to the process of calculating the final output of the network through layer-by-layer transmission, such as calculating the predicted value through the weighted sum of the output layer nodes. The output result and actual value error calculation module 11 refers to quantifying the difference between the predicted value and the actual value through a loss function, such as using the mean square error function to evaluate the recognition accuracy. The back propagation gradient calculation module 12 refers to calculating the gradient of each layer parameter based on the error back propagation chain rule, such as using automatic differentiation technology to realize efficient calculation of the gradient. The weight bias adjustment module 13 refers to updating the network parameters using optimization algorithms, such as using the stochastic gradient descent method to adjust the weight matrix. The parameter identification result optimization module 14 refers to achieving error convergence by iteratively adjusting parameters, such as setting an error threshold as the iteration termination condition.

[0066] Specifically, the sample data is first input into the forward propagation data input module 8, and data reception is completed through the input layer nodes. The input data weighting and nonlinear transformation module 9 performs linear combination on the input data and applies an activation function to generate hidden layer output, which is executed layer by layer until reaching the output layer. The output result calculation module 10 generates control parameter prediction results based on the activation values of the output layer nodes, and the output result and actual value error calculation module 11 compares the prediction results with the measured data labels to calculate the error index. The back propagation gradient calculation module 12 calculates the partial derivative of each layer weight and bias according to the error value along the network in the reverse direction through the chain rule, and the weight bias adjustment module 13 updates the network parameters according to the product of the learning rate and the gradient value. When the error value exceeds the preset threshold, the parameter identification result optimization module 14 triggers the parameter re-adjustment process until the error converges within the allowed range.

[0067] Compared with the prior art, the traditional parameter identification method usually adopts a single forward propagation structure, lacks a dynamic error feedback mechanism, and leads to a decrease in recognition accuracy under abnormal data interference. The present scheme constructs a closed-loop optimization mechanism, automatically triggers parameter adjustment when the error exceeds the limit, and realizes directional correction of parameters combined with back propagation gradient calculation. The fixed learning rate parameter updating method in the prior art is prone to local optimum, and the present scheme can adapt to different parameter identification requirements under different working conditions by dynamically adjusting the weight matrix.

[0068] By the technical solution, the application effectively solves the parameter identification deviation problem of the traditional BP neural network caused by abnormal input data. By establishing a closed-loop feedback mechanism of error calculation and parameter optimization, the network parameter adjustment is automatically triggered when the control parameter prediction error is out of limit, which significantly improves the identification accuracy of nonlinear control parameters. The modular data processing flow realizes the decoupling of data transmission, error calculation and parameter optimization, enhances the adaptability of the system under multiple working conditions, and provides high-precision parameter input for subsequent simulation verification.

[0069] In a specific implementation process, the learning rate of the weight bias adjustment module 13 can be dynamically adjusted: a fixed learning rate is used in the initial training stage; when the error reduction rate is less than the preset reduction (such as 5%) for 3 consecutive complete training periods, the learning rate is decayed to a preset percentage (such as 50%) of the original value; when the deviation verification subsystem feedback does not meet the standard, the learning rate is reset to the initial value.

[0070] In the application, when the convergence stagnates (the reduction rate is less than 5%), the step size is automatically reduced, which can break through the local optimum and improve the convergence speed. When the verification fails, only the learning rate is reset to the initial value, saving computing resources.

[0071] In some embodiments, the application further proposes a simulation verification subsystem including a simulation working condition configuration module 15 for configuring the same working conditions as the actual measurement experiment, including symmetric faults, asymmetric faults, low penetration working conditions and high penetration working conditions of high power, medium power and low power; a new energy power supply fault ride-through simulation loop module 16 for building a simulation loop corresponding to the working conditions; and a simulation data acquisition module 17 for executing simulation experiments and acquiring simulation data.

[0072] The simulation working condition configuration module 15 is a device for setting test conditions consistent with the real experimental environment, which can be realized by using a parameterized configuration tool, for example, by inputting parameters such as power level, fault type and voltage disturbance range to generate corresponding simulation scenes. Its role is to ensure that the simulation environment is completely matched with the actual measurement conditions, thereby avoiding verification errors caused by working condition differences.

[0073] The new energy power supply fault ride-through simulation loop module 16 is a device for building a power system simulation model, which can be realized by using the topology building function in the power system simulation software, for example, dynamically adjusting the grid topology, load characteristics and fault injection point according to the parameters generated by the working condition configuration module. Its role is to generate a simulation environment with consistent dynamic characteristics as the actual measurement experiment, providing a reliable basis for parameter verification.

[0074] The simulation data acquisition module 17 refers to a device for recording the change of electrical quantity during the simulation process, which can be realized by a data sampling device and a storage unit, for example, collecting voltage, current waveform and control signal time sequence data at a fixed frequency. Its role is to obtain the dynamic response data of the simulation system, and provide input for subsequent deviation calculation.

[0075] Specifically, the simulation working condition configuration module 15 first generates a working condition parameter set according to the power level and fault type of the measured experiment, for example, configures the same experimental working condition as the measured experiment, including symmetric fault, asymmetric fault, low penetration four working conditions (0%, 20%, 50%, 80%), high penetration two working conditions (120%, 130%) disturbance experiment of large, medium and small power. The simulation data acquisition module 17 records the electrical quantity data of the key nodes in real time during the simulation process, for example, captures the inverter output current waveform at a millisecond level sampling frequency during the fault occurrence, and stores the data as a time sequence consistent with the format of the measured data.

[0076] Compared with the prior art, the traditional simulation verification method usually only verifies a single working condition, for example, only tests the symmetric fault under the rated power, which leads to the insufficient adaptability of the model under complex working conditions. While the scheme can comprehensively verify the effectiveness of the parameter identification result under different operating conditions by configuring a combination of multiple power levels and multiple fault types, for example, covering extreme scenarios such as large power low penetration and small power high penetration.

[0077] Through the above technical scheme, the present application solves the problem of single simulation verification covering working condition in the traditional modeling process, and ensures the applicability of the parameter identification result under complex power grid disturbance by constructing diversified simulation scenes consistent with the measured experiment, thereby improving the reliability of the new energy power supply fault ride-through model in the actual grid-connected environment.

[0078] In some embodiments, the present application further proposes a deviation calculation and model verification subsystem including a deviation calculation module 18 and a parameter identification result verification module 19. The deviation calculation module 18 is used to calculate the deviation between the measured data and the simulation data, and the parameter identification result verification module 19 is used to verify whether the deviation is less than a preset standard deviation. If the deviation is less than the preset standard deviation, the new energy power supply measured model output subsystem 20 is triggered to output, and if the deviation is greater than or equal to the preset standard deviation, the parameter optimization is triggered.

[0079] The deviation calculation module 18 refers to a functional unit for quantifying the difference between the measured data and the simulation data through mathematical operation, and can be specifically implemented by using a mean square error algorithm or an absolute error integral algorithm, and is used to generate a comparable numerical deviation index. The parameter identification result verification module 19 refers to a functional unit for logical judgment based on a preset threshold, and can be specifically implemented by using a comparator or a conditional judgment program, and is used to determine whether the parameter identification result meets the model accuracy requirement and to transmit the verification conclusion to the downstream module.

[0080] Specifically, the deviation calculation module 18 receives the original measured data from the measured data preprocessing subsystem and the simulation data from the simulation verification subsystem, and performs difference analysis on the two types of data at the point-by-point or statistical level through a preset error calculation rule. The parameter identification result verification module 19 compares the calculated deviation value with the preset standard deviation threshold, and if the deviation is lower than the threshold, triggers the model generation instruction of the new energy power source measured model output subsystem 20; if the deviation exceeds the threshold, sends a parameter optimization instruction to the parameter identification subsystem based on the BP neural network, and drives the system to enter the iterative optimization process until the accuracy requirement is met.

[0081] Compared with the prior art, the traditional modeling method usually generates a model only through single parameter identification and simulation verification, and lacks a closed-loop feedback mechanism. The deviation verification link in the prior art is a one-way process, and cannot dynamically adjust the parameter identification process according to the verification result. The present scheme forms a closed-loop control link of parameter optimization and model verification by constructing a double-module linkage mechanism of deviation calculation and verification, so that the model output can be subjected to multiple rounds of iterative optimization, significantly improving the parameter identification accuracy and model reliability.

[0082] Through the above technical scheme, the present application realizes automatic closed-loop control of the model verification link, and solves the problem of insufficient model accuracy caused by the lack of dynamic correction mechanism in the traditional method. The subsystem drives the parameter optimization cycle through real-time deviation feedback, ensures the adaptability of the output model under complex working conditions, effectively reduces the risk of model misalignment caused by parameter identification error, and provides reliable protection for the stability of new energy power source grid connection.

[0083] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0084] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A new energy power source measured modeling system based on reinforcement learning measured data preprocessing, characterized in that, The method comprises the following steps: a measured data preprocessing subsystem is configured to process and preliminarily identify the measured data of the fault ride-through of the new energy power source, and obtain sample data; a parameter identification subsystem based on a BP neural network is configured to receive the sample data and identify the control parameters of the fault ride-through of the new energy power source; a simulation verification subsystem is configured to receive the control parameters, perform simulation experiments by using a simulation loop of the fault ride-through of the new energy power source, and collect simulation data; a deviation calculation and model verification subsystem is configured to calculate the deviation between the measured data and the simulation data, and verify whether the deviation is less than a preset standard deviation; a new energy power source measured model output subsystem is configured to output a new energy power source measured model when the deviation is less than the preset standard deviation; wherein, the deviation calculation and model verification subsystem is further configured to feed back the verification result to the parameter identification subsystem based on the BP neural network for parameter optimization when the deviation is greater than or equal to the preset standard deviation, and perform simulation verification again until the deviation is less than the preset standard deviation.

2. The new energy power source measured data modeling system based on reinforcement learning measured data preprocessing according to claim 1, characterized in that, The measured data preprocessing subsystem comprises: a measured data processing subsystem based on reinforcement learning, which is configured to eliminate abnormal value points of the deviation control model curve and out-of-limit value points of the reactive current in the measured data, and obtain processed data; a parameter identification module based on recursive least squares, which is configured to preliminarily identify the processed data, and obtain the sample data.

3. The new energy power source measured modeling system based on reinforcement learning measured data preprocessing according to claim 2, characterized in that, The measured data processing subsystem based on reinforcement learning comprises: an initialization Q table module, which is configured to initialize a Q table and set the Q values of all state-action pairs to initial values; a selection processing measured data action module, which is configured to select an action by using an ε-greedy strategy, including randomly selecting an action with a probability of ε and selecting an action with a maximum current Q value with a probability of 1-ε; an execution action record result module, which is configured to execute the selected action and observe the instant reward and new state fed back by the environment; a Q value updating module, which is configured to update the Q values according to a preset calculation formula; a repeated iteration module, which is configured to repeatedly execute the action selection and Q value updating until the Q table converges or a termination condition is reached; a measured data processing optimal strategy forming module, which is configured to output an optimal data processing strategy based on the converged Q table, so as to eliminate the abnormal value points of the deviation control model curve and the out-of-limit value points of the reactive current in the measured data.

4. The new energy power source measured modeling system based on reinforcement learning measured data preprocessing according to claim 3, characterized in that, The preset calculation formula is as follows: where, is the learning rate, controlling the degree of influence of new information; is the discount factor, controlling the current value of future rewards; is the maximum Q-value of all possible actions in the new state represents the maximum expectation of future rewards; is the actual experienced value, is the immediate reward obtained from the environment after performing action at time step .

5. The new energy power source measured data modeling system based on reinforcement learning measured data preprocessing according to claim 2, characterized in that, The parameter identification module based on recursive least squares is specifically configured to: set initial values of the recursive least squares method, and input initial response values and initial independent variable vectors corresponding to the processed data; sample the response values at the current time and the independent variable vectors at the current time; update the parameter estimation value vector by using a pre-recursion calculation formula; perform the sampling and the parameter updating in a loop until a stop identification condition is met, and obtain the sample data.

6. The new energy power source measured data modeling system based on reinforcement learning measured data preprocessing according to claim 5, characterized in that, Setting the initial values of the recursive least squares method comprises: in the reinforcement learning process, if the number of iterations is greater than or equal to a preset number, setting the initial value of the covariance matrix of the recursive least squares method as a first preset value; if the number of iterations is less than the preset number, setting the initial value of the covariance matrix of the recursive least squares method as a second preset value; the first preset value is greater than the second preset value. In the reinforcement learning process, if a maximum Q value change rate in a Q table of the measured data processing subsystem is less than or equal to a preset convergence threshold, a parameter estimation value vector of the recursive least square method is set as a final output vector of the least square in the last recursion; if the maximum Q value change rate is greater than the preset convergence threshold, the parameter estimation value vector of the recursive least square method is set as a zero vector or a sufficiently small positive real vector.

7. The new energy power source measured modeling system based on reinforcement learning measured data preprocessing according to claim 5, characterized in that, A calculation formula of the recursive least square method is: wherein, is a gain matrix, is a parameter estimate vector at the kth iteration, is a parameter estimate vector at the (k-1)th iteration, is a covariance matrix, is an independent variable vector corresponding to the kth measured data, is a response value corresponding to the kth measured data.

8. The new energy power source measured data modeling system based on reinforcement learning measured data preprocessing according to claim 2, characterized in that, The parameter identification subsystem based on the BP neural network comprises: A forward propagation data input module for inputting the sample data to a neural network input layer; An input data weighting and nonlinear transformation module for weighting and nonlinearly transforming the input layer data and transmitting the same to a next layer; An output result calculation module for calculating a prediction result of a neural network output layer; An output result and actual value error calculation module for calculating an error value of the prediction result and an actual value of the measured data; A backward propagation gradient calculation module for calculating gradients of weights and biases layer by layer according to the error value through a chain rule; A weight and bias adjustment module for adjusting the weights and biases according to the gradients to reduce the error; A parameter identification result optimization module for optimizing the control parameters when the error value is greater than an allowable error until the error value is less than or equal to the allowable error, and outputting to the simulation verification subsystem. 9.The new energy power source measured data modeling system based on reinforcement learning measured data preprocessing according to claim 2, characterized in that, The simulation verification subsystem comprises: A simulation working condition configuration module for configuring the same working conditions as the measured experiment, including symmetric faults, asymmetric faults, low penetration working conditions and high penetration working conditions of high power, medium power and low power; A new energy power supply fault ride-through simulation loop module for building a simulation loop corresponding to the working conditions; A simulation data acquisition module for executing a simulation experiment and acquiring simulation data.

10. The new energy power source measured data modeling system based on reinforcement learning measured data preprocessing according to any one of claims 1-9, characterized in that, The deviation calculation and model verification subsystem comprises: A deviation calculation module for calculating a deviation of the measured data and the simulation data; A parameter identification result verification module for verifying whether the deviation is less than a preset standard deviation, if the deviation is less than the preset standard deviation, triggering a new energy power supply measured model output subsystem to output, if the deviation is greater than or equal to the preset standard deviation, triggering parameter optimization and re-simulation.

Images