A Performance Evaluation Method and System for Relay Protection Devices Based on Multi-channel Closed-Loop Testing

Abstract

This invention discloses a method and system for performance evaluation of relay protection devices based on multi-channel closed-loop testing, belonging to the field of electrical performance evaluation technology. The method includes: evaluating and determining the current operational risk coefficient based on the historical operating status records of the relay protection device under test within a historical time range; determining an appropriate channel selection scheme as an appropriate performance test scheme based on a preset test channel group and the current operational risk coefficient analysis; performing multi-dimensional performance tests on the relay protection device under test according to the appropriate performance test scheme to obtain multi-dimensional performance test data; and using a pre-built device performance evaluation engine, determining the performance evaluation result of the relay protection device under test based on the multi-dimensional performance test data analysis. This invention solves the technical problem of inaccurate performance evaluation results of relay protection devices in the prior art.

Description

Technical Field

[0001] This invention relates to the field of electrical performance technology, and more specifically to a method and system for evaluating the performance of relay protection devices based on multi-channel closed-loop testing. Background Technology

[0002] As core safety equipment in power systems, relay protection devices directly affect the stable operation and fault isolation efficiency of the power grid. With the large-scale integration of smart grids and new energy sources, power system operating conditions are becoming increasingly complex and variable, placing higher demands on the speed, selectivity, sensitivity, and reliability of relay protection devices. To ensure their proper functioning, comprehensive and effective performance testing and evaluation are necessary. In existing technologies, the testing and evaluation of relay protection devices typically employ periodic calibration or fixed-mode testing schemes. While these methods can verify the basic performance of the device under standard operating conditions, they can lead to limitations in the accuracy and reliability of the evaluation results in practical applications. Summary of the Invention

[0003] This application provides a method and system for evaluating the performance of relay protection devices based on multi-channel closed-loop testing, which is used to address the technical problem of inaccurate performance evaluation results of relay protection devices in the prior art.

[0004] In view of the above problems, this application provides a method and system for evaluating the performance of relay protection devices based on multi-channel closed-loop testing.

[0005] In a first aspect, this application provides a method for evaluating the performance of relay protection devices based on multi-channel closed-loop testing, the method comprising:

[0006] The current operational risk coefficient is determined by evaluating the historical operating status records of the relay protection device under test within a historical time range.

[0007] Based on the preset test channel group, the adaptation channel selection scheme is determined as the adaptation performance test scheme according to the current operating risk coefficient analysis, and the relay protection device under test is subjected to multi-dimensional performance test according to the adaptation performance test scheme to obtain multi-dimensional performance test data.

[0008] The performance evaluation results of the relay protection device under test are determined by analyzing the multidimensional performance test data using a pre-built device performance evaluation engine.

[0009] Secondly, this application provides a relay protection device performance evaluation system based on multi-channel closed-loop testing, including:

[0010] The risk acquisition module is used to assess and determine the current operational risk coefficient based on the historical operational status records of the relay protection device under test within a historical time range.

[0011] The multidimensional test data acquisition module is used to determine the adaptation channel selection scheme as the adaptation performance test scheme based on the current operating risk coefficient analysis, based on the preset test channel group, and to perform multidimensional performance tests on the relay protection device under test according to the adaptation performance test scheme to acquire multidimensional performance test data.

[0012] The performance evaluation result acquisition module is used to determine the performance evaluation result of the relay protection device under test based on the analysis of the multi-dimensional performance test data using a pre-built device performance evaluation engine.

[0013] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0014] This application proposes a method and system for evaluating the performance of relay protection devices based on multi-channel closed-loop testing. First, it intelligently assesses the current operational risk coefficient of the device under test based on its historical operating status records. Then, based on this risk coefficient, it dynamically analyzes and determines the appropriate channel selection scheme matching the risk level from a pre-set group of multiple test channels, thus constructing a customized multi-dimensional performance test scheme and executing the test to acquire data. Finally, it utilizes a device performance evaluation engine built based on ensemble learning to comprehensively analyze the multi-dimensional performance test data to obtain the final performance evaluation result. This significantly improves the accuracy, efficiency, and intelligence level of relay protection device performance evaluation. Compared with traditional methods, the technical solution provided in this application significantly overcomes the rigidity of fixed test modes and the singularity of evaluation models, achieving dynamic optimization of test resources and strategies. This improves the reliability of the final performance evaluation results and provides more scientific and accurate data support for condition-based maintenance, life prediction, and update decisions of relay protection devices. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the performance evaluation method for relay protection devices based on multi-channel closed-loop testing provided in this application embodiment.

[0017] Figure 2 This is a schematic diagram of the structure of a relay protection device performance evaluation system based on multi-channel closed-loop testing provided in an embodiment of this application.

[0018] The components represented by each number in the attached diagram are explained below:

[0019] The system includes a risk acquisition module 100, a multi-dimensional test data acquisition module 200, and a performance evaluation result acquisition module 300. Detailed Implementation

[0020] This application provides a method and system for evaluating the performance of relay protection devices based on multi-channel closed-loop testing, which addresses the technical problem of inaccurate performance evaluation results of relay protection devices in the prior art.

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0023] Example 1, as Figure 1 As shown, this application provides a method for evaluating the performance of relay protection devices based on multi-channel closed-loop testing, wherein the method includes:

[0024] S10: Determine the current operating risk coefficient based on the historical operating status records of the relay protection device under test within the historical time range.

[0025] In the existing operation, maintenance, and testing system of relay protection devices, traditional methods typically rely on fixed maintenance cycles or general operating years to arrange testing, failing to fully explore and utilize the massive amounts of status data generated by the devices themselves during long-term operation. This one-size-fits-all approach cannot identify the potential risk stratification caused by differences in operating environment, load history, and component aging rates among different devices.

[0026] Step S10 in the method provided in this application embodiment includes:

[0027] The historical operating status records of the relay protection device under test within the historical time range are screened to obtain the historical sampled value effective value sequence, historical load rate sequence, historical CPU load rate sequence, historical time deviation sequence, and historical harmonic distortion rate sequence.

[0028] The mean values ​​and index volatility analyses of the historical sampled value effective value sequence, historical load rate sequence, historical CPU load rate sequence, historical time deviation sequence, and historical harmonic distortion rate sequence are calculated respectively. The mean value of historical sampled value effective value, the coefficient of variation of historical sampled value effective value, the mean value of historical load rate, the coefficient of variation of historical load rate, the mean value of historical CPU load rate, the coefficient of variation of historical CPU load rate, the mean value of historical time deviation, the coefficient of variation of historical time deviation, the mean value of historical harmonic distortion rate, and the coefficient of variation of historical harmonic distortion rate are calculated and used as the set of operating status parameters.

[0029] The operational risk coefficient within a future preset time range is determined based on the set of operational status parameters and used as the current operational risk coefficient.

[0030] The determination of operational risk coefficients within a preset future timeframe based on the set of operational status parameters includes:

[0031] The risk prediction complexity is determined by weighted evaluation of the coefficient of variation of the effective value of the historical sampled values, the coefficient of variation of the historical load rate, the coefficient of variation of the historical CPU load rate, the coefficient of variation of the historical time deviation, and the coefficient of variation of the historical harmonic distortion rate. The risk prediction complexity is positively correlated with the coefficient of variation of the effective value of the historical sampled values, the coefficient of variation of the historical load rate, the coefficient of variation of the historical CPU load rate, the coefficient of variation of the historical time deviation, and the coefficient of variation of the historical harmonic distortion rate.

[0032] Based on the historical operation logs of similar relay protection devices, several sample operation status parameter sets were collected, and the proportion of abnormal operation events in different sample operation status parameter sets within the historical preset time range was used as the sample operation risk coefficient to obtain several sample operation risk coefficients.

[0033] Based on the aforementioned set of sample operating status parameters and the set of sample operating risk coefficients, Q sample training sets are obtained, and long short-term memory networks are trained to convergence respectively to construct Q operating risk assessment models, where Q is an integer greater than 20;

[0034] The ratio of the risk prediction complexity to the maximum historical risk prediction complexity recorded within the historical time range is multiplied by Q and rounded to obtain the number of suitable models selected, K, where K is greater than or equal to 1 and less than or equal to Q.

[0035] K operational risk assessment models are randomly selected from the Q operational risk assessment models, and operational risk assessments are performed on each model based on the operational status parameter set. The average value of the K assessment results is then used as the operational risk coefficient within the preset future time range.

[0036] In this embodiment of the application, the current operating risk coefficient is determined by evaluating the historical operating status records of the relay protection device under test within a historical time range.

[0037] Specifically, firstly, the historical operating status records of the relay protection device under test within a historical time range are filtered to obtain the historical sampled value valid value sequence, historical load rate sequence, historical CPU load rate sequence, historical time deviation sequence, and historical harmonic distortion rate sequence. For example, by writing database query statements, such as SQL statements, the following numerical sequences, sorted by time, are extracted from the historical operating status records of the relay protection device under test within a historical time range, such as the past 6 months: historical sampled value valid value (unit: volts), historical load rate (percentage), historical CPU load rate (percentage), historical time deviation (unit: milliseconds), and historical harmonic distortion rate (percentage).

[0038] Furthermore, mean calculations and index volatility analyses are performed on the historical sampled value effective value sequence, historical load rate sequence, historical CPU load rate sequence, historical time deviation sequence, and historical harmonic distortion rate sequence, respectively. The mean of the historical sampled value effective value, the coefficient of variation of the historical sampled value effective value, the mean of the historical load rate, the coefficient of variation of the historical load rate, the mean of the historical CPU load rate, the coefficient of variation of the historical CPU load rate, the mean of the historical time deviation, the coefficient of variation of the historical time deviation, the mean of the historical harmonic distortion rate, and the coefficient of variation of the historical harmonic distortion rate are calculated and used as the operating state parameter set. For example, for each sequence, its arithmetic mean is first calculated. Further, the standard deviation of the sequence is calculated. Finally, the standard deviation is divided by the mean to obtain the coefficient of variation of the sequence. For example, for a historical CPU load rate sequence, its mean is calculated to be 30.5%, and its standard deviation is 4.2%, then its coefficient of variation is 4.2% / 30.5% = 0.138. Using the same method, ten specific parameter values ​​were calculated sequentially and ultimately obtained: the mean of historical sampled values, the coefficient of variation of historical sampled values, the mean of historical load rate, the coefficient of variation of historical load rate, the mean of historical CPU load rate, the coefficient of variation of historical CPU load rate, the mean of historical time synchronization deviation, the coefficient of variation of historical time synchronization deviation, the mean of historical harmonic distortion rate, and the coefficient of variation of historical harmonic distortion rate. These ten parameters constitute the operating status parameter set describing the historical operational stability of the device.

[0039] Furthermore, the operational risk coefficient within a preset time range is determined based on the set of operational status parameters and used as the current operational risk coefficient.

[0040] Specifically, firstly, the risk prediction complexity is determined by weighted evaluation based on the coefficients of variation of historical sampled values, historical load rate, historical CPU load rate, historical time deviation, and historical harmonic distortion rate. These coefficients are positively correlated. For example, a weighted summation method is used, assigning the same weight (e.g., 0.2) to each coefficient of variation, and then calculating the weighted sum of all coefficients: Risk Prediction Complexity = 0.2 × Historical Sampled Value Coefficient of Variation + 0.2 × Historical Load Rate Coefficient of Variation + 0.2 × Historical CPU Load Rate Coefficient of Variation + 0.2 × Historical Time Deviation Coefficient of Variation + 0.2 × Historical Harmonic Distortion Coefficient of Variation. The calculated risk prediction complexity is a numerical value that comprehensively reflects the fluctuation level of the device's historical operating status. The greater the fluctuation, the higher the value, meaning a higher uncertainty in predicting future risks.

[0041] Furthermore, based on the historical operation logs of similar relay protection devices, several sample operation status parameter sets are collected. The proportion of abnormal operation events in different sample operation status parameter sets within a preset historical time range is used as the sample operation risk coefficient, thus obtaining several sample operation risk coefficients. For example, historical operation data of other relay protection devices of the same model or type are collected. For each device, its "operation status parameter set" at a certain historical moment is calculated. Simultaneously, the operation logs of this device within the next six months (such as the historical operation data) within a preset future time range are queried. The number of recorded abnormal events, such as protection malfunctions, alarms, and communication interruptions, is counted, and this number is divided by the total operating time to obtain the sample operation risk coefficient.

[0042] Furthermore, based on several sets of sample running state parameters and several sets of sample running risk coefficients, Q sample training sets are obtained, and each set is trained with a Long Short-Term Memory (LSTM) network until convergence, constructing Q running risk assessment models, where Q is an integer greater than 20. For example, these sample data are randomly shuffled and then divided into Q equal parts, where Q is an integer greater than 20; for example, if Q=25, the sample data is divided into 25 equal parts. Using Python's TensorFlow library, 25 identical LSM network models are constructed. Each model has three LSTM layers with 64, 32, and 16 neurons respectively, and is finally connected to a fully connected output layer. Each of these 25 data parts is used as a training set, with the parameters of the sample running state parameter set as input and the sample running risk coefficient as the target output. Mean squared error is used as the loss function, and the Adam optimizer is used for training until the model's loss on the training set no longer decreases significantly, indicating that the running risk assessment model training is complete, resulting in 25 trained running risk assessment models.

[0043] Further, the ratio of the risk prediction complexity to the maximum historical risk prediction complexity recorded within the historical time range is multiplied by Q and rounded to obtain the number of suitable models selected, K, where K is greater than or equal to 1 and less than or equal to Q. For example, from the long-term operation log of the device under test, all historical risk prediction complexity values ​​calculated in the past year are retrieved, and the maximum value is identified and recorded as the maximum historical risk prediction complexity. Further, the ratio of the risk prediction complexity to the maximum historical risk prediction complexity is calculated. This ratio is multiplied by the total number of models, Q, and the result is rounded to the nearest integer to obtain the number of models to be selected, K. For example, if the current risk prediction complexity is 0.15 and the maximum historical value is 0.3, then the ratio = 0.15 / 0.3 = 0.5. Multiplying this by the total number of models, Q, we get K = 0.5 × 25 = 12.5. After rounding up, K = 13. When the calculated K is greater than or equal to Q, K = Q.

[0044] Furthermore, K operational risk assessment models are randomly selected from Q operational risk assessment models. Each model performs an operational risk assessment based on the set of operational state parameters, and the average of the K assessment results is used as the operational risk coefficient within the predetermined future timeframe. For example, 13 models are randomly selected from 25 trained models. The operational state parameter set of the device under test is input into these 13 selected models, and each model outputs a predicted risk coefficient value. Finally, the arithmetic mean of the outputs of these 13 models is calculated as the operational risk coefficient of the relay protection device under test within the predetermined future timeframe, such as the next 6 months.

[0045] By deeply mining and quantitatively analyzing the historical operating status records of the relay protection device under test, a dynamic and quantitative assessment of the individualized operational risk of the device is achieved. The method provided in this application does not simply list historical data, but rather extracts key parameter sequences such as effective sample values, load rate, CPU load rate, time synchronization deviation, and harmonic distortion rate, and further calculates their mean and coefficient of variation to construct an operating status parameter set that comprehensively reflects the steady-state level and fluctuation characteristics of the device's operation. Based on the current operational risk coefficient obtained from this parameter set, the abstract and multifaceted historical operational performance of the device is condensed into a specific and comparable quantitative risk indicator.

[0046] S20: Based on the preset test channel group, determine the adaptation channel selection scheme as the adaptation performance test scheme according to the current operating risk coefficient analysis, and perform multi-dimensional performance tests on the relay protection device under test according to the adaptation performance test scheme to obtain multi-dimensional performance test data.

[0047] The testing process for existing relay protection devices generally suffers from rigid testing strategies and suboptimal resource allocation. Testing typically employs pre-set, fixed test channels and combinations, failing to dynamically correlate with the actual risk status of the device under test. Regardless of the device's risk level, a nearly identical testing procedure is implemented, resulting in a dual drawback: for devices with stable operation and low risk, fixed and comprehensive testing may include numerous unnecessary test items, prolonging testing time, consuming valuable testing resources, and leading to low operation and maintenance efficiency; while for devices with high potential risks, fixed test combinations may fail to cover specific weaknesses or deep-seated hidden dangers, causing some critical defects to be missed during testing, resulting in insufficient depth and relevance of the testing.

[0048] Step S20 in the method provided in this application embodiment includes:

[0049] Obtain a preset test channel group, wherein the preset test channel group includes a first test channel, a second test channel, a third test channel, a fourth test channel, a fifth test channel, and a sixth test channel;

[0050] If the current operating risk coefficient is less than or equal to the first preset risk threshold, then the first test channel and the second test channel are selected as the adaptation channel selection scheme.

[0051] If the current operating risk coefficient is greater than the first preset risk threshold and less than or equal to the second preset risk threshold, then the first test channel, the second test channel, the third test channel and the sixth test channel are selected as the adaptation channel selection scheme.

[0052] If the current operating risk coefficient is greater than the second preset risk threshold and less than or equal to the third preset risk threshold, then the first test channel, the second test channel, the fourth test channel, the fifth test channel, and the sixth test channel are selected as the adaptation channel selection scheme.

[0053] If the current operating risk coefficient is greater than the third preset risk threshold, then the first test channel, the second test channel, the third test channel, the fourth test channel, the fifth test channel, and the sixth test channel are selected as the adaptation channel selection scheme.

[0054] Wherein, the first preset risk threshold is less than the second preset risk threshold, and the second preset risk threshold is less than the third preset risk threshold;

[0055] The first test channel is an analog output channel, the second test channel is a digital input channel, the third test channel is a digital output channel, the fourth test channel is a digital sampling value channel, the fifth test channel is a time synchronization test channel, and the sixth test channel is a real-time monitoring channel for operating status. The analog output channel includes an AC voltage sub-channel and an AC current sub-channel.

[0056] In this embodiment of the application, based on the preset test channel group, the adaptation channel selection scheme is determined as the adaptation performance test scheme according to the current operation risk coefficient analysis, and multi-dimensional performance tests are carried out on the relay protection device under test in accordance with the adaptation performance test scheme to obtain multi-dimensional performance test data.

[0057] Specifically, first, a preset test channel group is obtained, which includes a first test channel, a second test channel, a third test channel, a fourth test channel, a fifth test channel, and a sixth test channel.

[0058] The test channels are as follows: the first test channel is an analog output channel, the second test channel is a digital input channel, the third test channel is a digital output channel, the fourth test channel is a digital sampling value channel, the fifth test channel is a time synchronization test channel, and the sixth test channel is a real-time monitoring channel for operating status. The analog output channel includes an AC voltage sub-channel and an AC current sub-channel.

[0059] Specifically, the first test channel is an analog output channel, used by the test instrument to generate and output analog electrical signals, including an AC voltage sub-channel and an AC current sub-channel. This channel generates voltage and current waveforms that meet test requirements through a built-in digital-to-analog converter and power amplifier. For example, when testing overcurrent protection, the AC current sub-channel can output a current signal that linearly rises from a normal value (e.g., 1.0 amps) to exceed the protection setting (e.g., 5 amps), used to verify the protection device's operating threshold and delay characteristics. The second test channel is a digital input channel, used by the test instrument to monitor and record the physical switching status of the relay protection device's output. This channel is connected to the trip, close, or signal output relay contacts of the protection device via an optocoupler isolation circuit. When the protection device operates, its contacts close or open, and the digital input channel detects this state change and accurately records the timing of the operation. For example, after applying a fault current, this channel records the closing event of a trip contact and marks its time difference relative to the fault start time, such as 35.2 milliseconds, thereby verifying the protection's operating speed. The third test channel is a digital output channel, used by the test instrument to simulate input digital signals to the relay protection device. This channel can be programmed to control the on / off state of the output node to simulate digital input conditions such as circuit breaker position, remote / local switching, and external protection interlocking. For example, when testing the automatic reclosing function, this channel can simulate a "circuit breaker has opened" signal after the protection trips for the first time, and then simulate a "circuit breaker has closed" signal after a delay, to trigger and verify the device's reclosing logic. The fourth test channel is a digital sampling value channel, used to send digital sampling value messages to the relay protection device via Ethernet according to standard protocols. This channel replaces the traditional analog cable, directly transmitting the instantaneous current and voltage values ​​collected by the merging unit in the form of data packets. For example, this channel can continuously send SV messages representing normal three-phase voltage and current, and modify the sampled value of one phase current at a specific time to simulate the magnitude and phase of a fault. The fifth test channel is a time synchronization test channel, a dedicated channel used to verify the time synchronization performance of the relay protection device. This channel serves two purposes: firstly, it acts as a time source, sending synchronization signals to the device; secondly, it functions as a monitoring device, measuring the deviation between the device's own clock and a standard source. For example, this channel continuously monitors the device's time synchronization deviation, potentially recording fluctuations within ±2 microseconds. This channel can also simulate clock anomalies during testing, such as interrupting the synchronization signal, to observe the device's timekeeping performance and alarm behavior. The sixth test channel is an operational status monitoring channel, used to test instruments that establish a communication connection with the relay protection device via manufacturing message specifications or other plant communication protocols to read its internal status information in real time. This channel continuously queries and records the device's internal logic node status, measured values, setpoint ranges, self-test alarm information, etc.For example, when a fault is injected through an analog or digital channel, the channel can simultaneously record the changes in internal variables such as protection initiation elements, measurement elements, and tripping logic, as well as whether any non-critical self-test alarms are generated.

[0060] Furthermore, a first preset risk threshold, a second preset risk threshold, and a third preset risk threshold are obtained. The first preset risk threshold is less than the second preset risk threshold, and the second preset risk threshold is less than the third preset risk threshold. For example, the first preset risk threshold is set to 0.1, the second preset risk threshold to 0.2, and the third preset risk threshold to 0.4.

[0061] Furthermore, if the current operating risk coefficient is less than or equal to the first preset risk threshold, then the first test channel and the second test channel are selected as the adaptation channel selection scheme. For example, when the current operating risk coefficient of the relay protection device under test is assessed to be less than or equal to the first preset risk threshold, such as setting the first preset risk threshold to 0.1, it is determined that the current risk level is low. In this case, the adaptation channel selection scheme is determined to select only the first test channel and the second test channel. For example, the test instrument outputs a standard analog test signal to the relay protection device under test through its first test channel; for example, the AC current sub-channel outputs a power frequency current signal with an amplitude of 5.0 amperes. Simultaneously, the second test channel of the test instrument is connected to the output relay contact of the relay protection device under test to monitor and record the closing and opening states of the contact in real time. After the test is performed, the acquired multi-dimensional performance test data includes: all applied voltage and current waveform data recorded by the first test channel, and the device action response time sequence data recorded by the second test channel. These test results together constitute the multi-dimensional performance test data under the low-risk level, confirming whether the core functions are normal.

[0062] Furthermore, if the current operational risk coefficient is greater than the first preset risk threshold and less than or equal to the second preset risk threshold, then the first test channel, the second test channel, the third test channel, and the sixth test channel are selected as the adaptation channel selection scheme. For example, if the second preset risk threshold is set to 0.2, and the current operational risk coefficient is greater than the first preset risk threshold and less than or equal to the second preset risk threshold, it indicates that there is a certain risk possibility, classified as a medium risk level. In this case, the first test channel, the second test channel, the third test channel, and the sixth test channel are selected as the adaptation channel selection scheme. For example, a preset switching signal, such as simulating a circuit breaker position signal, is output to the relay protection device under test through the third test channel. Simultaneously, a connection is established with the communication management interface of the relay protection device under test through the sixth test channel to read the internal state variables and self-test information data stream of the device in real time. After the test is executed, the acquired multi-dimensional performance test data, in addition to the data from the first and second channels, includes the excitation signal timing data output by the third test channel, as well as the internal logic node status and self-test message sequence collected by the sixth test channel.

[0063] Furthermore, if the current operational risk coefficient is greater than the second preset risk threshold but less than or equal to the third preset risk threshold, then the first, second, fourth, fifth, and sixth test channels are selected as the adaptation channel selection scheme. For example, if the third preset risk threshold is set to 0.4, and the current operational risk coefficient is greater than the second preset risk threshold but less than or equal to the third preset risk threshold, it indicates that the current operational risk is high, classified as a high-risk level. While performing basic analog excitation and response monitoring through the first and second test channels, the fourth test channel sends digital sampled value messages to the relay protection device under test according to the standard protocol. The fifth test channel injects a standard clock synchronization signal into the device and continuously monitors the actual time deviation value of the device. The sixth test channel synchronously monitors the device's reception status of digital messages and its internal clock synchronization status during the test. After the test, the acquired multi-dimensional performance test data includes the sampled value message sequence sent by the fourth test channel, the clock deviation data recorded by the fifth test channel, and the internal status data of the sixth test channel, forming a test dataset.

[0064] Furthermore, if the current operational risk coefficient is greater than the third preset risk threshold, then the first, second, third, fourth, fifth, and sixth test channels are selected as the adaptation channel selection scheme. When the current operational risk coefficient is greater than the third preset risk threshold, it indicates that the current operational risk is extremely high, classified as an "extremely high risk level." All test channels are activated simultaneously. For example, the first and fourth test channels can provide electrical quantity excitation synchronously or alternately; the third test channel simulates complex switching input sequences; the fifth test channel dynamically changes the clock signal during the test to create synchronization disturbances; and the second and sixth test channels comprehensively record the device's output actions and the continuous changes in all relevant internal logic and state parameters. After the test is executed, the acquired multi-dimensional performance test data is a complete set of data recorded by all six test channels, comprehensively characterizing the current operational status and thoroughly identifying potential faults.

[0065] By introducing a mechanism to determine the appropriate channel selection scheme based on the current operational risk coefficient analysis, a fundamental shift in testing strategy from static preset to dynamic adaptation is achieved. The method provided in this application is based on a preset test channel group containing multiple types of test channels. According to the quantified risk coefficient generated in step S10, it dynamically selects the test channel combination that best matches the risk level through logical judgment. This allows low-risk devices to undergo only core and necessary tests, quickly completing status confirmation; while medium- and high-risk devices can automatically trigger more comprehensive and in-depth multi-dimensional tests, focusing on identifying potential fault points. This on-demand testing approach significantly optimizes the allocation of testing resources, ensuring sufficient testing for high-risk devices while reducing unnecessary waste of testing resources for low-risk devices. This significantly improves the overall efficiency and relevance of testing, achieving a precise match between testing intensity and device risk level.

[0066] S30: Using a pre-built device performance evaluation engine, determine the performance evaluation result of the relay protection device under test based on the analysis of the multi-dimensional performance test data.

[0067] After obtaining multi-dimensional performance test data of relay protection devices, how to conduct comprehensive, accurate, and reliable analysis and evaluation is a technical problem that urgently needs to be solved in the current technology. Traditional evaluation methods often rely on simple threshold comparison, fixed-weight scoring, or a single evaluation model. These methods have poor generalization ability and robustness when faced with heterogeneous and complex test data from different test channels that reflect different aspects of the device's performance.

[0068] Step S30 in the method provided in this application embodiment includes:

[0069] The method for constructing the device performance evaluation engine includes:

[0070] Based on the historical operation logs of similar relay protection devices, a multidimensional performance test dataset of samples was collected, and the performance of different multidimensional performance test data of samples was evaluated through a power grid expert system to obtain a sample performance score set.

[0071] The sample multidimensional performance test dataset and sample performance score set are used as training data. The training data is divided into L parts. The first sample training set is constructed by randomly selecting L parts of the dataset with replacement. The selection is iterated L times to obtain L sample training sets, where L is an integer greater than or equal to 10.

[0072] The deep neural networks are trained to convergence using the L sample training sets to generate L device performance evaluation branches, and the device performance evaluation engine is constructed by integrating them according to the mean fusion strategy.

[0073] The component complexity of the multidimensional performance test data is evaluated to obtain the test data complexity, wherein the test data complexity is the ratio of the number of data types of the multidimensional performance test data to the total number of preset data types;

[0074] The number of adapted branches P is obtained by taking the floor of the product of the test data complexity and L, where P is greater than or equal to 1 and less than or equal to L.

[0075] P device performance evaluation branches are randomly selected from the L device performance evaluation branches of the device performance evaluation engine. The current performance score is determined based on the analysis of the multidimensional performance test data and is used as the performance evaluation result of the relay protection device under test.

[0076] In this embodiment, a pre-built device performance evaluation engine is used to determine the performance evaluation result of the relay protection device under test based on multi-dimensional performance test data analysis.

[0077] Specifically, a device performance evaluation engine is constructed, including:

[0078] First, based on historical operation logs of similar relay protection devices, a sample multidimensional performance test dataset is collected. The performance of different sample multidimensional performance test data is then evaluated using a power grid expert system to obtain a sample performance score set. For example, sample multidimensional performance test datasets of similar relay protection devices recorded in various past tests are collected from a historical operation and maintenance database. Each sample data entry contains test data recorded by multiple test channels. Subsequently, each sample multidimensional performance test data is submitted to the power grid expert system. The power grid expert system is a computer program that can mimic the reasoning and decision-making of human experts in a specific domain. It has a built-in software rule base consisting of industry standards, procedures, and expert experience. It automatically parses the input data and outputs a quantitative performance score according to preset evaluation rules. For example, for a set of test data containing parameters such as action time, measurement accuracy, and communication status, the expert system might output a score such as 92.5. By performing this operation on all sample data, a sample performance score set corresponding one-to-one with the sample multidimensional performance test dataset is obtained.

[0079] Furthermore, using a multidimensional performance test dataset and a performance score set as training data, the training data is divided into L equal parts. L random selections with replacement are made from these L parts to construct the first training set. This process is repeated L times to obtain L training sets, where L is an integer greater than or equal to 10. For example, the original training data is randomly sorted and then divided into L approximately equal training sets, where L is 10, resulting in 10 training sets. A new training set is constructed using random sampling with replacement: one set is randomly selected from these 10 training sets, and the selected set is replaced to ensure it may be selected again. This process is repeated 10 times, and all the selected data from the 10 samplings are merged to form the first training set. This process is repeated 10 times to obtain 10 training sets that are different from each other.

[0080] Furthermore, deep neural networks are trained to convergence using L sample training sets, generating L device performance evaluation branches. These branches are then integrated using a mean fusion strategy to construct a device performance evaluation engine. For example, multiple deep neural network models are constructed as multiple device performance evaluation branches. Each model contains an input layer, two fully connected hidden layers, and an output layer. The number of neurons in the input layer is the same as the feature dimension of the multidimensional performance test data. The first hidden layer has 64 neurons and uses the ReLU activation function; the second hidden layer has 32 neurons and uses the ReLU activation function; the output layer has one neuron, corresponding to the performance score. Using this model structure as a basis, each model is trained independently using L sample training sets. During training, the multidimensional performance test data is used as the input feature, the corresponding sample performance score is used as the target value, mean squared error is used as the loss function, and the Adam optimizer is used for iterative parameter updates until the model's prediction error on the training set stabilizes. Finally, L deep neural network models with identical structures but slightly different internal parameters are trained, serving as L device performance evaluation branches. The L device performance evaluation branches are integrated using a mean fusion strategy to obtain the device performance evaluation engine. When a multidimensional performance test dataset is input, the average of the outputs of the selected device performance evaluation branches is calculated to obtain the device performance evaluation result.

[0081] Furthermore, the component complexity of the multidimensional performance test data is evaluated to obtain the test data complexity, which is the ratio of the number of data types in the multidimensional performance test data to the total number of preset data types. For example, the total number of data types of interest in the evaluation system, i.e., the preset total number of data types, is 50. These types cover all key indicators of different test channels, such as action time, amplitude error, and message latency. The actual number of valid data types contained in the current multidimensional performance test data is counted, for example, 40, which is the number of non-empty and valid indicator types in the data. Then the test data complexity is 40 / 50 = 0.8.

[0082] Furthermore, the product of the test data complexity and L is rounded down to obtain the number of adapted branches P, where P is greater than or equal to 1 and less than or equal to L. For example, if the test data complexity is 0.8 and L is 10, then P = 0.8 × 10 = 8, and the result is rounded up if it is not an integer. Further, 8 branches are randomly selected from the 10 device performance evaluation branches of the device performance evaluation engine.

[0083] Furthermore, P device performance evaluation branches are randomly selected from the L device performance evaluation branches in the device performance evaluation engine. The current performance score is determined based on multi-dimensional performance test data analysis and used as the performance evaluation result of the relay protection device under test. For example, the average of the device performance evaluation results output by the P device performance evaluation branches based on the multi-dimensional performance test data is calculated and used as the performance evaluation result of the relay protection device under test. For instance, if the scores of the 8 branches are [95, 93, 94, 96, 92, 95, 94, 93], then the performance evaluation result = (95 + 93 + 94 + 96 + 92 + 95 + 94 + 93) / 8 = 94.

[0084] By employing a pre-built integrated device performance evaluation engine to process multidimensional performance test data, the accuracy, robustness, and adaptability of the evaluation results are effectively improved. The limitations of a single evaluation model are overcome by utilizing ensemble learning; multiple deep neural network evaluation branches are constructed and integrated through bootstrapping sampling technology, forming a powerful evaluation system. During the evaluation process, the component complexity of the input multidimensional test data itself is first evaluated, and the number of model branches participating in the evaluation is dynamically determined based on the resulting complexity. This design allows the granularity of the evaluation process to automatically adapt to the richness of the test data: for simple test data, fewer branches are used for efficient evaluation; for complex, comprehensive test data, more branches are mobilized for in-depth analysis. This dynamic fusion mechanism significantly enhances the evaluation system's adaptability to diverse test scenarios, reduces evaluation errors, and makes the final performance evaluation results more objective, stable, and reliable.

[0085] Example 2, as Figure 2 As shown, based on the same inventive concept as the relay protection device performance evaluation method based on multi-channel closed-loop testing provided in Embodiment 1, this embodiment of the invention also provides a relay protection device performance evaluation system based on multi-channel closed-loop testing, including:

[0086] The operational risk acquisition module 100 is used to assess and determine the current operational risk coefficient based on the historical operational status records of the relay protection device under test within a historical time range.

[0087] The multidimensional test data acquisition module 200 is used to determine the adaptation channel selection scheme as the adaptation performance test scheme based on the current operating risk coefficient analysis, based on the preset test channel group, and to perform multidimensional performance tests on the relay protection device under test according to the adaptation performance test scheme to acquire multidimensional performance test data.

[0088] The performance evaluation result acquisition module 300 is used to determine the performance evaluation result of the relay protection device under test based on the analysis of the multi-dimensional performance test data using a pre-built device performance evaluation engine.

[0089] In one embodiment, the risk acquisition module 100 is further configured to:

[0090] The historical operating status records of the relay protection device under test within the historical time range are screened to obtain the historical sampled value effective value sequence, historical load rate sequence, historical CPU load rate sequence, historical time deviation sequence, and historical harmonic distortion rate sequence.

[0091] The mean values ​​and index volatility analyses of the historical sampled value effective value sequence, historical load rate sequence, historical CPU load rate sequence, historical time deviation sequence, and historical harmonic distortion rate sequence are calculated respectively. The mean value of historical sampled value effective value, the coefficient of variation of historical sampled value effective value, the mean value of historical load rate, the coefficient of variation of historical load rate, the mean value of historical CPU load rate, the coefficient of variation of historical CPU load rate, the mean value of historical time deviation, the coefficient of variation of historical time deviation, the mean value of historical harmonic distortion rate, and the coefficient of variation of historical harmonic distortion rate are calculated and used as the set of operating status parameters.

[0092] The operational risk coefficient within a future preset time range is determined based on the set of operational status parameters and used as the current operational risk coefficient.

[0093] The determination of operational risk coefficients within a preset future timeframe based on the set of operational status parameters includes:

[0094] The risk prediction complexity is determined by weighted evaluation of the coefficient of variation of the effective value of the historical sampled values, the coefficient of variation of the historical load rate, the coefficient of variation of the historical CPU load rate, the coefficient of variation of the historical time deviation, and the coefficient of variation of the historical harmonic distortion rate. The risk prediction complexity is positively correlated with the coefficient of variation of the effective value of the historical sampled values, the coefficient of variation of the historical load rate, the coefficient of variation of the historical CPU load rate, the coefficient of variation of the historical time deviation, and the coefficient of variation of the historical harmonic distortion rate.

[0095] Based on the historical operation logs of similar relay protection devices, several sample operation status parameter sets were collected, and the proportion of abnormal operation events in different sample operation status parameter sets within the historical preset time range was used as the sample operation risk coefficient to obtain several sample operation risk coefficients.

[0096] Based on the aforementioned set of sample operating status parameters and the set of sample operating risk coefficients, Q sample training sets are obtained, and long short-term memory networks are trained to convergence respectively to construct Q operating risk assessment models, where Q is an integer greater than 20;

[0097] The ratio of the risk prediction complexity to the maximum historical risk prediction complexity recorded within the historical time range is multiplied by Q and rounded to obtain the number of suitable models selected, K, where K is greater than or equal to 1 and less than or equal to Q.

[0098] K operational risk assessment models are randomly selected from the Q operational risk assessment models, and operational risk assessments are performed on each model based on the operational status parameter set. The average value of the K assessment results is then used as the operational risk coefficient within the preset future time range.

[0099] In one embodiment, the multidimensional test data acquisition module 200 is further configured to:

[0100] Obtain a preset test channel group, wherein the preset test channel group includes a first test channel, a second test channel, a third test channel, a fourth test channel, a fifth test channel, and a sixth test channel;

[0101] If the current operating risk coefficient is less than or equal to the first preset risk threshold, then the first test channel and the second test channel are selected as the adaptation channel selection scheme.

[0102] If the current operating risk coefficient is greater than the first preset risk threshold and less than or equal to the second preset risk threshold, then the first test channel, the second test channel, the third test channel and the sixth test channel are selected as the adaptation channel selection scheme.

[0103] If the current operating risk coefficient is greater than the second preset risk threshold and less than or equal to the third preset risk threshold, then the first test channel, the second test channel, the fourth test channel, the fifth test channel, and the sixth test channel are selected as the adaptation channel selection scheme.

[0104] If the current operating risk coefficient is greater than the third preset risk threshold, then the first test channel, the second test channel, the third test channel, the fourth test channel, the fifth test channel, and the sixth test channel are selected as the adaptation channel selection scheme.

[0105] Wherein, the first preset risk threshold is less than the second preset risk threshold, and the second preset risk threshold is less than the third preset risk threshold;

[0106] The first test channel is an analog output channel, the second test channel is a digital input channel, the third test channel is a digital output channel, the fourth test channel is a digital sampling value channel, the fifth test channel is a time synchronization test channel, and the sixth test channel is a real-time monitoring channel for operating status. The analog output channel includes an AC voltage sub-channel and an AC current sub-channel.

[0107] In one embodiment, the performance evaluation result acquisition module 300 is further configured to:

[0108] Based on the historical operation logs of similar relay protection devices, a multidimensional performance test dataset of samples was collected, and the performance of different multidimensional performance test data of samples was evaluated through a power grid expert system to obtain a sample performance score set.

[0109] The sample multidimensional performance test dataset and sample performance score set are used as training data. The training data is divided into L parts. The first sample training set is constructed by randomly selecting L parts of the dataset with replacement. The selection is iterated L times to obtain L sample training sets, where L is an integer greater than or equal to 10.

[0110] The deep neural networks are trained to convergence using the L sample training sets to generate L device performance evaluation branches, and the device performance evaluation engine is constructed by integrating them according to the mean fusion strategy.

[0111] The component complexity of the multidimensional performance test data is evaluated to obtain the test data complexity, wherein the test data complexity is the ratio of the number of data types of the multidimensional performance test data to the total number of preset data types;

[0112] The number of adapted branches P is obtained by taking the floor of the product of the test data complexity and L, where P is greater than or equal to 1 and less than or equal to L.

[0113] P device performance evaluation branches are randomly selected from the L device performance evaluation branches of the device performance evaluation engine. The current performance score is determined based on the analysis of the multidimensional performance test data and is used as the performance evaluation result of the relay protection device under test.

[0114] In summary, the embodiments of this application have at least the following technical effects:

[0115] This application proposes a method and system for evaluating the performance of relay protection devices based on multi-channel closed-loop testing. First, it intelligently assesses the current operational risk coefficient of the device under test based on its historical operating status records. Then, based on this risk coefficient, it dynamically analyzes and determines the appropriate channel selection scheme matching the risk level from a pre-set group of multiple test channels, thus constructing a customized multi-dimensional performance test scheme and executing the test to acquire data. Finally, it utilizes a device performance evaluation engine built based on ensemble learning to comprehensively analyze the multi-dimensional performance test data to obtain the final performance evaluation result. This significantly improves the accuracy, efficiency, and intelligence level of relay protection device performance evaluation. Specifically, in-depth analysis based on historical operational data enables quantitative perception of individualized operational risks of devices. This allows performance testing to move beyond a one-size-fits-all approach and instead intelligently adjust based on the actual health status of the device. For low-risk devices, test items can be streamlined, focusing on core functional verification, effectively saving testing time and resources while ensuring assessment reliability and improving overall operation and maintenance efficiency. For medium- and high-risk devices, a more comprehensive and in-depth multi-channel test combination can be automatically adapted, ensuring that potential defects and performance degradation tendencies are more fully exposed and detected, thereby improving test coverage and fault detection capabilities, and enhancing the power grid's safety and proactive control capabilities. Simultaneously, by introducing an integrated performance evaluation engine and dynamically adjusting the number of model branches involved in the evaluation based on the complexity of the test data itself, the evaluation model's ability to process multi-source heterogeneous test data and its generalization performance are enhanced. Compared with traditional methods, the technical solution provided in this application significantly overcomes the rigidity of fixed test modes and the singleness of evaluation models, realizes the dynamic optimization and configuration of test resources and strategies, and achieves the technical effect of improving the credibility of the final performance evaluation results, providing more scientific and accurate data support for condition-based maintenance, life prediction and update decisions of relay protection devices.

[0116] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0117] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0118] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for performance evaluation of a protective relay based on multi-channel closed-loop testing, characterized in that, The method comprises: According to the historical operation state record of the relay protection device to be tested in the historical time range, the current operation risk coefficient is determined, comprising: Screening the historical operation state record of the relay protection device to be tested in the historical time range, obtaining the historical sampling value effective value sequence, the historical load rate sequence, the historical CPU load rate sequence, the historical time deviation sequence and the historical harmonic distortion rate sequence; Respectively, the historical sampling value effective value sequence, the historical load rate sequence, the historical CPU load rate sequence, the historical time deviation sequence, the historical harmonic distortion rate sequence are calculated and analyzed, and the historical sampling value effective value mean, the historical sampling value effective value variation coefficient, the historical load rate mean, the historical load rate variation coefficient, the historical CPU load rate mean, the historical CPU load rate variation coefficient, the historical time deviation mean, the historical time deviation variation coefficient, the historical harmonic distortion rate mean, the historical harmonic distortion rate variation coefficient are calculated as the operation state parameter set; According to the operation state parameter set, the operation risk coefficient in the future preset time range is evaluated and determined as the current operation risk coefficient, comprising: According to the historical sampling value effective value variation coefficient, the historical load rate variation coefficient, the historical CPU load rate variation coefficient, the historical time deviation variation coefficient and the historical harmonic distortion rate variation coefficient, the risk prediction complexity is determined by weighted evaluation, wherein the risk prediction complexity is positively correlated with the historical sampling value effective value variation coefficient, the historical load rate variation coefficient, the historical CPU load rate variation coefficient, the historical time deviation variation coefficient and the historical harmonic distortion rate variation coefficient. Based on the historical operation log of the same type of relay protection device, a plurality of sample operation state parameter sets are collected, and the proportion of abnormal operation events of different sample operation state parameter sets in the historical preset time range is taken as the sample operation risk coefficient, and a plurality of sample operation risk coefficients are obtained. Based on the plurality of sample operation state parameter sets and the plurality of sample operation risk coefficients, Q sample training sets are divided and obtained, and a long short-term memory network is trained to convergence, and Q operation risk evaluation models are constructed, wherein Q is an integer greater than 20. The ratio of the risk prediction complexity to the maximum historical risk prediction complexity recorded in the historical time range is multiplied by Q to obtain the adaptive model selection number K, wherein K is greater than or equal to 1 and less than or equal to Q. Randomly select K operation risk evaluation models from the Q operation risk evaluation models, respectively evaluate the operation risk according to the operation state parameter set, and take the average value of the K evaluation results as the operation risk coefficient in the future preset time range. Based on the preset test channel group, according to the current operation risk coefficient, an adaptive channel selection scheme is determined as an adaptive performance test scheme, and a multi-dimensional performance test is performed on the relay protection device to be tested according to the adaptive performance test scheme, and multi-dimensional performance test data is obtained. Using the pre-constructed device performance evaluation engine, the performance evaluation result of the relay protection device to be tested is determined according to the multi-dimensional performance test data.

2. The method for performance evaluation of a protective relay based on multi-channel closed-loop testing according to claim 1, characterized in that, Based on the preset test channel group, a suitable channel selection scheme is determined according to the current operation risk coefficient analysis, comprising: Obtaining a preset test channel group, wherein the preset test channel group comprises a first test channel, a second test channel, a third test channel, a fourth test channel, a fifth test channel and a sixth test channel; If the current operation risk coefficient is less than or equal to a first preset risk threshold, the first test channel and the second test channel are selected as the suitable channel selection scheme; If the current operation risk coefficient is greater than the first preset risk threshold and less than or equal to a second preset risk threshold, the first test channel, the second test channel, the third test channel and the sixth test channel are selected as the suitable channel selection scheme.

3. The method for performance evaluation of a protective relay based on multi-channel closed-loop testing according to claim 2, characterized in that, The first test channel is an analog output channel, the second test channel is a switching quantity input channel, the third test channel is a switching quantity output channel, the fourth test channel is a digital sampling value channel, the fifth test channel is a time synchronization test channel, and the sixth test channel is a real-time monitoring channel of the running state.

4. The method for performance evaluation of a protective relay based on multi-channel closed-loop testing according to claim 2, wherein, If the current operation risk coefficient is greater than the second preset risk threshold and less than or equal to a third preset risk threshold, the first test channel, the second test channel, the fourth test channel, the fifth test channel and the sixth test channel are selected as the suitable channel selection scheme; If the current operation risk coefficient is greater than the third preset risk threshold, the first test channel, the second test channel, the third test channel, the fourth test channel, the fifth test channel and the sixth test channel are selected as the suitable channel selection scheme.

5. The method for performance evaluation of a protective relay based on multi-channel closed-loop testing according to claim 4, wherein, The first preset risk threshold is less than the second preset risk threshold, and the second preset risk threshold is less than the third preset risk threshold.

6. The method for performance evaluation of a protective relay based on multi-channel closed-loop testing according to claim 1, wherein, The construction method of the device performance evaluation engine comprises: Based on the historical operation log of the same type of relay protection device, a sample multi-dimensional performance test data set is collected, and different sample multi-dimensional performance test data is evaluated for performance by an expert system in the power grid field to obtain a sample performance score set; The sample multi-dimensional performance test data set and the sample performance score set are used as training data, the training data is equally divided into L parts, L times are randomly selected with replacement in the L data sets, a first sample training set is constructed, and L times are iteratively selected to obtain L sample training sets, wherein L is an integer greater than or equal to 10; The L sample training sets are used to train a deep neural network to convergence, respectively, to generate L device performance evaluation branches, which are integrated according to a mean fusion strategy to construct a device performance evaluation engine.

7. The method for performance evaluation of a protective relay based on multi-channel closed-loop testing according to claim 6, characterized in that, The performance evaluation result of the relay protection device to be tested is determined according to the multi-dimensional performance test data analysis, comprising: The composition complexity of the multi-dimensional performance test data is evaluated to obtain a test data complexity, wherein the test data complexity is the ratio of the number of data types of the multi-dimensional performance test data to the total number of preset data types; The product of the test data complexity and L is rounded to obtain the number of adaptive branches P, wherein P is greater than or equal to 1 and less than or equal to L; Randomly select P device performance evaluation branches in L device performance evaluation branches of the device performance evaluation engine, and determine a current performance score according to the multi-dimensional performance test data analysis as a performance evaluation result of the relay protection device to be tested.

8. A system for performance evaluation of a protective relay based on multi-channel closed-loop testing, characterized in that, The system comprises: The operation risk acquisition module is configured to determine a current operation risk coefficient according to historical operation state records of the relay protection device to be tested within a historical time range, including: The historical operation state records of the relay protection device to be tested within the historical time range are filtered to obtain a historical sampling value effective value sequence, a historical load rate sequence, a historical CPU load rate sequence, a historical time deviation sequence, and a historical harmonic distortion rate sequence; The historical sampling value effective value sequence, the historical load rate sequence, the historical CPU load rate sequence, the historical time deviation sequence, and the historical harmonic distortion rate sequence are subjected to mean value calculation and index fluctuation analysis, respectively, to obtain a historical sampling value effective value mean, a historical sampling value effective value variation coefficient, a historical load rate mean, a historical load rate variation coefficient, a historical CPU load rate mean, a historical CPU load rate variation coefficient, a historical time deviation mean, a historical time deviation variation coefficient, a historical harmonic distortion rate mean, and a historical harmonic distortion rate variation coefficient as an operation state parameter set; The operation risk acquisition module is configured to determine a current operation risk coefficient according to historical operation state records of the relay protection device to be tested within a historical time range, including: The historical sampling value effective value variation coefficient, the historical load rate variation coefficient, the historical CPU load rate variation coefficient, the historical time deviation variation coefficient, and the historical harmonic distortion rate variation coefficient are weighted and evaluated to determine a risk prediction complexity, wherein the risk prediction complexity is positively correlated with the historical sampling value effective value variation coefficient, the historical load rate variation coefficient, the historical CPU load rate variation coefficient, the historical time deviation variation coefficient, and the historical harmonic distortion rate variation coefficient; Based on the historical operation logs of the same type of relay protection devices, a plurality of sample operation state parameter sets are collected, and the proportion of abnormal operation events of different sample operation state parameter sets within a historical preset time range is taken as a sample operation risk coefficient, and a plurality of sample operation risk coefficients are obtained. Based on the plurality of sample operation state parameter sets and the plurality of sample operation risk coefficients, Q sample training sets are divided and obtained, and a long short-term memory network is trained to convergence, and Q operation risk evaluation models are constructed, wherein Q is an integer greater than 20. The ratio of the risk prediction complexity to the maximum historical risk prediction complexity recorded within the historical time range is multiplied by Q to obtain an adaptive model selection number K, wherein K is greater than or equal to 1 and less than or equal to Q; Randomly select K operation risk evaluation models from the Q operation risk evaluation models, respectively evaluate the operation risk according to the operation state parameter set, and take the average value of the K evaluation results as the operation risk coefficient within the future preset time range. The multi-dimensional test data acquisition module is configured to determine an adaptive channel selection scheme as an adaptive performance test scheme based on a preset test channel group according to the current running risk coefficient analysis, and perform multi-dimensional performance test on the relay protection device to be tested according to the adaptive performance test scheme, and acquire multi-dimensional performance test data. The performance evaluation result acquisition module is configured to determine a performance evaluation result of the relay protection device to be tested according to the multi-dimensional performance test data analysis by using a pre-constructed device performance evaluation engine.

Images