Real-time analysis and verification method and system for automatic copying test result

Abstract

The invention discloses a real-time analysis and verification method and system for an automatic copying test result, and the method comprises the steps: carrying out the real-time analysis based on a communication protocol of a scheduling master station, and obtaining a device action message of an automatic copying test; by taking a test scene as a unit, defining device action messages with correct known test results as template action messages, and combining the template action messages into template action events; respectively extracting each set of template action event to establish a corresponding key information set; starting copying, regularly acquiring device action messages of the automatic copying test, and combining the device action messages into a test action event by taking the fault judgment time of the control substation device as a unit; carrying out key information matching verification on each test action event and the template action event in sequence, and outputting a device test result and the accuracy of an automatic copying test result of the safety and stability control system; according to the method and the device, the action message analysis and verification capability of the automatic copying test in the test result analysis stage can be effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of power system safety and stability control technology, and in particular relates to a method and system for real-time analysis and verification of automatic burn-in test results. Background Technology

[0002] Safety and stability control devices are control equipment installed in power plants or substations to ensure the stability of the power system when encountering large disturbances. They enable functions such as generator tripping, load shedding, rapid output reduction, and emergency DC power boosting or reduction. A safety and stability control system is a system composed of safety and stability control devices from two or more power plants connected by communication equipment. With the construction and development of new power systems, the risks faced by safety and stability control systems are gradually increasing. Burn-in testing is an important means of verifying the stability of software operation and functional reliability under extreme conditions. It mainly involves continuously running the system under a designed test environment for a period of time and observing its status. The automatic burn-in testing method for safety and stability control systems includes four aspects: 1. Customization of burn-in test tasks; 2. Reservation of burn-in test resources; 3. Scheduling of burn-in test execution; 4. Analysis of burn-in test results.

[0003] The aforementioned characteristics of automated stress testing—including multi-type data interaction, remote automatic control of device hardware and software pressure plate activation and deactivation, accelerated testing with full-scenario and full-logic test cases, and automatic monitoring of device data—make the analysis of stress test results particularly challenging due to the continuous, long-term automated testing, complex device message content, and a high proportion of correct actions but few errors. This necessitates manual verification of the correctness of the safety and stability control system's actions during 24-hour automated stress testing, leading to low analysis efficiency and accuracy. Therefore, a method and system for real-time analysis and verification of automated stress test results is urgently needed to address these issues. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a method and system for real-time analysis and verification of automatic burn-in test results that can improve the ability of automatic burn-in test to analyze and verify action messages during the test result management stage.

[0005] Technical solution: The present invention provides a method for real-time analysis and verification of automatic stress test results, comprising:

[0006] Based on real-time parsing of the scheduling master station communication protocol, the device action messages of the automatic burn-in test are obtained; the device action messages include: action information group and action report file;

[0007] Based on the test scenario, device action messages with known correct test results are defined as template action messages and combined into template action events;

[0008] Extract the action events of each template and establish a corresponding set of key information.

[0009] Start the burn test and periodically obtain the device action messages of the automatic burn test. Combine the device action messages into test action events based on the fault judgment time of the control substation device.

[0010] Each test action event is matched and verified with the template action event in sequence, and the results of the device test and the accuracy of the automatic burn-in test results of the safety and stability control system are output.

[0011] Furthermore, the real-time parsing based on the dispatch master station communication protocol specifically includes using the communication protocol of the access dispatch terminal based on the security and stability control system or the IEC 61850 protocol; the dispatch terminal communication protocol includes the IEC 60870-5-103 protocol and the IEC 60870-5-104 protocol;

[0012] The action information group is a combination of action information of a single action event, with the device as the unit;

[0013] The action information refers to the displacement message and action quantity information generated when the stability control device takes action.

[0014] The change message includes at least one of the following: stability control start-up, stability control exit, or fault exit;

[0015] The action quantity information includes at least one of the following: line disconnection, generator disconnection, DC boost or pullback quantity, required generator disconnection quantity, required load disconnection quantity, operating mode, stable control execution generator disconnection quantity, stable control execution load disconnection quantity, or stable control modulation DC quantity;

[0016] The action report file is a file generated by the device after it determines that a fault has occurred or the device has started, containing fault information, action sequence, and action behavior during the device's operation.

[0017] Furthermore, the test scenarios include at least one of the following: multiple fault testing, redundant system testing, control strategy testing, DC actuator testing, action logic testing, or main transformer overload testing;

[0018] The device action message with a correct test result is a device action message that conforms to at least one of the following in a long-term and full-logic test environment: correct start-up discrimination logic, correct component start-up / stop discrimination logic, correct component tripping discrimination logic, correct control strategy function, or correct control strategy function in a multi-fault test environment.

[0019] Furthermore, in the step of extracting the action events of each template to establish a corresponding key information set, let the key information set... ,in This refers to the operational status of the device. The absolute time of device startup. This is an array of relative action times. Identify the ID array for each action entry. An array describing the action entries. Attach an array of values ​​to the action. An array of hash values ​​for the action message sequence;

[0020] The device operation status refers to whether each device of the safety and stability control system operates during the test.

[0021] The hash value of the action message sequence is the hash value of the action information sorted according to the action entry identifier ID and the carry action attached value sequence concatenated together.

[0022] The carry action is accompanied by a value that is attached to the action. The value after base 50 ,Right now ,in .

[0023] Furthermore, the key information matching and verification of each test action event and the template action event includes:

[0024] S51: Define the set of key information for the test action events. ,in This refers to the operational status of the device. The absolute time of device startup. This is an array of relative action times. Identify the ID array for each action entry. An array describing the action entries. Attach an array of values ​​to the action. An array of hash values ​​for the action message sequence;

[0025] S52: Sequentially hash the action message sequence values Match all template action messages to obtain a set of key information. set , ,in for The Middle One element, A collection of key information for template action messages The first in indivual element;

[0026] S53: Will All Search results The intersection operation yields a set. ,like If the set is not empty and its size is 1, then proceed to step S54; if If not unique, the output result is not a match; if If the set is empty, the output will be an error.

[0027] S54: Judgment In Whether or not In If they are the same, proceed to S55; otherwise, output an error.

[0028] S55: Judgment Size of the action item set in Whether or not Size of the action item set in If they are the same, proceed to S56; otherwise, output an error.

[0029] S56: Judgment The values ​​attached to each action in the middle and Action Added Value Do they satisfy sequentially? If the condition is met, the output result is correct; otherwise, the output result is incorrect. A collection of key information about action events of array number One element, for The action append value array in the middle Each element.

[0030] Furthermore, the accuracy rate of the automatic stress test results is... This represents the percentage of correct test results obtained by the secure and stable control system during test case execution since the stress test was initiated, out of the total number of executions. .

[0031] Based on the same inventive concept, the present invention also provides an automatic stress test result real-time analysis and verification system, comprising:

[0032] The acquisition module is used to parse and obtain the device action messages of the automatic burn-in test in real time based on the communication protocol of the scheduling master station; the device action messages include: action information group and action report file;

[0033] The template combination module is used to define device action messages with known correct test results as template action messages and combine them into template action events, based on test scenarios.

[0034] The extraction module is used to extract the action events of each template and establish a corresponding set of key information.

[0035] The test combination module is used to start the burn-in test, periodically acquire device action messages from the automatic burn-in test, and combine the device action messages into test action events based on the fault judgment time of the control substation device.

[0036] The analysis and verification module is used to sequentially match and verify the key information of each test action event with the template action event, and output the test results of the device and the accuracy of the automatic burn-in test results of the safety and stability control system.

[0037] Furthermore, the real-time parsing based on the dispatch master station communication protocol in the acquisition module specifically includes, based on the communication protocol of the access dispatch terminal of the security and stability control system or the IEC 61850 protocol; the communication protocol of the dispatch terminal includes the IEC 60870-5-103 protocol and the IEC 60870-5-104 protocol;

[0038] The action information group is a combination of action information of a single action event, with the device as the unit;

[0039] The action information refers to the displacement message and action quantity information generated when the stability control device takes action.

[0040] The change message includes at least one of the following: stability control start-up, stability control exit, or fault exit;

[0041] The action quantity information includes at least one of the following: line disconnection, generator disconnection, DC boost or pullback quantity, required generator disconnection quantity, required load disconnection quantity, operating mode, stable control execution generator disconnection quantity, stable control execution load disconnection quantity, or stable control modulation DC quantity;

[0042] The action report file is a file generated by the device after it determines that a fault has occurred or the device has started, containing fault information, action sequence, and action behavior during the device's operation.

[0043] Furthermore, the test scenarios described in the template combination module include at least one of the following: multiple fault testing, redundant system testing, control strategy testing, DC actuator testing, action logic testing, or main transformer overload testing;

[0044] The device action message with a correct test result is a device action message that conforms to at least one of the following in a long-term and full-logic test environment: correct start-up discrimination logic, correct component start-up / stop discrimination logic, correct component tripping discrimination logic, correct control strategy function, or correct control strategy function in a multi-fault test environment.

[0045] Furthermore, in the extraction module, the key information set... ,in This refers to the operational status of the device. The absolute time of device startup. This is an array of relative action times. Identify the ID array for each action entry. An array describing the action entries. Attach an array of values ​​to the action. An array of hash values ​​for the action message sequence;

[0046] The device operation status refers to whether each device of the safety and stability control system operates during the test.

[0047] The hash value of the action message sequence is the hash value of the action information sorted according to the action entry identifier ID and the carry action attached value sequence concatenated together.

[0048] The carry action is accompanied by a value that is attached to the action. The value after base 50 ,Right now ,in .

[0049] Furthermore, the analysis and verification module performs key information matching and verification between each test action event and the template action event, including:

[0050] S51: Define the set of key information for the test action events. ,in This refers to the operational status of the device. The absolute time of device startup. This is an array of relative action times. Identify the ID array for each action entry. An array describing the action entries. Attach an array of values ​​to the action. An array of hash values ​​for the action message sequence;

[0051] S52: Sequentially hash the action message sequence values Match all template action messages to obtain a set of key information. set , ,in for The Middle One element, A collection of key information for template action messages The first in indivual element;

[0052] S53: Will All Search results The intersection operation yields a set. ,like If the set is not empty and its size is 1, then proceed to step S54; if If not unique, the output result is not a match; if If the set is empty, the output will be an error.

[0053] S54: Judgment In Whether or not In If they are the same, proceed to S55; otherwise, output an error.

[0054] S55: Judgment Size of the action item set in Whether or not Size of the action item set in If they are the same, proceed to S56; otherwise, output an error.

[0055] S56: Judgment The values ​​attached to each action in the middle and Action Added Value Do they satisfy sequentially? If the condition is met, the output result is correct; otherwise, the output result is incorrect. A collection of key information about action events of array number One element, for The action append value array in the middle Each element.

[0056] Furthermore, the accuracy rate of the automatic burn-in test results in the analysis and verification module is... This represents the percentage of correct test results obtained by the secure and stable control system during test case execution since the stress test was initiated, out of the total number of executions. .

[0057] Based on the same inventive concept, the present invention also provides a computing device, comprising: one or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memory and configured to be executed by the processor, and when the programs are loaded onto the processor, they implement the steps of the real-time analysis and verification method based on the automatic burn-in test results described in any of the preceding claims.

[0058] Based on the same inventive concept, the present invention also provides a storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the steps of the automatic burn-in test result real-time analysis and verification method according to any one of the above claims.

[0059] Beneficial effects: Compared with the prior art, the present invention effectively improves the ability of automatic burn-in testing to analyze and verify action messages in the test result management stage, realizes automatic replacement of manual work, and also solves the problems of difficulty in manual analysis and verification and missed judgment of erroneous execution results caused by long-term, full-logic testing. Attached Figure Description

[0060] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0062] Example 1, such as Figure 1 As shown, the real-time analysis and verification method for automatic stress testing results in this embodiment includes:

[0063] S1: Based on the real-time parsing of the scheduling master station communication protocol, obtain the device action message of the automatic burn-in test; the device action message includes: action information group and action report file;

[0064] S2: Based on the test scenario, define the device action message with known correct test results as a template action message and combine them into a template action event;

[0065] S3: Extract the action events of each template and establish a corresponding set of key information;

[0066] S4: Start the burn test, periodically obtain the device action messages of the automatic burn test, and combine the device action messages into test action events based on the fault judgment time of the control substation device.

[0067] S5: Perform key information matching and verification on each test action event and template action event in sequence, and output the test results of the device and the accuracy of the automatic burn-in test results of the safety and stability control system.

[0068] Specifically, in step S1, based on the IEC 61850 protocol, the device action report file is summoned in real time through the communication gateway and parsed into a device action message;

[0069] Step S2: Taking the N-2 fault DC-DC strategy function test as a unit, define the device action message with known correct test results as a template action message and combine them into a template action event;

[0070] Step S3: Extract key information from each template action event. , The structure is shown in Table 1.

[0071] Table 1 Set Structure Table

[0072]

[0073] Step S4: Start the burn test, periodically obtain the device action messages of the automatic burn test, and combine the device action messages into test action events based on the fault judgment time of the control substation device;

[0074] Step S5: Perform key information matching and verification on each test action event and template action event in sequence, and output the test results of the device and the accuracy rate of the automatic burn-in test results of the safety and stability control system;

[0075] Step S51: Calculate the key information set of the test action events. ,Right now , The structure and contents are shown in Table 2.

[0076] Table 2 Set Structure Table

[0077]

[0078] Step S52: Sequentially hash the action message sequence values Match all template action messages to obtain a set of key information. set , ,in for The Middle One element, A collection of key information for template action messages The first in indivual element;.

[0079] Step S53: ... All Search results The intersection operation yields a set. , If the set is not empty and its size is 1, proceed to step S54.

[0080] Step S54: Determine In Whether or not In If they are the same, then proceed to S55.

[0081] Step S55: Determine Size of the action item set in Whether or not Size of the action item set in If they are the same, proceed to S56.

[0082] Step S56: Determine The values ​​attached to each action in the middle and Action Added Value Do they satisfy sequentially? If the condition is met, the output result is correct; otherwise, the output result is incorrect. A collection of key information about action events of array number One element, for The action append value array in the middle 1 element; output the accuracy rate of the automatic burn-in test results. This refers to the percentage of correct results obtained by the security and stability control system when executing test cases since the stress test was started, out of the total number of executions. .

[0083] Example 2, based on the same inventive concept, also provides an automatic stress test result real-time analysis and verification system, including:

[0084] The acquisition module is used to parse and obtain the device action messages of the automatic burn-in test in real time based on the communication protocol of the scheduling master station; the device action messages include: action information group and action report file;

[0085] The template combination module is used to define device action messages with known correct test results as template action messages and combine them into template action events, based on test scenarios.

[0086] The extraction module is used to extract the action events of each template and establish a corresponding set of key information.

[0087] The test combination module is used to start the burn-in test, periodically acquire device action messages from the automatic burn-in test, and combine the device action messages into test action events based on the fault judgment time of the control substation device.

[0088] The analysis and verification module is used to sequentially match and verify the key information of each test action event with the template action event, and output the test results of the device and the accuracy of the automatic burn-in test results of the safety and stability control system.

[0089] Furthermore, the real-time parsing based on the dispatch master station communication protocol in the acquisition module specifically includes, based on the communication protocol of the access dispatch terminal of the security and stability control system or the IEC 61850 protocol; the communication protocol of the dispatch terminal includes the IEC 60870-5-103 protocol and the IEC 60870-5-104 protocol;

[0090] The action information group is a combination of action information of a single action event, with the device as the unit;

[0091] The action information refers to the displacement message and action quantity information generated when the stability control device takes action.

[0092] The change message includes at least one of the following: stability control start-up, stability control exit, or fault exit;

[0093] The action quantity information includes at least one of the following: line disconnection, generator disconnection, DC boost or pullback quantity, required generator disconnection quantity, required load disconnection quantity, operating mode, stable control execution generator disconnection quantity, stable control execution load disconnection quantity, or stable control modulation DC quantity;

[0094] The action report file is a file generated by the device after it determines that a fault has occurred or the device has started, containing fault information, action sequence, and action behavior during the device's operation.

[0095] Furthermore, the test scenarios described in the template combination module include at least one of the following: multiple fault testing, redundant system testing, control strategy testing, DC actuator testing, action logic testing, or main transformer overload testing;

[0096] The device action message with the correct test result is a device action message that conforms to at least one of the following in a long-term and full-logic test environment: correct start-up discrimination logic, correct component start-up / stop discrimination logic, correct component tripping discrimination logic, correct control strategy function, or correct control strategy function in a multi-fault test environment under a safe and stable control system.

[0097] Furthermore, in the extraction module, the key information set... ,in This refers to the operational status of the device. The absolute time of device startup. This is an array of relative action times. Identify the ID array for each action entry. An array describing the action entries. Attach an array of values ​​to the action. An array of hash values ​​for the action message sequence;

[0098] The device operation status refers to whether each device of the safety and stability control system operates during the test.

[0099] The hash value of the action message sequence is the hash value of the action information sorted according to the action entry identifier ID and the carry action attached value sequence concatenated together.

[0100] The carry action is accompanied by a value that is attached to the action. The value after base 50 ,Right now ,in .

[0101] Furthermore, the analysis and verification module performs key information matching and verification between each test action event and the template action event, including:

[0102] S51: Define the set of key information for the test action events. ,in This refers to the operational status of the device. The absolute time of device startup. This is an array of relative action times. Identify the ID array for each action entry. An array describing the action entries. Attach an array of values ​​to the action. An array of hash values ​​for the action message sequence;

[0103] S52: Sequentially hash the action message sequence values Match all template action messages to obtain a set of key information. set , ,in for The Middle One element, A collection of key information for template action messages The first in indivual element;

[0104] S53: Will All Search results The intersection operation yields a set. ,like If the set is not empty and its size is 1, then proceed to step S54; if If not unique, the output result is not a match; if If the set is empty, the output will be an error.

[0105] S54: Judgment In Whether or not In If they are the same, proceed to S55; otherwise, output an error.

[0106] S55: Judgment Size of the action item set in Whether or not Size of the action item set in If they are the same, proceed to S56; otherwise, output an error.

[0107] S56: Judgment The values ​​attached to each action in the middle and Action Added Value Do they satisfy sequentially? If the condition is met, the output result is correct; otherwise, the output result is incorrect. A collection of key information about action events of array number One element, for The action append value array in the middle Each element.

[0108] Furthermore, the accuracy rate of the automatic burn-in test results in the analysis and verification module is... This represents the percentage of correct test results obtained by the secure and stable control system during test case execution out of the total number of executions since the stress test was initiated. .

[0109] Example 3, the real-time analysis and verification method for automatic stress test results in this example includes:

[0110] Step S1: Based on the IEC 60870-5-103 and IEC 60870-5-104 protocols, the D5000 system's SCMS application receives device action information in real time, reads the historical database, and parses it into device action messages.

[0111] Step S2: Taking multiple fault tests as a unit, define the device action messages with known correct test results as template action messages, and combine them into template action events;

[0112] Step S3: Extract key information from each template action event. ;

[0113] Step S4: Start the burn test, periodically obtain the device action messages of the automatic burn test, and combine the device action messages into test action events based on the fault judgment time of the control substation device;

[0114] Step S5: Perform key information matching and verification on each test action event and template action event in sequence, and output the test results of the device and the accuracy rate of the automatic burn-in test results of the safety and stability control system;

[0115] Step S51: Calculate the key information set of the test action events. ;

[0116] Step S52: Sequentially hash the action message sequence values Search template action message for all scenario sets ;

[0117] Step S53: ... All Search results The intersection operation yields a set. , If the set is not empty and its size is 1, proceed to step S54.

[0118] Step S54: Determine In Whether or not In If they are the same, then proceed to S55.

[0119] Step S55: Determine Size of the action item set in Whether or not Size of the action item set in If they are the same, an error is output; otherwise, the accuracy rate of the automatic stress test result is displayed. This refers to the percentage of correct results obtained by the security and stability control system when executing test cases since the stress test was started, out of the total number of executions. .

[0120] Example 4, based on the same inventive concept, also provides a computing device, including: one or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memory and configured to be executed by the processor, and when the programs are loaded onto the processor, they implement the steps of the real-time analysis and verification method based on the automatic burn-in test results described in any of the above.

[0121] Example 5, based on the same inventive concept, also provides a storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the steps of the automatic burn-in test result real-time analysis and verification method according to any of the above.

[0122] By real-time parsing of device action messages from automatic stress testing based on the scheduling master station communication protocol, and using test scenarios as units, device action messages with known correct test results are defined as template action messages and combined into template action events. Key information is extracted from each set of template action events. Then, stress testing is initiated, and device action messages from automatic stress testing are periodically acquired. Using the fault judgment time of the control substation device as a unit, device action messages are combined into test action events, and key information is matched and verified between each test action event and the template action events. Finally, the device test results and the accuracy rate of the automatic stress testing results for the safety and stability control system are output. This effectively improves the ability of automatic stress testing to analyze and verify action messages in the test result management stage, achieving automation to replace manual work and solving the problems of difficult manual analysis and verification, and missed judgments of erroneous execution results caused by long-term, full-logic testing.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0128] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for real-time analysis and verification of automatic test equipment test results, comprising: The application relates to a device action message acquisition device for automatic machine copying test based on real-time analysis of dispatch master station communication protocol. ​ The device action message comprises an action information group and an action report file. A device action message with a known correct test result is defined as a template action message and is combined into a template action event in units of test scenes. A corresponding key information set is established by extracting each template action event. The machine copying is started, and device action messages for automatic machine copying test are acquired regularly. The device test result and the correct rate of the automatic machine copying test result of the safety and stability control system are output by sequentially matching and checking the key information of each test action event and the template action event. The real-time analysis based on the dispatch master station communication protocol comprises accessing a dispatch end communication protocol or an IEC61850 protocol based on the safety and stability control system.

2. The method of claim 1, wherein the method further comprises: The action information group is a combination of action information of a single action event in units of devices. The action information is a shift message and action amount information generated when the device acts. The shift message contains at least one of a stability control start, a stability control outlet or a fault outlet. The action amount information contains at least one of a line cut-off, a unit cut-off, a DC boost or drop amount, a required unit cut-off amount, a required load cut-off amount, an operation mode, a stability control executed unit cut-off amount, a stability control executed load cut-off amount or a stability control modulated DC amount. The action report file is a file containing fault information, action timing and action behavior in the device action process generated by the device after judging the fault occurrence or device start according to the logic of the device. The test scene comprises at least one of multiple fault test, redundant system test, control strategy test, DC execution station test, action logic test or main transformer overload test.

3. The method of claim 1, wherein the method further comprises: The device action message with a correct test result is a device action message of the safety and stability control system in a long-time and full-logic test environment, which meets at least one of correct start judgment logic, correct element start-stop judgment logic, correct element tripping judgment logic, correct control strategy function or correct control strategy function in a multiple fault test environment. The device action condition is whether each device in the test of the safety and stability control system acts or not.

4. The method of claim 1, wherein the method further comprises: The respective extraction of each set of template action events establishes a corresponding key information set, and the key information set , wherein is the device action case, is the device start absolute time, is the action relative time array, is the action entry ID array, is the action entry description array, is the action attached value array, is the action message sequence hash value array; The action message sequence hash value is a hash value of a sequence of action information and carry action auxiliary values spliced according to the action item ID in sequence. The matching and checking of the key information of each test action event and the template action event comprises: The carry action is accompanied by a value that is the action- accompanied value The number after 50 carries That is Where .

5. The method of claim 1, wherein the method further comprises: The application relates to a device action message acquisition device for automatic machine copying test based on real-time analysis of dispatch master station communication protocol. S51: Set the key information set of the test action event wherein is the device action case, is the device start absolute time, is the action relative time array, is the action entry ID array, is the action entry description array, is the action accompanying value array, is the action message sequence hash value array; S52: Hash the sequence of action messages in turn Match all template action messages to obtain a set of key information , Wherein is the th element in , is the th element in the set of key information of the template action message ;​​ S53: If all searched sets are not empty, then go to step S54; if the intersection of the searched sets is not unique, then output the result as not matched; if the intersection of the searched sets is empty, then output the result as error. S54: If the intersection of the searched sets is a singleton, then output the result as matched. S54: Judgment In Whether or not In If they are the same, proceed to S55; otherwise, output an error. S55: judging the action entry set size in is the same as the action entry set size in , if the same, go to S56; otherwise, output the result as an error; ​ S56: Judgment The values ​​attached to each action in the middle and Action Added Value Do they satisfy sequentially? If the condition is met, the output result is correct; otherwise, the output result is incorrect. A collection of key information about action events of array number One element, for The action append value array in the middle Each element.

6. The method of claim 1, wherein the method further comprises: The automatic hacking test result correctness is The proportion of the number of correct results of the test case executed by the safety and stability control system since the hacking is started to the total number of executions .

7. An automatic test result real-time analysis and verification system, characterized in that, The device action message comprises an action information group and an action report file. A device action message with a known correct test result is defined as a template action message and is combined into a template action event in units of test scenes. A corresponding key information set is established by extracting each template action event. A corresponding key information set is established by extracting each template action event. ​ The test combination module is configured to start the test, periodically acquire the device action message of the automatic test, and combine the device action message into a test action event in a judgment fault time unit of the substation device. The analysis and verification module is configured to sequentially perform key information matching verification on each test action event and template action event, and output the test result of the device and the correctness rate of the automatic test result of the security and stability control system.

8. The automatic test result real-time analysis and verification system of claim 7, wherein, The acquisition module is configured to analyze and verify the test result in real time based on the dispatching master station communication protocol, and the real-time analysis and verification based on the dispatching master station communication protocol specifically includes accessing a dispatching end communication protocol or an IEC 61850 protocol based on the security and stability control system. The action information group is a combination of action information of a single action event in a device unit. The action information is a shift message and action amount information generated when the stability control device acts. The shift message includes at least one of stability control starting, stability control outlet, or fault outlet. The action amount information includes at least one of cut-off line, cut-off unit, DC boost or drop amount, required cut-off unit amount, required cut-off load amount, operation mode, stability control execution cut-off unit amount, stability control execution cut-off load amount, or stability control modulation DC amount. The action report file is a file generated by the device after the device judges the fault occurrence or the device starts, and the file includes fault information, action timing, and action behavior in the device action process.

9. The automatic test result real-time analysis and verification system of claim 7, wherein, The test scene includes at least one of multiple fault test, redundant system test, control strategy test, DC execution station test, action logic test, or main transformer overload test. The device action message with correct test result is a device action message that meets at least one of the following conditions: correct starting judgment logic, correct element start-stop judgment logic, correct element tripping judgment logic, correct control strategy function, or correct control strategy function in a multiple fault test environment.

10. The method of claim 7, wherein the method further comprises: The key information set is extracted from the module Wherein The device action case, The device start absolute time, The action relative time array, The action entry identification ID array, The action entry description array, The action attached value array, The action message sequence hash value array; The device action condition is whether each device in the security and stability control system acts in the test. The action message sequence hash value is a hash value obtained by concatenating the action information and the carry action attached value in sequence according to the action bar identification ID. The carry action is accompanied by a value that is the action- accompanied value The number after 50 carries That is Where .

11. The method of claim 7, wherein the method further comprises: The analysis and verification module is configured to sequentially perform key information matching verification on each test action event and template action event, and output the test result of the device and the correctness rate of the automatic test result of the security and stability control system. S51: Set the key information set of the test action event wherein is the device action case, is the device start absolute time, is the action relative time array, is the action entry ID array, is the action entry description array, is the action accompanying value array, is the action message sequence hash value array; S52: Hash the sequence of action messages in turn Match all template action messages to obtain a set of key information , Wherein is the th element in the set of key information of the template action message , the th element in the set of key information of the template action message​ S53: If all searched sets are not empty, then go to step S54; if the intersection of the searched sets is not unique, then output the result as not matched; if the intersection of the searched sets is empty, then output the result as error. S54: If the intersection of the searched sets is a singleton, then output the result as matched. S54: Judgment In Whether or not In If they are the same, proceed to S55; otherwise, output an error. S55: judge the action entry set size in is same as the action entry set size in , if same, go to S56; otherwise, output result as error; S56: Judgment The values ​​attached to each action in the middle and Action Added Value Do they satisfy sequentially? If the condition is met, the output result is correct; otherwise, the output result is incorrect. A collection of key information about action events of array number One element, for The action append value array in the middle Each element.

12. The method of claim 7, wherein the method further comprises: The automatic copy machine test result correctness rate in the analysis verification module is The proportion of the number of correct results of the test case executed by the safety and stability control system since the copy machine is started to the total number of executions .

13. A computing device, comprising: The analysis and verification module is configured to sequentially perform key information matching verification on each test action event and template action event, and output the test result of the device and the correctness rate of the automatic test result of the security and stability control system. The storage medium stores a computer program, and the computer program includes program instructions, which, when executed by a processor, cause the processor to perform the steps of the automatic test result real-time analysis and verification method according to any one of claims 1 to 6.

14. A storage medium, characterized by ​

Images