An automatic evaluation method for SCD configuration capability

Abstract

The present invention discloses an automatic assessment method for SCD configuration capability. The method comprises the following steps: constructing scoring items and error correction items; importing an exam SCD file and a standard answer SCD file; parsing to generate an SCD exam item set and an SCD standard item set; generating each error correction item and its corresponding error correction item score; parsing the candidate's SCD file to obtain an SCD candidate item set; and automatically assessing all candidate SCD files. The present invention can automatically assess candidate SCD files configured by the candidate in batches, not only improving the rate of SCD configuration capability assessment, but also comprehensively detecting and discovering errors in the candidate's SCD files, greatly improving work efficiency and significantly reducing the auditing workload of relevant professionals.

Description

Technical Field

[0001] The invention is applicable to the technical field of substation secondary system examination and training, and relates to an automatic evaluation method for SCD configuration capability. Background Art

[0002] To promote the healthy and stable development of the power system, departments at all levels of the power system regularly organize training and examinations for personnel involved in the power secondary system, particularly focusing on understanding and correctly configuring SCD files. SCD is the secondary system configuration file that is central to the safe operation of smart substations. Grid professionals must be intimately familiar with it. Frontline professionals must be able to review and verify substation configuration files and verify their correctness through debugging. Some power companies have conducted pilot programs and training for system operations and maintenance personnel on configuring SCD files and downloading instantiated relay protection device configuration files (CIDs and CCDs), laying a solid foundation for the safe operation of smart substations. Currently, smart substation configuration SCD files are large in size and contain numerous lines, ranging from tens of thousands to millions. When assessing professionals' SCD configuration skills, it is difficult to visually identify errors in the SCD files configured by candidates, making manual grading extremely difficult and prone to errors. The automatic evaluation method that uses SCD configuration capabilities can automatically evaluate candidates' SCD files in batches, which can not only improve the speed of marking, but also comprehensively detect and discover error information, greatly improve work efficiency, and significantly reduce the labor intensity of manual review. Summary of the Invention

[0003] The purpose of the present invention is to provide an automatic evaluation method for SCD configuration capability in response to the above-mentioned problems in the prior art.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] An automatic evaluation method for SCD configuration capability includes the following steps:

[0006] Step 1: Construct scoring items and error correction items;

[0007] Each scoring item includes type, scoring mode, scoring value, and deduction value;

[0008] The error correction entry includes erroneous features, or correct features, or both erroneous features and correct features;

[0009] Step 2. Import the exam SCD file and the standard answer SCD file;

[0010] Step 3: Parse the imported exam SCD file to generate an SCD exam item set; parse the imported standard answer SCD file to generate an SCD standard item set.

[0011] Both SCD standard items and SCD exam items include: type, reference address, and data value;

[0012] Step 4: Generate SCD test items and SCD non-test items by comparing SCD test items and SCD standard items, and generate each error correction item and the corresponding error correction item score. The error characteristics of the error correction item are items selected from the SCD test items, and the correct characteristics of the error correction item are items selected from the SCD standard items.

[0013] Step 5: Parse the candidate SCD file to obtain the SCD candidate entry set;

[0014] Step 6: Automatically evaluate all candidates' SCD files according to the SCD standard items and error correction items.

[0015] The automatic evaluation in step 6 mentioned above includes an item-by-item comparison and evaluation process:

[0016] Compare all SCD candidate items with all SCD standard items one by one.

[0017] If an SCD candidate entry matches an SCD standard entry, no points will be deducted for the SCD candidate entry. If no SCD standard entry matches the SCD candidate entry, points will be deducted based on the scoring item according to the type of the SCD candidate entry.

[0018] Deducting points based on the scoring items as described above includes the following steps:

[0019] The scoring mode corresponding to the type is determined as absolute deduction mode or weighted deduction mode;

[0020] If the absolute deduction mode is used, for an SCD candidate entry that cannot find a consistent SCD standard entry, the corresponding type of score will be deducted once the deduction point value is deducted.

[0021] If the weighted deduction mode is used, the score corresponding to the category is divided by the total number of SCD test items corresponding to the same category as the individual score. For an SCD candidate item for which no consistent SCD standard item can be found, the score of the corresponding type is deducted once from the individual score.

[0022] The automatic evaluation in step 6 mentioned above also includes the error correction item evaluation process:

[0023] Traverse all error correction entries and match them with SCD candidate entries:

[0024] If the correction item only includes correct features, if there is no SCD candidate item that matches the correct features of the correction item, the correction item score corresponding to the correction item will be directly deducted;

[0025] If the correction item only includes error characteristics, if there is an SCD candidate item with the same error characteristics as the correction item, the correction item score corresponding to the correction item will be directly deducted;

[0026] If an error correction item includes correct and incorrect features, if there is an SCD candidate item with the same incorrect features as the error correction item, the error correction item score corresponding to the error correction item will be directly deducted; if there is no SCD candidate item with the same incorrect features as the error correction item and no SCD candidate item with the same correct features as the error correction item, the error correction item score corresponding to the error correction item will be directly deducted.

[0027] The automatic evaluation in step 6 mentioned above also includes the evaluation deduplication process:

[0028] The SCD candidate's entry with deducted points in the item-by-item assessment process is compared with the correct features and incorrect features in all corrected entries in the error correction assessment process. If the comparison shows that they are the same entries, the SCD candidate's entry with deducted points is marked as a duplicate deducted point entry.

[0029] The automatic evaluation in step 6 mentioned above also includes the score calculation process:

[0030] Accumulate the deductions for all SCD candidate items that were deducted during the item-by-item comparison assessment process, excluding those marked as repeated deductions, to obtain the total deduction value for the item-by-item comparison assessment; accumulate the deductions for all incorrect items during the error correction assessment process to obtain the total deduction value for the error correction assessment;

[0031] The total score of the item-by-item comparison assessment is deducted from the total deduction of the item-by-item comparison assessment to obtain the total score of the item-by-item comparison assessment; the total score of the error correction item assessment is deducted from the total deduction of the error correction item assessment to obtain the total score of the error correction item assessment; the total score of the item-by-item comparison assessment and the total score of the error correction item assessment are added together to obtain the final test score.

[0032] The beneficial effects of adopting the above technical solution are:

[0033] The present invention can automatically evaluate the candidate's SCD files configured by the candidates in batches, which can not only improve the rate of SCD configuration capability evaluation, but also comprehensively detect and discover error information in the candidate's SCD files, greatly improving work efficiency and significantly reducing the audit labor intensity of relevant professionals. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flowchart of the present invention. DETAILED DESCRIPTION

[0035] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0036] like Figure 1 As shown, an automatic evaluation method for SCD configuration capability includes the following steps:

[0037] Step 1: Configure the evaluation parameters, which include scoring and correction configuration. The scoring configuration consists of multiple scoring items, each of which includes: type, scoring mode, scoring score, and penalty score. The correction configuration consists of multiple correction entries, each of which includes error features, correct features, or both.

[0038] The types described in step 1 include: subnet name, subnet type, subnet description, control block subnet name, control block MAC address, control block APPID, control block VLAN, control block priority, control block MaxTime, control block MinTime, MMS network IP, IED name, IED description, IED type, IED manufacturer, IED version, virtual terminal entry, virtual terminal input, and virtual terminal output.

[0039] The scoring mode described in step 1 is a method for calculating deductions for incorrect SCD candidate entries in the candidate's SCD file, including two modes: absolute deduction mode and weighted deduction mode.

[0040] The scoring value refers to the score corresponding to the type. The initial scoring value is the highest score of the type, and the minimum scoring value is 0. The deduction value refers to the score deduction amount in the absolute deduction mode. The total score of the item-by-item comparison evaluation is obtained by accumulating the scoring values of all types.

[0041] Each correction entry can be manually configured to include all required error and correct features. Both error and correct features contain three keywords: type, reference address, and data value. The types in the error and correct features correspond to the types in the scoring items.

[0042] Step 2: Import the exam SCD file and the model answer SCD file. The exam SCD file is the original SCD file prepared before the assessment, containing pre-set entries for missing or incorrect information. The model answer SCD file is the correct SCD file prepared before the assessment.

[0043] Step 3: Parse the imported exam SCD file and generate an SCD exam entry set. The SCD exam entry includes: type, reference address, and data value. The type of the SCD exam entry is consistent with the type of the scoring item described in step 1.

[0044] Parse the imported standard answer SCD file and generate a set of SCD standard entries. SCD standard entries include: type, reference address, and data value. The type of SCD standard entry is consistent with the type of the scoring item described in step 1;

[0045] The parsed imported test SCD file and standard answer SCD file described in step 3 are both to parse the Communication parameter nodes and IED device nodes of the corresponding test SCD file and the Communication parameter nodes and IED device nodes of the standard answer SCD file, and generate an SCD test item set based on the test SCD file and an SCD standard item set based on the standard answer SCD file.

[0046] Step 4: Based on the exam SCD file and the standard answer SCD file, generate SCD exam items and SCD non-exam items by comparing the SCD exam items with the SCD standard items. The SCD standard items are compared with the SCD exam items. If the comparison is inconsistent, the SCD standard item is an SCD exam item; if the comparison is consistent, the SCD standard item is an SCD non-exam item.

[0047] Correction items can be selected and generated based on SCD exam items and SCD standard items. Each correction item includes incorrect features, correct features, or both. Incorrect features are selected from the SCD exam items obtained from the exam SCD file, while correct features are selected from the SCD standard items obtained from the standard answer SCD file. Each correction item must have a corresponding correction item score. The total correction item score is calculated by summing the correction item scores for all correction items.

[0048] Step 5: Batch import and parse candidate SCD files. Due to the large number of candidate SCD files, each SCD file takes up a lot of memory, consumes a lot of CPU, and takes a long time to parse. Simultaneously importing a large number of candidate SCD files can lead to unpredictable issues. Therefore, a serial parsing process is proposed, parsing candidate SCD files in batches. After parsing one candidate SCD file, resources are released before parsing the next candidate SCD file.

[0049] The parsing of the candidate SCD file described in step 5 adopts the same parsing process as step 3 to create an SCD candidate entry set based on the candidate SCD file. The SCD candidate entry includes: type, reference address, and data value.

[0050] Step 6: Automatically evaluate all candidates' SCD files. The automatic evaluation process includes item-by-item comparison, error correction, duplicate removal, and score calculation.

[0051] The item-by-item comparison and evaluation process includes the following steps: item-by-item comparison of all SCD candidate entries created by the candidate SCD file with all SCD standard entries created by the standard answer SCD file. The comparison content includes: type, reference address, and data value.

[0052] If an SCD candidate entry matches an SCD standard entry, no points will be deducted for the SCD candidate entry. If no SCD standard entry matches the SCD candidate entry, points will be deducted based on the scoring item according to the type of the SCD candidate entry.

[0053] Deducting points based on the scoring items includes the following steps:

[0054] The scoring mode corresponding to the type is determined as absolute deduction mode or weighted deduction mode;

[0055] If the absolute deduction mode is used, for an SCD candidate entry that cannot find a consistent SCD standard entry, the corresponding type of score will be deducted once the deduction point value is deducted.

[0056] If the weighted deduction mode is used, the score corresponding to the category is divided by the total number of SCD test items corresponding to the same category as the individual score. For an SCD candidate item for which no consistent SCD standard item can be found, the score of the corresponding type is deducted once from the individual score.

[0057] The lowest score is 0, and no further points will be deducted after the score reaches 0.

[0058] The error correction evaluation process described in step 6 includes: traversing all error correction entries and matching the error correction entries with the SCD candidate entries. The comparison includes the error characteristics, correct feature types, reference addresses, and data values in the error correction entries with the types, reference addresses, and data values of the original SCD entries. Three situations are handled: 1. If the error correction entry contains only correct features, if there is no SCD candidate entry that matches the correct features of the error correction entry, the error correction item score corresponding to the error correction entry is directly deducted; 2. If the error correction entry contains only incorrect features, if there is an SCD candidate entry that matches the incorrect features of the error correction entry, the error correction item score corresponding to the error correction entry is directly deducted; 3. If the error correction entry contains both correct and incorrect features, if there is an SCD candidate entry that matches the incorrect features of the error correction entry, the error correction item score corresponding to the error correction entry is directly deducted; if there is no SCD candidate entry that matches the incorrect features of the error correction entry and matches the correct features of the error correction entry, the error correction item score corresponding to the error correction entry is directly deducted.

[0059] Furthermore, during the assessment deduplication process described in step 6, duplicate deductions may be stored during the item-by-item comparison between the assessment process and the error correction process. This requires comparing the deduction items from both methods. This comparison involves comparing the correct and incorrect features of the SCD candidate's item that was deducted during the item-by-item assessment process with the correct and incorrect features of all corrected items during the error correction process. If the comparison reveals identical items, the SCD candidate's item that was deducted is marked as a duplicate deduction item.

[0060] Furthermore, in the scoring calculation process described in step 6, the deductions of the SCD candidate items that were deducted during the item-by-item comparison and evaluation process, except for the items marked as repeated deductions, are accumulated to obtain the total deduction value of the item-by-item comparison and evaluation; all incorrect items in the error correction item evaluation process are accumulated according to their deduction values to obtain the total deduction value of the error correction item evaluation.

[0061] The total score of the item-by-item comparison assessment is deducted from the total deduction of the item-by-item comparison assessment to obtain the total score of the item-by-item comparison assessment; the total score of the error correction item assessment is deducted from the total deduction of the error correction item assessment to obtain the total score of the error correction item assessment; the total score of the item-by-item comparison assessment and the total score of the error correction item assessment are added together to obtain the final test score.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. An automatic evaluation method for SCD configuration capability, characterized in that: The following steps are involved: Step 1: Construct scoring items and error correction items; Each scoring item includes type, scoring mode, scoring value, and deduction value; The error correction entry includes erroneous features, or correct features, or both erroneous features and correct features; Step 2. Import the exam SCD file and the standard answer SCD file; Step 3: Parse the imported exam SCD file to generate an SCD exam item set; parse the imported standard answer SCD file to generate an SCD standard item set. Both SCD standard items and SCD exam items include: type, reference address, and data value; Step 4: Generate SCD test items and SCD non-test items by comparing SCD test items and SCD standard items, and generate each error correction item and the corresponding error correction item score. The error characteristics of the error correction item are items selected from the SCD test items, and the correct characteristics of the error correction item are items selected from the SCD standard items. Step 5: Parse the candidate SCD file to obtain the SCD candidate entry set; Step 6: Automatically evaluate all candidates' SCD files according to the SCD standard items and error correction items. The automatic evaluation in step 6 mentioned above includes an item-by-item comparison and evaluation process: Compare all SCD candidate items with all SCD standard items one by one. If an SCD candidate entry matches an SCD standard entry, no points will be deducted for the SCD candidate entry. If no SCD standard entry matches the SCD candidate entry, points will be deducted based on the scoring item according to the type of the SCD candidate entry. Deducting points based on the scoring items as described above includes the following steps: The scoring mode corresponding to the type is determined as absolute deduction mode or weighted deduction mode; If the absolute deduction mode is used, for an SCD candidate entry that cannot find a consistent SCD standard entry, the corresponding type of score will be deducted once the deduction point value is deducted. If the weighted deduction mode is used, the score corresponding to the category is divided by the total number of SCD test items corresponding to the same category as the individual score. For an SCD candidate item for which no consistent SCD standard item can be found, the score of the corresponding type is deducted once from the individual score. The automatic evaluation in step 6 mentioned above also includes the error correction item evaluation process: Traverse all error correction entries and match them with SCD candidate entries: If the correction item only includes correct features, and there is no SCD candidate item that matches the correct features of the correction item, the correction item score corresponding to the correction item will be directly deducted; If the correction item only includes error characteristics, if there is an SCD candidate item with the same error characteristics as the correction item, the correction item score corresponding to the correction item will be directly deducted; If the correction item includes correct features and incorrect features, if there is an SCD candidate item that is consistent with the incorrect features of the correction item, the correction item score corresponding to the correction item will be directly deducted; if there is no SCD candidate item that is consistent with the incorrect features of the correction item and no SCD candidate item that is consistent with the correct features of the correction item, the correction item score corresponding to the correction item will be directly deducted. The automatic evaluation in step 6 mentioned above also includes the evaluation deduplication process: Compare the SCD candidate's entry that was deducted in the item-by-item assessment process with the correct features and incorrect features of all corrected entries in the error correction assessment process. If the comparison shows that they are the same entry, mark the SCD candidate's entry that was deducted as a duplicate deduction entry. The automatic evaluation in step 6 mentioned above also includes the score calculation process: Accumulate the deductions for all SCD candidate items that were deducted during the item-by-item comparison assessment process, excluding those marked as repeated deductions, to obtain the total deduction value for the item-by-item comparison assessment; accumulate the deductions for all incorrect items during the error correction assessment process to obtain the total deduction value for the error correction assessment; The total score of the item-by-item comparison assessment is deducted from the total deduction of the item-by-item comparison assessment to obtain the total score of the item-by-item comparison assessment; the total score of the error correction item assessment is deducted from the total deduction of the error correction item assessment to obtain the total score of the error correction item assessment; the total score of the item-by-item comparison assessment and the total score of the error correction item assessment are added together to obtain the final test score.

Images