Verification coverage analysis method for fault diagnosis and related device

Abstract

The embodiment of the application provides a verification coverage analysis method for fault diagnosis and related equipment, and belongs to the technical field of network intelligent operation and maintenance. The method comprises the following steps: determining a corresponding initial verification dimension list according to a fault type indicated by a fault event, and determining a device context according to a device involved in the fault event; performing dimension pruning on the initial verification dimension list according to the device context to obtain a target verification dimension list; obtaining a test task that has been successfully executed in a diagnosis process for the fault event, and then querying a task dimension mapping table according to the test task to determine a covered verification dimension; and performing statistical analysis on the covered verification dimension in the target verification dimension list to determine a verification coverage. The application can realize reliable measurement of verification sufficiency in a fault diagnosis process.

Description

Technical Field

[0001] This application relates to the field of network intelligent operation and maintenance technology, and in particular to a verification coverage analysis method and related equipment for fault diagnosis. Background Technology

[0002] In large-scale network operations and maintenance, fault diagnosis is a core component of ensuring business continuity. Operations personnel typically rely on automated testing systems to perform a series of diagnostic tasks (such as Ping, Traceroute, SNMP queries, etc.) to pinpoint the root cause of faults. However, for a long time, objective quantitative indicators have been lacking regarding the adequacy of the diagnostic process and the reliability of the conclusions.

[0003] Existing automated maintenance diagnostic systems can only track the completion rate of diagnostic tasks. However, the task completion rate is not directly related to the adequacy of diagnostic verification. Even if 10 tasks are executed, they may all focus on only 1-2 dimensions, completely omitting key verification dimensions and failing to reflect the completeness and reliability of the diagnosis. Even with standardized diagnostic processes, they only involve the arrangement of fixed task sequences. If the arrangement is reasonable and all fixed tasks are successfully executed, the verification coverage can be considered complete. However, when there are task execution failures or dynamically arranged tasks, the adequacy of the fault diagnosis process cannot be measured at all, which may lead to incorrect root cause localization or fault recurrence. Summary of the Invention

[0004] The main objective of this application is to propose a verification coverage analysis method and related equipment for fault diagnosis, aiming to achieve a reliable measurement of verification adequacy in the fault diagnosis process.

[0005] To achieve the above objectives, one aspect of this application proposes a verification coverage analysis method for fault diagnosis, comprising the following steps: The corresponding initial verification dimension list is determined based on the fault type indicated by the fault event, and the device context is determined based on the device involved in the fault event; Based on the device context, the initial verification dimension list is pruned to obtain the target verification dimension list; Obtain the test tasks that have been successfully executed in the diagnostic process for the fault event, and then query the task dimension mapping table based on the test tasks to determine the verification dimensions that have been covered. Perform statistical analysis on the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate.

[0006] In some embodiments, determining the corresponding initial verification dimension list based on the fault type indicated by the fault event includes the following steps: When a fault event indicates a fault type, the corresponding standard verification dimension template is queried according to the fault type, and the standard verification dimension template is used as the initial verification dimension list. When a fault event indicates multiple fault types, the corresponding standard verification dimension template is queried according to each fault type, and the multiple standard verification dimension templates are merged to obtain an initial verification dimension list.

[0007] In some embodiments, the step of pruning the initial verification dimension list according to the device context to obtain the target verification dimension list includes the following steps: Based on the device role, supported protocols, and network topology location indicated by the device context, the initial verification dimension list is pruned to obtain an intermediate verification dimension list. Based on the applicable conditions indicated in the intermediate verification dimension list, verify whether the device context-related verification data meets the applicable conditions. If not, delete the verification dimension corresponding to the applicable conditions from the intermediate verification dimension list to obtain the target verification dimension list.

[0008] In some embodiments, determining the covered verification dimensions by querying the task dimension mapping table based on the probing task includes the following steps: For each test task that has been successfully executed, the task dimension mapping table is queried to determine the verification dimension associated with the test task, and the query result of the test task is obtained. Perform dimension deduplication on the query results of each successfully executed test task to determine the verified dimensions that have been covered.

[0009] In some embodiments, the statistical analysis of the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate includes the following steps: Perform statistical analysis on the verification dimensions already covered in the target verification dimension list to determine the number of dimensions already covered; The base coverage rate, used to measure the verification coverage rate, is calculated based on the number of covered dimensions and the total number of dimensions in the target verification dimension list.

[0010] In some embodiments, the statistical analysis of the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate includes the following steps: Obtain the weight values ​​of all verification dimensions in the target verification dimension list; The weight values ​​of the verification dimensions already covered in the target verification dimension list are accumulated to obtain the verification dimension score; and the weight values ​​of all verification dimensions in the target verification dimension list are accumulated to obtain the verification dimension total score. Based on the verification dimension scores and the total verification dimension scores, a weighted coverage rate is calculated to measure verification coverage.

[0011] In some embodiments, the verification coverage analysis method for fault diagnosis further includes the following steps: If the verification coverage is lower than a preset threshold, a corresponding probing task is matched based on the verification dimensions not covered in the target verification dimension list. The matched test task is pushed to the upstream diagnostic decision engine for task execution decision, so as to update the first verification coverage.

[0012] To achieve the above objectives, another aspect of this application proposes a verification coverage analysis system for fault diagnosis, comprising: The data management and analysis module is used to determine the corresponding initial verification dimension list based on the fault type indicated by the fault event, and to determine the device context based on the device involved in the fault event. The trimming module is used to trim the initial verification dimension list according to the device context to obtain the target verification dimension list. The mapping management module is used to obtain the test tasks that have been successfully executed in the diagnostic process for the fault event, and then query the task dimension mapping table according to the test tasks to determine the verification dimensions that have been covered. The verification coverage analysis module is used to perform statistical analysis on the verification dimensions that have been covered in the target verification dimension list to determine the verification coverage rate.

[0013] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0015] The embodiments of this application include at least the following beneficial effects: This application provides a verification coverage analysis method and related equipment for fault diagnosis. This method determines a corresponding initial verification dimension list based on the fault type indicated by the fault event, and determines the device context based on the devices involved in the fault event. Then, it prunes the initial verification dimension list according to the device context to obtain a target verification dimension list. This target verification dimension list is adapted to the specific device capabilities, avoids interference from invalid dimensions, accurately provides the dimensions to be verified, and improves diagnostic efficiency. It obtains the test tasks that have been successfully executed in the fault event diagnosis process, and then queries the task dimension mapping table based on these test tasks to determine the covered verification dimensions. Finally, it performs statistical analysis on the covered verification dimensions in the target verification dimension list to determine the verification coverage rate, thus achieving a reliable measurement of the adequacy of verification in the fault diagnosis process. Attached Figure Description

[0016] Figure 1 This is a flowchart of the verification coverage analysis method for fault diagnosis provided in the embodiments of this application; Figure 2 yes Figure 1 The flowchart of step S102 in the document; Figure 3 yes Figure 1 The flowchart of step S103 in the process; Figure 4 yes Figure 1 The flowchart of step S104 in the process; Figure 5 yes Figure 1 A flowchart of another embodiment of step S104 in the process; Figure 6 This is a flowchart of a verification coverage analysis method for fault diagnosis provided in another embodiment of this application; Figure 7 This is a schematic diagram of a verification coverage analysis system for fault diagnosis provided in an embodiment of this application; Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0019] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0020] Fault types are diagnostic classifications based on the root cause or symptoms of a fault, such as link faults, device faults, service faults, DNS faults, etc., used to index the corresponding verification dimension templates. Fault types can be flexibly defined and expanded according to actual operation and maintenance scenarios.

[0021] The standard verification dimension template is a predefined list of standard verification dimensions for each type of fault, containing the core observable state quantities that should be verified for that type of fault. For example, the link fault template includes dimensions such as link connectivity, the liveness of devices at both ends, the impact on downstream services, and interface status.

[0022] Verification dimensions are the metrics that need to be verified for a fault. They are observable state variables used to verify a specific aspect of the fault, such as "link connectivity," which can be verified via Traceroute or Ping. Each dimension has a clear verification method and expected results. Whether a dimension is covered (i.e., whether the dimension is verified in this diagnosis) is only related to whether the corresponding task was successfully executed, and is unrelated to whether the result returned by the test task is positive or negative.

[0023] Device context refers to the device information involved in the fault, including device role (core / edge), supported management protocols (SNMP / NetConf, etc.), network topology location, and other relevant information about the devices involved in this fault event. In this embodiment, the device context is used to trim inapplicable dimensions from the fault type template during template instantiation.

[0024] Template instantiation and trimming: Based on the device context, inapplicable verification dimensions are removed from the standard fault type templates, generating a list of dimensions that should be verified for this diagnosis. For example, if the device does not support the SNMP network protocol, the "Interface Status" dimension is removed; if the applicable conditions of a dimension are not met, it is also removed.

[0025] The task dimension mapping table stores the relationships between testing tasks and validation dimensions. A task can be associated with multiple dimensions, and a dimension can be covered by multiple tasks. Predefined many-to-many mapping relationships are used to count covered dimensions.

[0026] Successful execution means the test command is issued, executed, and returns a result normally, regardless of whether the result is positive (e.g., Ping successful) or negative (e.g., Ping timed out). If a system-level exception prevents the task from completing, it is not considered a successful execution.

[0027] Covered dimensions (i.e., covered verification dimensions) are the verification dimensions covered by successfully executed probing tasks during the diagnostic process. If multiple tasks cover the same dimension, it is counted only once. Tasks incomplete solely due to system-level anomalies are not associated with any verification dimensions and are not included in the covered dimension statistics. Once a dimension is marked as covered, it generally remains permanently effective throughout the entire diagnostic cycle of the fault event, unless it is deemed uncovered according to the rules.

[0028] The total number of dimensions should be verified after the template has been instantiated and clipped. If the clipped list is empty, the total number of dimensions is zero.

[0029] Validation coverage is a metric that measures the adequacy of validation in the diagnostic process, including basic coverage and weighted coverage.

[0030] The dimension deduplication rule marks the dimension as covered only once when multiple tasks cover the same verification dimension, avoiding duplicate counting and improving the accuracy of coverage calculation.

[0031] The Configuration Management Database (CMDB) is a database that stores basic operational data such as device information, asset information, and relationships. It serves as the data source for verifying the applicability of dimensions.

[0032] In operational practice, different fault types (such as link failures, device downtime, and service anomalies) require different verification dimensions, and the verification tasks that can be executed for the same fault in different device environments are also limited by device capabilities (such as whether SNMP is supported, device roles, etc.). Currently, automated operational diagnostic systems can only count the completion rate of probing tasks (e.g., 8 out of 10 tasks were executed), but cannot measure whether the diagnostic process covers all the key dimensions that should be verified. Operational personnel often rely on personal experience to judge "whether enough checks have been done," leading to huge differences in the sufficiency of verification when different personnel diagnose the same fault, and even omitting key verification steps, resulting in incorrect root cause location or fault recurrence. According to industry operational experience statistics, in fault diagnosis relying on manual experience, the omission of key verification dimensions is one of the main reasons for incorrect root cause location and fault recurrence.

[0033] In view of this, this application provides a verification coverage analysis method and related equipment for fault diagnosis. This method determines an initial verification dimension list based on the fault type indicated by the fault event, and determines the device context based on the devices involved in the fault event. Then, it prunes the initial verification dimension list according to the device context to obtain a target verification dimension list. This target verification dimension list is adapted to the specific device capabilities, avoids interference from invalid dimensions, accurately provides the dimensions to be verified, and improves diagnostic efficiency. It obtains the test tasks that have been successfully executed in the fault event diagnosis process, and then queries the task dimension mapping table based on these test tasks to determine the covered verification dimensions. Finally, it performs statistical analysis on the covered verification dimensions in the target verification dimension list to determine the verification coverage rate, thus achieving a reliable measurement of the adequacy of verification in the fault diagnosis process.

[0034] The verification coverage analysis method for fault diagnosis provided in this application relates to the field of intelligent network operation and maintenance technology. This verification coverage analysis method for fault diagnosis can be applied to terminals or servers. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network.

[0035] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0036] Figure 1 This is an optional flowchart of the verification coverage analysis method for fault diagnosis provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104.

[0037] S101, determine the corresponding initial verification dimension list based on the fault type indicated by the fault event, and determine the device context based on the device involved in the fault event; S102, trim the initial verification dimension list according to the device context to obtain the target verification dimension list; S103, obtain the test tasks that have been successfully executed in the diagnostic process for the fault event, and then query the task dimension mapping table based on the test tasks to determine the verification dimensions that have been covered. S104. Perform statistical analysis on the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate.

[0038] Steps S101 to S104, as illustrated in this embodiment, determine the corresponding initial verification dimension list based on the fault type indicated by the fault event, and determine the device context based on the devices involved in the fault event. Then, the initial verification dimension list is pruned according to the device context to obtain the target verification dimension list. This target verification dimension list is adapted to the specific device capabilities, avoids interference from invalid dimensions, accurately provides the dimensions to be verified, and improves diagnostic efficiency. The successfully executed test tasks in the diagnostic process for the fault event are obtained, and the task dimension mapping table is queried based on these test tasks to determine the covered verification dimensions. Then, statistical analysis is performed on the covered verification dimensions in the target verification dimension list to determine the verification coverage rate, achieving a reliable measurement of the sufficiency of verification in the fault diagnosis process.

[0039] In step S101 of some embodiments, the fault event is obtained based on network alarm data, which indicates one or more fault types of the current fault event. A corresponding initial verification dimension list can be determined by querying the fault type. The initial verification dimension list defines the verification dimensions that should be completed when diagnosing the fault event. The fault event-related data also indicates the network devices involved in this network fault and their context. The device context includes device roles, supported management protocols, network topology location, etc.

[0040] In some embodiments, determining the corresponding initial verification dimension list based on the fault type indicated by the fault event in step S101 may include, but is not limited to, the following steps: When a fault event indicates a fault type, query the corresponding standard verification dimension template based on the fault type and use the standard verification dimension template as the initial verification dimension list. When a fault event indicates multiple fault types, the corresponding standard verification dimension template is queried according to each fault type, and the multiple standard verification dimension templates are merged to obtain an initial verification dimension list.

[0041] In this embodiment, the system predefines standard verification dimension templates for each fault type. Each verification dimension includes information such as dimension ID, dimension description, default verification method, and applicable conditions. The standard verification dimension templates can be obtained from expert experience or historical data mining and support dynamic updates. Fault types can be flexibly defined and expanded according to actual operation and maintenance scenarios, and are not limited to the types listed in this embodiment.

[0042] When a fault event has only one fault type, the corresponding standard verification dimension template is queried according to the fault type, and the standard verification dimension template is the initial verification dimension list.

[0043] When a fault event involves multiple fault types, the system merges the standard verification dimension templates corresponding to these fault types and removes duplicate verification dimensions to form an initial verification dimension list. In another example, the system can directly merge the standard verification dimension templates corresponding to multiple fault types to form the initial verification dimension list. Subsequently, for each fault type in the initial verification dimension list, the standard verification dimension template is instantiated and trimmed based on the corresponding device context. All trimmed dimension lists are then merged, and duplicate dimensions are removed to obtain the target verification dimension list. It is understood that if the verification method or applicable conditions for the same dimension differ in different templates, all differing items are retained as independent dimensions, and coverage is subsequently calculated separately.

[0044] The criteria for determining dimension deduplication are as follows: if the core verification object of two verification dimensions is the same (e.g., both are "device survival"), and the difference in verification methods does not change their essential verification objective, then they are considered as the same dimension and deduplicated; if the difference in verification methods causes an essential change in the verification objective (e.g., one verifies "device survival" and the other verifies "device performance"), then they are retained as independent dimensions.

[0045] This embodiment provides a predefined standard verification dimension template for each type of fault, offering a unified verification checklist for different fault types, avoiding bias from human experience, and improving the standardization and reproducibility of the diagnostic process.

[0046] In step S102 of some embodiments, inapplicable dimensions are trimmed according to the device role, supported protocols, topology location and applicable conditions of the verification dimensions in the device context, and a list of dimensions to be verified (i.e., the target verification dimension list) is dynamically generated to adapt to specific device capabilities, avoid interference from invalid dimensions, ensure the accuracy of the verification list, and improve diagnostic efficiency.

[0047] In some embodiments, please refer to Figure 2 Step S102 may include, but is not limited to, the following steps: S201, based on the device role, supported protocols, and network topology location indicated by the device context, the initial verification dimension list is pruned to obtain an intermediate verification dimension list; S202, based on the applicable conditions indicated in the intermediate verification dimension list, verify whether the verification data related to the device context meets the applicable conditions. If not, delete the verification dimension corresponding to the applicable conditions from the intermediate verification dimension list to obtain the target verification dimension list.

[0048] In this embodiment, the fault template is trimmed according to the device context. The core actions include: Remove dimensions that do not support the protocol of the device (e.g., remove the "interface status" dimension if SNMP is not supported). Remove dimensions that are irrelevant to the device role (e.g., edge devices do not need to verify the backbone link). Based on the predefined applicable conditions for each verification dimension, the relevant verification data queried based on the device context is validated. If the current fault scenario represented by the verification data does not meet the applicable condition (e.g., "related service does not exist"), the dimension is removed from the initial verification dimension list. The verification data for the applicable conditions comes from fault alarm information of devices related to the device context, the device context itself, the configuration management database (CMDB), and other basic operational data related to the faulty device. All applicable dimensions are retained to generate a verification dimension list for this diagnostic target, and the total number of dimensions after instantiation is recorded. .

[0049] It should be noted that static validation of the applicable conditions for dimensions is performed during pruning, meaning that the basic operational data at the time of pruning is used as the standard. Changes in the basic operational data regarding the applicable conditions during task execution (such as the temporary shutdown of related services) will not affect the already generated list of target validation dimensions. Accordingly, after pruning is completed, The process remains constant throughout the entire diagnostic cycle; regardless of subsequent changes in the underlying data regarding the device context and applicable conditions, the trimming and modification will not be re-executed. Static validation rules ensure consistency in coverage calculations, avoiding statistical biases caused by dynamic changes.

[0050] In step S103 of some embodiments, the task dimension mapping table is pre-established and can be dynamically expanded. The task dimension mapping table can adopt a many-to-many relationship design, that is, one task can be associated with multiple dimensions, and one dimension can be covered by multiple tasks. In the diagnostic process for fault events, the pre-established task dimension mapping table is queried according to the successfully executed probing tasks to determine the covered verification dimensions, and the covered verification dimensions in the target verification dimension list can be marked for subsequent statistics.

[0051] It should be noted that a successfully executed test task means that as long as the task instruction is issued, executed and returns a result (regardless of whether the result is positive or negative), it is considered successfully executed and participates in the dimension coverage marking. If the task is not completed due to system-level anomalies such as probe failure or network timeout, it is not considered successfully executed and will not participate in the dimension coverage statistics.

[0052] In some embodiments, please refer to Figure 3 Step S103, which involves querying the task dimension mapping table based on the testing task to determine the covered verification dimensions, may include, but is not limited to, the following steps: S301. For each test task that has been successfully executed, query the task dimension mapping table to determine the verification dimension associated with the test task and obtain the query results of the test task. S302, perform dimension deduplication on the query results of each successfully executed test task to determine the verified dimensions that have been covered.

[0053] In this embodiment, for each successfully executed probing task, the mapping table is queried to obtain its associated verification dimensions, and these verification dimensions are marked as covered in the target verification dimension list. If multiple probing tasks cover the same verification dimension, the same verification dimension is deduplicated, meaning that verification dimensions covered by multiple probing tasks in the target verification dimension list are only marked once. This dimension deduplication rule avoids subsequent errors in the number of covered dimensions. Duplicate statistics are not included. Tasks that fail to execute (i.e., tasks that are not completed due to system-level exceptions) are not associated with any validation dimensions and are not included in the statistics of covered dimensions.

[0054] Understandably, the verification dimension coverage marker for the same fault event remains in effect throughout the entire diagnostic process and will not be reset due to supplementary task execution or task retries. Once a dimension is marked as covered, it remains permanently effective throughout the entire fault event's diagnostic cycle and will not be reset due to subsequent execution anomalies in the same dimension (including system-level anomalies causing task incompleteness). The retry strategy for a single probing task does not affect the verification dimension coverage marker and coverage statistics logic. Only when the result of an executed task is manually determined to be invalid or erroneous can the corresponding dimension's coverage marker be manually reset, and the number of covered dimensions and verification coverage be recalculated.

[0055] This embodiment establishes a task dimension mapping table, counts the dimensions covered by executed testing tasks, and obtains the number of covered dimensions through deduplication rules, achieving automatic statistics of covered dimensions. It supports deduplication of multiple tasks covering the same dimension, improving statistical accuracy. The many-to-many mapping relationship allows multiple dimensions to be verified simultaneously in a single task execution, avoiding dimension coverage omissions and redundant task execution caused by one-to-one mapping. The rule of marking permanent effect ensures the stability of statistical results, and the manual intervention rule provides anomaly handling capabilities.

[0056] In step S104 of some embodiments, statistical analysis is performed on the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate. The verification coverage rate is used to measure the comprehensiveness of the verification in the current diagnostic process, and the verification coverage rate may include at least one of the basic coverage rate and the weighted coverage rate.

[0057] It is understood that the application scenarios of the verification coverage index in this application embodiment include: as input for root cause confidence measurement; used alone for the integrity assessment of the diagnostic process, and if the verification coverage is lower than a preset threshold (such as 0.8), it indicates that the verification may be insufficient and additional tasks need to be performed.

[0058] In some embodiments, please refer to Figure 4 Step S104 may include, but is not limited to, the following steps: S401, Perform statistical analysis on the verification dimensions already covered in the target verification dimension list to determine the number of dimensions already covered; S402, calculate the base coverage rate to measure the validation coverage rate based on the number of covered dimensions and the total number of dimensions in the target validation dimension list.

[0059] In this embodiment, the base coverage is calculated as the verification coverage according to the following rules: like ,but ; The number of dimensions already covered. This represents the total number of dimensions; like Then, the value retrieval method is selected according to the business rules: If no verification method is available due to device limitations (e.g., the device does not support any protocol), the coverage is recorded as 1.0, indicating that further verification is not possible under the current conditions. If the fault type itself does not require verification in this scenario (e.g., edge devices do not need to verify backbone links), it is marked as "not applicable".

[0060] This embodiment provides objective and reproducible diagnostic process validation adequacy metrics, and clarifies extreme scenarios ( The processing rules ensure the integrity of the solution and avoid statistical ambiguity in boundary cases.

[0061] In other embodiments, please refer to Figure 5 Step S104 may include, but is not limited to, the following steps: S501, obtain the weight values ​​of all verification dimensions in the target verification dimension list; S502, accumulate the weight values ​​of the verification dimensions already covered in the target verification dimension list to obtain the verification dimension score; and accumulate the weight values ​​of all verification dimensions in the target verification dimension list to obtain the verification dimension total score; S503, calculate the weighted coverage rate used to measure validation coverage based on the validation dimension scores and the total validation dimension scores.

[0062] In this embodiment, weight values ​​are preset for different validation dimensions in the standard validation dimension template (e.g., core dimension weight 0.6, secondary dimension weight 0.4), and the weighted coverage is calculated. : ; in, This indicates the first item in the target validation dimension list. Whether each verification dimension is covered (0 or 1). Weight values ​​can be preset based on expert experience or dynamically generated by statistically analyzing the contribution of each dimension to root cause localization using historical fault diagnosis data. Weighted coverage can be output in parallel with basic coverage, each adapting to different scenarios: basic coverage is used to assess the overall completeness of the diagnostic process, while weighted coverage is used to focus on assessing the verification status of core fault dimensions.

[0063] In some embodiments, please refer to Figure 6 The verification coverage analysis method for fault diagnosis in this application embodiment may also include, but is not limited to, the following steps: S601, when the verification coverage is lower than the preset threshold, match the corresponding probing task according to the verification dimensions not covered in the target verification dimension list; S602 pushes the matched probing task to the upstream diagnostic decision engine for task execution decision-making, in order to update the first verification coverage.

[0064] In this embodiment, when the verification coverage (such as basic coverage) is lower than a preset threshold, the system can automatically match the corresponding probing task based on the uncovered dimensions, generate supplementary diagnostic task suggestions, and push them to the upstream diagnostic decision engine, which will then decide whether to execute the supplementary task and the execution order. After the supplementary task is successfully executed, the system updates the covered verification dimensions in real time, thereby updating the verification coverage.

[0065] The system updates the covered dimension markers and verification coverage in real time based on the execution progress of the testing tasks, and synchronizes this information to the diagnostic management interface. This allows maintenance personnel to monitor the adequacy of the diagnostics in real time and trigger supplementary diagnostics in advance. If the fault diagnosis process is terminated or closed midway, the system calculates the current verification coverage based on the tasks that have been successfully executed, and synchronizes this information to the fault work order archive.

[0066] In this embodiment, the standard verification dimension template can also be dynamically optimized based on historical diagnostic data. For example, frequently missed verification dimensions can be automatically added to the standard verification dimension template for the corresponding fault type based on statistics of historical successful diagnostic cases; or the applicable conditions of verification dimensions can be dynamically adjusted based on the pruning frequency of device context. For example, if a certain verification dimension is pruned in 80% of scenarios (80% is an exemplary threshold, which can be flexibly adjusted according to the actual operation and maintenance scenario), then the verification dimension is marked as "non-core" or its weight is adjusted. Template optimization needs to be manually reviewed and confirmed before it takes effect to avoid verification standard deviation caused by automatic adjustment.

[0067] The methods of the embodiments of this application will now be described in conjunction with specific applications.

[0068] Example 1: Calculate the coverage rate to verify the full coverage scenario for link failure verification.

[0069] The standard verification dimension templates for link failures are shown in Table 1: Table 1

[0070] Device context: This fault event involves devices CE01 (core router, supporting SNMP) and PE02 (aggregation switch, not supporting SNMP).

[0071] Instantiation pruning: Dimension D04 is removed because PE02 does not support SNMP; the applicable condition "associated service exists" for D03 is met (confirmed by CMDB query), and it is retained. After instantiation, the dimension list should be verified to be {D01, D02, D03}. .

[0072] The task dimension mapping table (predefined) is shown in Table 2: Table 2

[0073] Executed tasks: During the diagnostic process, T1 (Ping CE01Loopback), T2 (Traceroute CE01→PE02), and T3 (HTTPGet video source station) were executed successfully (Ping timeout or success is considered as success). SNMP query was not executed because the device does not support it.

[0074] Dimensions already covered: D01, D02, D03 .

[0075] Validation coverage calculation: The verification is sufficient.

[0076] Example 2: The following scenarios are covered in the source station fault verification coverage calculation.

[0077] The standard verification dimensions template for origin server failures is shown in Table 3: Table 3

[0078] Device context: The origin server is located in a data center, with no special restrictions; all dimensions apply. .

[0079] Task Execution: This diagnostic only executed T1 (HTTPGet) and T2 (dig@authoritative DNS), both of which were successful. Based on the task dimension mapping table, T1 is associated with D11, and T2 is associated with D12.

[0080] Covered dimensions: {D11, D12} .

[0081] Validation coverage calculation: If the system's preset verification sufficiency threshold is 0.8 (this threshold can be flexibly configured according to business scenarios), then 0.5 < 0.8, indicating that the current diagnostic verification is insufficient, and prompting that additional verification tasks for the uncovered dimensions, such as T3 (dig@recursive DNS) and T4 (Traceroute), need to be executed to improve coverage.

[0082] Example 3: The following is a deduplication scenario where multiple tasks cover the same dimension: Template: A fault template contains dimension D01 (link connectivity), which can be verified by two tasks: Traceroute and bidirectional Ping.

[0083] Task execution: During the diagnosis, Traceroute and Ping were executed successfully.

[0084] Deduplication rule: Dimension D01 is marked as covered the first time it is covered, and it is not counted again the second time. Therefore It remains at 1, which does not affect the accuracy of the coverage calculation.

[0085] Example 4: Extreme boundary scenario, i.e. Scene: Template: A certain fault template includes dimension D04 (interface status), and the applicable condition is "the device supports SNMP".

[0086] Device context: SNMP is not supported in the current device.

[0087] Instantiated clipping: D04 has been removed; verification should be performed to confirm the dimension list is empty. .

[0088] Verification coverage calculation: According to business rules, if no verification method is available due to device capability limitations, the coverage is recorded as 1.0.

[0089] Example 5: Multiple Fault Type Overlap Scenario: Fault Event: Simultaneous link failure and source station failure. First, trim the link failure template and source station failure template separately: the trimmed link failure template becomes {D01, D02, D03}, and the trimmed source station failure template becomes {D11, D12, D13, D14}. The merged list contains {D01, D02, D03, D11, D12, D13, D14}. Dimensions already covered. Then the overall coverage rate .

[0090] The default parameters (such as template dimension, applicable conditions, preset threshold of 0.8, weight coefficient, etc.) involved in the embodiments of this application are exemplary configurations and are not intended to limit this solution. In practical applications, they can be flexibly adjusted according to network scale and device type.

[0091] Please refer to Figure 7 This application also provides a verification coverage analysis system for fault diagnosis, comprising: The data management and analysis module is used to determine the corresponding initial verification dimension list based on the fault type indicated by the fault event, and to determine the device context based on the device involved in the fault event; The trimming module is used to trim the initial verification dimension list according to the device context to obtain the target verification dimension list. The mapping management module is used to obtain the test tasks that have been successfully executed in the diagnostic process for fault events, and then query the task dimension mapping table based on the test tasks to determine the verification dimensions that have been covered. The validation coverage analysis module is used to perform statistical analysis on the validation dimensions that have been covered in the target validation dimension list to determine the validation coverage rate.

[0092] In one embodiment, the data management and analysis module is divided into a template management module and a context parsing module. The modules of this system are described below: Template Management Module: Stores standard verification dimension templates for each fault type, supporting dynamic updates. Templates can be built based on expert experience or historical data mining and stored in a template library.

[0093] Context resolution module: When a fault occurs, it obtains the device context involved in the fault event, including device role (core / edge), supported management protocols (SNMP / NetConf / CLI, etc.), network topology location (backbone / aggregation / access layer), etc.

[0094] The trimming module trims the template based on the device context, removes inapplicable dimensions, generates a list of dimensions to be verified in this diagnostic, and records the total number of dimensions after instantiation. .

[0095] The mapping management module maintains a task dimension mapping table, recording the verifiable dimensions for each test task. The mapping table uses a many-to-many relationship design, meaning a task can be associated with multiple dimensions, and a dimension can be covered by multiple tasks. The mapping table is predefined during system initialization and supports dynamic expansion, but expansion rules only apply to newly added fault events and do not affect ongoing diagnostic processes. During the diagnostic process, for successfully executed test tasks (excluding tasks incomplete due to system-level anomalies such as probe failure or network timeout), the module queries the table for the associated covered verification dimensions and further deduplicates the covered dimensions found for each test task.

[0096] Validation coverage analysis module: Determines the currently covered validation dimensions based on the query mapping table and counts the number of covered dimensions. Calculate the validation coverage: , .like If so, it will be set to 1.0 or marked as "not applicable" according to the business rules.

[0097] It is understood that the methods described in the above method embodiments are applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0098] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including a tablet computer.

[0099] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0100] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0101] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0102] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0103] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0104] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0105] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0106] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0107] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0108] The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0110] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0111] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0112] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0113] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0115] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A verification coverage analysis method for fault diagnosis, characterized in that, Includes the following steps: The corresponding initial verification dimension list is determined based on the fault type indicated by the fault event, and the device context is determined based on the device involved in the fault event; Based on the device context, the initial verification dimension list is pruned to obtain the target verification dimension list; Obtain the test tasks that have been successfully executed in the diagnostic process for the fault event, and then query the task dimension mapping table based on the test tasks to determine the verification dimensions that have been covered. Perform statistical analysis on the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate.

2. The verification coverage analysis method for fault diagnosis according to claim 1, characterized in that, Determining the corresponding initial verification dimension list based on the fault type indicated by the fault event includes the following steps: When a fault event indicates a fault type, the corresponding standard verification dimension template is queried according to the fault type, and the standard verification dimension template is used as the initial verification dimension list. When a fault event indicates multiple fault types, the corresponding standard verification dimension template is queried according to each fault type, and the multiple standard verification dimension templates are merged to obtain an initial verification dimension list.

3. The verification coverage analysis method for fault diagnosis according to claim 1, characterized in that, The step of pruning the initial verification dimension list according to the device context to obtain the target verification dimension list includes the following steps: Based on the device role, supported protocols, and network topology location indicated by the device context, the initial verification dimension list is pruned to obtain an intermediate verification dimension list. Based on the applicable conditions indicated in the intermediate verification dimension list, verify whether the device context-related verification data meets the applicable conditions. If not, delete the verification dimension corresponding to the applicable conditions from the intermediate verification dimension list to obtain the target verification dimension list.

4. The verification coverage analysis method for fault diagnosis according to claim 1, characterized in that, The step of querying the task dimension mapping table based on the probing task to determine the covered verification dimensions includes the following steps: For each test task that has been successfully executed, the task dimension mapping table is queried to determine the verification dimension associated with the test task, and the query result of the test task is obtained. Perform dimension deduplication on the query results of each successfully executed test task to determine the verified dimensions that have been covered.

5. The verification coverage analysis method for fault diagnosis according to claim 1, characterized in that, The statistical analysis of the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate includes the following steps: Perform statistical analysis on the verification dimensions already covered in the target verification dimension list to determine the number of dimensions already covered; The base coverage rate, used to measure the verification coverage rate, is calculated based on the number of covered dimensions and the total number of dimensions in the target verification dimension list.

6. The verification coverage analysis method for fault diagnosis according to claim 1, characterized in that, The statistical analysis of the verification dimensions already covered in the target verification dimension list to determine the verification coverage rate includes the following steps: Obtain the weight values ​​of all verification dimensions in the target verification dimension list; The weight values ​​of the verification dimensions already covered in the target verification dimension list are accumulated to obtain the verification dimension score; and the weight values ​​of all verification dimensions in the target verification dimension list are accumulated to obtain the verification dimension total score. Based on the verification dimension scores and the total verification dimension scores, a weighted coverage rate is calculated to measure verification coverage.

7. The verification coverage analysis method for fault diagnosis according to claim 5, characterized in that, The verification coverage analysis method for fault diagnosis also includes the following steps: If the verification coverage is lower than a preset threshold, a corresponding probing task is matched based on the verification dimensions not covered in the target verification dimension list. The matched test task is pushed to the upstream diagnostic decision engine for task execution decision, so as to update the first verification coverage.

8. A verification coverage analysis system for fault diagnosis, characterized in that, include: The data management and analysis module is used to determine the corresponding initial verification dimension list based on the fault type indicated by the fault event, and to determine the device context based on the device involved in the fault event. The trimming module is used to trim the initial verification dimension list according to the device context to obtain the target verification dimension list. The mapping management module is used to obtain the test tasks that have been successfully executed in the diagnostic process for the fault event, and then query the task dimension mapping table according to the test tasks to determine the verification dimensions that have been covered. The verification coverage analysis module is used to perform statistical analysis on the verification dimensions that have been covered in the target verification dimension list to determine the verification coverage rate.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images