Fault processing method, apparatus, device, medium, and product

Abstract

Embodiments of the present application provide a fault processing method, device, equipment, medium and product, and relate to the field of computer software fault detection and automatic repair. The method comprises: collecting network state data of a target service through a kernel probe and identifying abnormal network data from the network state data, a host node where the target service is located comprising the kernel probe; matching a network firewall policy based on the abnormal network data to determine a network fault matched with the abnormal network data, the network firewall policy being changed within a preset time period; and performing a repair operation associated with the network fault according to the network fault. The method is used to improve the timeliness and accuracy of fault perception and root cause positioning, and takes into account the efficiency of fault disposal, so as to ensure high availability and low latency requirements.

Description

Technical Field

[0001] This application relates to the field of computer software fault detection and automatic repair, and in particular to a fault handling method, apparatus, equipment, medium and product. Background Technology

[0002] In cloud-native microservice systems, inter-service communication typically relies on dynamically configured network firewall policies, combined with logs, network detection, and application monitoring for operational assurance.

[0003] However, during application deployment, network firewall policies often block application-layer transactions. Once an application-layer transaction fails, the process typically relies on passive manual troubleshooting and handling of network faults, resulting in long recovery times and impacting business continuity and stability.

[0004] Therefore, how to improve the timeliness and accuracy of fault detection and root cause localization, while taking into account the efficiency of fault handling, in order to ensure high availability and low latency requirements, has become a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a fault handling method, apparatus, device, medium, and product to improve the timeliness and accuracy of fault detection and root cause localization, while also taking into account the efficiency of fault handling, so as to ensure high availability and low latency requirements.

[0006] In a first aspect, embodiments of this application provide a fault handling method, comprising: collecting network status data of a target service through a kernel probe and identifying abnormal network data from the network status data, wherein the host node where the target service is located includes the kernel probe; matching network firewall policies based on the abnormal network data to determine network faults that match the abnormal network data, wherein the network firewall policies have been changed within a preset time period; and performing repair operations associated with the network faults according to the network faults.

[0007] In one possible embodiment, the network status data includes a network request return status code, which indicates the status returned by the target service after sending the network request; identifying abnormal network data from the network status data includes: identifying abnormal network request return status codes from the network status data, whereby the abnormal network request return status code indicates that the network request was not received, or that the network request was received and an error was reported.

[0008] In one possible embodiment, the abnormal network data also includes at least one of the following associated with the abnormal network request return status code: the path, method, and header data of the network request; the path, method, and header data of the network request are detected by the observation platform of the target service.

[0009] In one possible embodiment, the header data of the network request includes an IP address and a port number, and the network firewall policy includes allow policies and security rules. Determining network faults by matching abnormal network data with preset network policies includes: if the IP address and port of the abnormal network data do not match any allow policy, determining the network fault as a missing allow policy; if the abnormal network data matches multiple allow policies simultaneously and the security rules indicate a conflict between the multiple allow policies, determining the network fault as a conflict between network firewall policies.

[0010] In one possible embodiment, performing network fault-related repair operations based on network faults includes performing repair operations based on the network fault, the level of the target service, and the frequency of occurrence of the network fault.

[0011] In one possible embodiment, the repair operation, based on the network failure, the level of the target service, and the frequency of the network failure, includes: if the target service is a core service and the frequency of the network failure exceeds a preset threshold, performing an automated repair operation, which includes allowing the network to proceed or rolling back the network. The rollback operation is used to roll back the target network firewall policy that caused the network failure to the version before the change. If the target service is a non-core service, or if the frequency of the network failure is less than a preset threshold, performing an alarm operation, which is used to alert relevant information about the network failure.

[0012] In one possible embodiment, if the network failure is due to a missing allowance policy, the repair operation includes an allowance operation; if the network failure is due to a conflict in network firewall policies and the benefit-risk ratio of a rollback operation is greater than a preset threshold, the repair operation includes a rollback operation; if the network failure is due to a conflict in network firewall policies and the benefit-risk ratio of a rollback operation is greater than a preset threshold, the repair operation includes a temporary allowance operation.

[0013] In one possible implementation, the benefit-risk ratio of a rollback operation is determined based on at least one of the following: the scope of impact on the target service, the feasibility of the rollback operation, and whether rolling back the target network firewall policy would affect services that have already been allowed.

[0014] Secondly, embodiments of this application provide a fault handling apparatus, comprising: an identification module, configured to collect network status data of a target service through a kernel probe and identify abnormal network data from the network status data, wherein the host node where the target service is located includes a kernel probe; a matching module, configured to match network firewall policies based on the abnormal network data to determine network faults that match the abnormal network data, wherein the network firewall policies have been changed within a preset time period; and a repair module, configured to perform repair operations associated with the network faults based on the network faults.

[0015] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the first aspect and / or various possible embodiments of the first aspect as described above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible embodiments of the first aspect.

[0017] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible embodiments of the first aspect as described above.

[0018] The fault handling method, device, medium, and product provided in this application collect network status data of the target service through a kernel probe. This allows for the automatic collection of kernel-level network status data and accurate capture of events related to the network communication process of the target service. By identifying abnormal network data from the network status data, the massive amount of network status data collected from the kernel probe can be preliminarily cleaned and anomalies identified. By matching abnormal network data with network firewall policies, network faults matching the abnormal network data can be determined, and recently changed network firewall policies can be combined to perform fault judgment and convert abnormal network data into understandable root causes. By performing network fault-related repair operations based on the network fault, a closed-loop fault handling system can be achieved.

[0019] Therefore, the embodiments of this application enable network anomaly detection, fault location, and fault handling to be completed within the same processing link. Furthermore, during application deployment, the target service and the network firewall policy based on a whitelist mechanism can achieve automatic linkage verification, preventing the network firewall policy from blocking application layer transactions. Moreover, the embodiments of this application can automatically perform fault repair, eliminating the tedious process of manually troubleshooting network faults, thereby reducing fault handling latency and ensuring consistency and accuracy in fault handling, ensuring business continuity and stability, and ensuring high availability and low latency requirements. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a system architecture diagram of the fault handling method according to an embodiment of this application;

[0022] Figure 2This is a flowchart of a fault handling method according to an embodiment of this application;

[0023] Figure 3 A schematic diagram of the fault handling device provided in this application;

[0024] Figure 4 A schematic diagram of the structure of the electronic device provided in this application.

[0025] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0028] It should be noted that the fault handling methods, devices, equipment, media and products provided in this application can be applied to the field of financial technology, or to any field other than financial technology. This application does not limit the application field of the fault handling methods, devices, equipment, media and products.

[0029] Fault handling technologies for cloud-native microservice systems are typically applied to business environments built on containerized platforms, particularly suitable for scenarios involving dynamic scaling of service instances, high-frequency inter-service calls, and continuous adjustments to network access rules. In these scenarios, the target service is usually deployed on a host node, service communication relies on network firewall policies for access control, and the related operations and maintenance system also needs to incorporate network status detection to ensure business continuity.

[0030] When an application is released for production, it involves service deployment, scaling up and down, and new network connections. If the network firewall policy, based on a whitelist mechanism, is out of sync with these changes, or if the network firewall is frequently adjusted, service communication connections may be blocked. The reason for this blocking is a non-business logic factor related to the network firewall policy, which usually cannot be automatically corrected and requires manual judgment and intervention.

[0031] This approach not only increases labor costs, but also takes a long time from fault detection and manual intervention to completing network firewall policy rollback or traffic allowance, resulting in prolonged business interruption, reduced business throughput, and failure to meet high availability and low latency requirements.

[0032] Furthermore, fault handling typically relies on manual repair actions, which can easily lead to secondary impacts due to inaccurate judgment or improper operation. In core business processes, persistent faults can cause service interruptions, affect other services, and degrade business continuity.

[0033] Therefore, how to improve the timeliness and accuracy of fault detection and root cause localization, while taking into account the efficiency of fault handling, in order to ensure high availability and low latency requirements, has become a technical problem that needs to be solved.

[0034] In view of this, embodiments of this application provide a fault handling method.

[0035] Figure 1 This is a system architecture diagram of the fault handling method according to an embodiment of this application. Figure 1 As shown, the system architecture can be applied to host node 11, backend server 12, and operation and maintenance node 13.

[0036] The host node 11 is an electronic device that runs the target service. The host node 11 and the backend server 12 communicate and interact to realize the target service. Here, the backend server 12 is the server that provides the target service.

[0037] The maintenance node 13 is an electronic device that executes the embodiments of this application.

[0038] For example, an electronic device can be a terminal device or a server. A server can be a physical server or a virtual cloud server.

[0039] Combination Figure 1 The system architecture, Figure 2 This is a flowchart illustrating a fault handling method according to an embodiment of this application. In this embodiment, the fault handling method is executed by an operation and maintenance node, which will be described below as an electronic device.

[0040] like Figure 2 As shown, the fault handling method includes steps S201 to S203.

[0041] S201. Collect network status data of the target service through kernel probes and identify abnormal network data from the network status data.

[0042] The host node where the target service resides includes a kernel probe. For example, the kernel probe may be an extended Berkeley Packet Filter (eBPF). In this embodiment, the kernel probe is deployed within the host node where the target service resides, and is associated with the kernel network stack of the host node to detect events related to the network communication process. The network state data is a set of data characterizing the communication state of the target service, and may include one or more of the following: communication endpoint information, protocol type, event time, and request result information.

[0043] In one possible embodiment, the electronic device first establishes an association between the target service and its corresponding running entity on the host node. After the kernel probe starts, it filters network events entering the collection link according to this association, retaining only those related to the target service. For each retained network event, the electronic device extracts the corresponding communication identifier, event timestamp, protocol status, and result information, and associates multiple events within the same request session to form a network status data record for a single network request. Of course, in addition to this, the electronic device can also combine other data sources to supplement the contextual information of abnormal network data.

[0044] In one possible embodiment, the electronic device first performs rule-based judgment on the collected network status data. Rule-based judgment can be based on at least one abnormal characteristic in the network status data, such as connection failure, timeout, no response, or abnormal return results. When an anomaly is detected in network communication, the electronic device identifies the corresponding record as abnormal network data.

[0045] Step S201: By directly collecting and identifying abnormal network data in the host node kernel, abnormal network data can be obtained without modifying the target service business code, so that subsequent fault judgment is based on actual communication events.

[0046] S202. Match network firewall policies based on abnormal network data to identify network faults that match the abnormal network data.

[0047] The network firewall policy has been changed within a preset time period.

[0048] In this embodiment, the network firewall policy is a set of rules that control network access relationships between services. The preset time period is a time window established around the time an anomaly occurs, and can be specifically set according to the frequency of operation and maintenance changes. "Changes have occurred" means that the network firewall policy has been added, deleted, disabled, enabled, or modified within the preset time period.

[0049] In one possible embodiment, the electronic device can first retrieve records of network firewall policy versions that have changed within a preset time period from the network firewall policy configuration center, network controller, or firewall management platform, and construct a change policy index. The index entries include at least the policy identifier, the content before the change, the content after the change, the change time, the change type, the scope of effect, and the associated objects. Next, the electronic device extracts communication-related feature information from the abnormal network data and matches this information with the change policy index to identify network faults matching the abnormal network data.

[0050] In one possible embodiment, the electronic device can also perform supplementary verification of the correlation between abnormal network data and changes in network firewall policies to improve the accuracy of fault location.

[0051] Step S202: By associating and matching abnormal network data with network firewall policies that have changed within a preset time period, a correspondence can be established between recent network firewall policy adjustments and real-time communication anomalies, thereby improving fault location efficiency.

[0052] S203. Based on the network fault, perform the repair operation associated with the network fault.

[0053] In this embodiment, the repair operation is a handling action triggered in response to a confirmed network fault.

[0054] In one possible embodiment, upon determining a network failure, and if the network failure is associated with a specific failure type, the electronic device performs corresponding repair procedures based on the failure type. After repair, the network status data of the target service can be re-observed to verify the repair results. Once the repair operation is complete, the electronic device can write the repair operation, repair content, and post-repair verification results into an audit log.

[0055] Step S203: Trigger corresponding repair operations based on the identified network faults to form an automated fault handling closed loop.

[0056] In this embodiment, network status data of the target service is collected via a kernel probe. This automatically collects kernel-level network status data and accurately captures events related to the network communication process of the target service. By identifying abnormal network data from the network status data, the massive amount of network status data collected from the kernel probe can be preliminarily cleaned and anomalies identified. By matching abnormal network data with network firewall policies, network faults matching the abnormal network data can be determined, and recently changed network firewall policies can be combined to determine the fault and convert the abnormal network data into an understandable root cause. By performing network fault-related repair operations based on the network fault, a closed-loop fault handling system can be achieved.

[0057] Therefore, the embodiments of this application enable network anomaly detection, fault location, and fault handling to be completed within the same processing link. Furthermore, during application deployment, the target service and the network firewall policy based on a whitelist mechanism can achieve automatic linkage verification, preventing the network firewall policy from blocking application layer transactions. Moreover, the embodiments of this application can automatically perform fault repair, eliminating the tedious process of manually troubleshooting network faults, thereby reducing fault handling latency and ensuring consistency and accuracy in fault handling, ensuring business continuity and stability, and ensuring high availability and low latency requirements.

[0058] In one possible embodiment, network status data includes a network request return status code. The network request return status code indicates the status returned by the target service after sending the network request.

[0059] Step S201, which involves identifying abnormal network data from network status data, includes step S2011.

[0060] S2011. Identify abnormal network request return status codes from network status data.

[0061] An abnormal network request returns a status code indicating that the network request was not received, or that the network request was received but an error occurred.

[0062] For example, in a network request based on the Hypertext Transfer Protocol (HTTP), the network request return status code includes `resp_status`. An abnormal network request return status code indicating that the network request was not received is `resp_status == 0`. More specifically, `resp_status == 0` indicates a situation where a network request was sent but no HTTP response was received, such as connection timeout, Transmission Control Protocol (TCP) resolution failure, TCP reset packet resolution failure, or domain name resolution failure. An abnormal network request return status code indicating that the network request was received but an error occurred is `resp_status >= 500`.

[0063] In this embodiment, the electronic device first categorizes the network request return status codes in the collected network status data and filters out abnormal network request return status codes according to preset judgment rules. Abnormal network request return status codes can be stored separately from normal return status codes and used as input data for subsequent fault matching.

[0064] In this embodiment, through the above operations, the electronic device can directly map the network request status of the target service to an anomaly marker, and uniformly include both no response received and request errors into the scope of abnormal network data. This allows subsequent network fault location to be based on clear network request return status code results. In this way, the identification of abnormal network data does not rely on manual analysis of raw logs one by one, but directly extracts and identifies abnormal network request return status codes from the network request return status code level, and uses them as abnormal network data for subsequent fault location.

[0065] In one possible embodiment, the abnormal network data also includes at least one of the following associated with the abnormal network request return status code: the path, method, and header data of the network request.

[0066] The path, method, and header data of the network request are obtained through the observation platform of the target service.

[0067] For example, the network request path may include "api" or "login," and the network request method may include "post." The network request header data may include the name of the target service, transaction identifier, etc. The target service's observation platform is responsible for parsing the network request's path, method, header data, and other Layer 7 protocol context information. The raw network state data collected by the kernel probe is correlated with the context information parsed by the observation platform to form a complete record of abnormal network data. This combines the kernel's low-latency monitoring capabilities with the observation platform's Layer 7 protocol parsing capabilities to accurately pinpoint the root cause of abnormal network data, thereby improving the real-time performance and accuracy of fault detection.

[0068] In one possible embodiment, when an electronic device detects an abnormal network request return status code, it can bind and save the abnormal network request return status code along with the path, method, and header data associated with the same network request, forming abnormal network data containing the network request return status code and its context. The path, method, and header data can be individual components of the abnormal network data, or they can be combined according to a preset field structure to establish a correspondence with the network request return status code. This allows for the subsequent matching of network firewall policies based on the abnormal network data, enabling the use of the request interface, request action, and header content to jointly locate potentially affected access rules. The fields collected by the observation platform typically originate from HTTP message parsing results, and can be obtained through bypass mirroring, application instrumentation, or proxy collection methods.

[0069] In this embodiment, through the above operations, the electronic device first uses a kernel probe to identify the abnormal network request return status code, and then the observation platform of the target service extracts the path, method, and header data corresponding to the abnormal network request return status code, and summarizes this information into abnormal network data. Since the abnormal network data includes the request-response result and its protocol context, this enhances the matching capability between abnormal network requests and network firewall policies, and makes fault determination closer to the actual access semantics. Therefore, when comparing with recently changed network firewall policies, it can more accurately point to specific restricted access points, calling methods, and packet characteristics, thereby improving the accuracy of network fault location and thus enhancing the accuracy of anomaly identification and subsequent repair.

[0070] In one possible implementation, the header data of the network request includes an IP (Internet Protocol) address and port number. Network firewall policies include allow / allow policies and security rules.

[0071] Step S202, which involves matching abnormal network data with a preset network strategy to determine network faults that match the abnormal network data, includes steps S2021 to S2022.

[0072] S2021. If the IP address and port number of the abnormal network data do not match any allow policy, the network failure is determined to be due to a missing allow policy.

[0073] S2022. When abnormal network data matches multiple allow policies and security rules indicate that multiple allow policies conflict, the network failure is determined to be a network firewall policy conflict.

[0074] In one possible embodiment, the electronic device can extract the source IP, destination IP, and corresponding port number from abnormal network data, and compare the extraction results with the address range and port rules in the allowance policy. If the comparison result shows that the IP address and port number combination does not fall within the coverage of any allowance policy that permits communication, the corresponding fault type is determined to be a missing allowance policy. If the same abnormal network data hits multiple allowance policies simultaneously, the security rules are further used to check whether these policies correspond to different allowance conditions, conflicting port numbers, or mutually exclusive address ranges. When the security rules output a conflict indication, the corresponding fault type is determined to be a network firewall policy conflict. To adapt to different cloud-native microservice deployment methods, the IP address and port number can be the source address, destination address, source port, or destination port of the target service, respectively. The electronic device can select the corresponding field to participate in the matching according to the communication direction of the abnormal network data.

[0075] In this embodiment, by precisely associating the IP address and port number of abnormal network data with allowance policies and combining this with security rules to identify multi-policy conflicts, network faults of not hitting allowance policies and policy conflicts can be distinguished, making the determination of the root cause of the fault more targeted. This provides clear fault category input for subsequent repair operations and reduces misjudgments of the cause of abnormal communication.

[0076] In one possible embodiment, step S203, which involves performing a network fault-related repair operation based on the network fault, includes step S2031.

[0077] S2031. Perform repair operations based on the network failure, the level of the target service, and the frequency of network failure occurrence.

[0078] In one possible embodiment, after a network fault is identified, the electronic device first reads the level information of the target service and then makes a comprehensive judgment based on the frequency of the network fault within the current statistical window. When the network fault corresponds to a core-level target service and the frequency of the network fault reaches a preset threshold, the electronic device can directly trigger automatic repair to restore the corresponding network firewall policy to its state before the fault occurred. When the network fault corresponds to a normal-level target service and the frequency of the network fault is below the threshold, the electronic device can only generate alarm information and retain the fault context for subsequent processing by maintenance personnel. When the frequency of the network fault is in the intermediate range, semi-automatic handling can be performed, that is, first performing repair actions within a limited scope, and then outputting a notification pending confirmation. The execution results of the repair actions can be written back to the fault record for subsequent network fault frequency statistics and fault handling policy updates.

[0079] In one possible embodiment, the level of the target service can also be associated with the service tag in the service catalog so as to quickly obtain the handling priority through the pre-defined mapping relationship; the frequency of network failures can also be achieved by using a sliding time window statistical method so that when the same network failure occurs continuously in a short period of time, it can be identified as a high-frequency failure.

[0080] In this embodiment of the application, by using network failure, target service level and frequency of network failure as the basis for repair, electronic devices can implement differentiated handling for scenarios with different service importance and different degrees of failure repetition, and perform repair operations that match the severity of the failure.

[0081] In one possible embodiment, step S2031, which involves performing a repair operation based on the network failure, the level of the target service, and the frequency of the network failure, includes steps S2032 to S2033.

[0082] S2032. When the target service is a core service and the frequency of network failures exceeds a preset threshold, perform automated repair operations.

[0083] Automated remediation operations include allowing or rolling back. Rollback operations are used to revert the target network firewall policy that caused the network failure to the version before the change.

[0084] S2033. If the target service is a non-core service, or if the frequency of network failures is less than a preset threshold, perform an alarm operation.

[0085] Alarm operations are used to alert for information related to network faults.

[0086] In one possible implementation, when the target service is marked as a core service and the frequency of network failures exceeds a threshold, the electronic device automatically enters the automated repair branch. Here, the allowance operation restores the reachability of the communication endpoint corresponding to the target service by writing or enabling matching allowance rules in the network firewall policy. When there is a clear correspondence between the failure and the most recent network firewall policy change, the electronic device rolls back the target network firewall policy to the version before the change and reissues it to the firewall execution plane. The electronic device can also save the historical version number, change timestamp, and effective scope of the target network firewall policy, facilitating accurate location of recoverable versions during rollback. However, when the target service is a non-core service, or the frequency of network failures does not reach the preset threshold, the electronic device may not perform automatic recovery. Instead, it generates an alarm message and pushes it to the operation and maintenance monitoring platform. The alarm message may carry a fault identifier, abnormal network data, and associated network firewall policy records for further confirmation by operation and maintenance personnel.

[0087] In this embodiment, automated repair operations (such as allowing or rolling back) are applicable to core services and scenarios where the frequency of failures exceeds a preset threshold, requiring no manual intervention. For non-core services or low-frequency failure scenarios, only alarm information is generated and pushed to the operation and maintenance monitoring platform for further manual confirmation.

[0088] In this embodiment, the above operations distinguish between the triggering conditions for automated repair and manual approval: by combining service level with fault frequency, repair operations are dynamically adjusted to ensure priority handling of core service faults. This allows high-priority services to directly restore communication in high-frequency fault scenarios, while low-priority services or low-frequency fault scenarios retain the space for manual judgment, thus ensuring that repair operations are consistent with the importance of the service and the severity of the fault.

[0089] In one possible embodiment, if the network failure is due to a missing allowance policy, the remediation operation includes allowing access. If the network failure is due to a conflict in network firewall policies and the benefit-risk ratio of a rollback operation is greater than a preset threshold, the remediation operation includes a rollback operation. If the network failure is due to a conflict in network firewall policies and the benefit-risk ratio of a rollback operation is greater than a preset threshold, the remediation operation includes a temporary allowance operation.

[0090] In this embodiment, when a network failure is determined to be due to a missing allowance policy based on abnormal network data, the electronic device can set the corresponding repair operation as an allowance operation and write an allowance policy matching the abnormal network data into the network firewall policy library. The allowance policy at least includes the IP address, port number, and communication direction of the target service. This allowance policy can be written into the kernel mode or control plane policy table using a configuration format at the same level as existing network firewall policies, enabling the target service to restore connectivity in scenarios where an allowance policy is missing.

[0091] When a network failure is caused by a conflict in network firewall policies, electronic devices can first calculate the risk-benefit ratio of a rollback operation and then compare it with a preset threshold. If the risk-benefit ratio is greater than the preset threshold, a rollback operation is performed, reverting the target network firewall policy that caused the conflict to its previous version. The rollback operation can be completed by reading historical snapshots in the network firewall policy version repository, locating the most recent change record, and replacing the currently effective configuration.

[0092] When a network failure is caused by a conflict in network firewall policies and an immediate rollback is not appropriate, electronic devices can set the repair operation as a temporary allowance operation. A temporary allowance operation involves writing a temporary allowance policy within a limited time window, which automatically expires after the timeout. This temporary allowance operation can be set with independent expiration and revocation flags so that it can be automatically cleared after subsequent policy verification.

[0093] Through the above operations, this application embodiment can directly supplement the allowance policy when it is missing, restore the previous version when the network firewall policy conflicts and the risk assessment meets the conditions, and maintain communication with temporary allowance when rollback is not appropriate. This makes the repair action consistent with the fault type and risk assessment results, improves the accuracy of fault handling and the controllability of policy recovery, and shortens the service interruption time caused by abnormal network firewall policies.

[0094] In one possible implementation, the benefit-risk ratio of a rollback operation is determined based on at least one of the following: the scope of impact on the target service, the feasibility of the rollback operation, and whether rolling back the target network firewall policy would affect services that have already been allowed.

[0095] In one possible embodiment, after identifying a network firewall policy conflict, the electronic device can obtain the target service's call domain, the number of access objects, and the importance of related services, and generate an impact range score accordingly. Next, the electronic device can read the target network firewall policy version link, rollback baseline, and currently effective rule table to determine whether the network firewall policy file, rollback image, or configuration snapshot exists completely, thus forming a feasibility score. Then, the electronic device cross-compares the target network firewall policy to be rolled back with the IP address, port number, path, or method of the already allowed service. If a situation exists that would change the already allowed mapping relationship, the risk score is increased. The electronic device can assign values ​​to at least one of the above factors and then perform a weighted synthesis, or map them to a ratio of benefit score to risk score, outputting the benefit-risk ratio of the rollback operation, and using this ratio as the basis for determining whether to perform a subsequent rollback operation. For example, if the target service has a high impact range and rollback feasibility is high, the benefit-risk ratio weight is tilted towards benefit; if the rollback operation may affect the already allowed service, the risk score weight is increased. That is, by dynamically adjusting the weights of each factor, a reasonable risk-reward ratio can be generated, thereby determining whether to perform a rollback operation.

[0096] In this embodiment of the application, the above operations enable the rollback decision to no longer rely solely on a single conflict phenomenon, but to comprehensively evaluate the impact of the target service, the rollback implementation conditions, and the disturbance to the existing allowed services, so that the selection result of the rollback operation is closer to the actual handling needs in the current network firewall policy change scenario.

[0097] Table 1 below illustrates the repair operations for different scenarios.

[0098] Table 1

[0099]

[0100] In one possible embodiment, steps S2021 to S2022 can be performed by a diagnostic model. The risk-benefit ratio of the rollback operation can be determined by a change impact model, based on the scope of impact of the target service, the feasibility of the rollback operation, and whether rolling back the target network firewall policy affects at least one of the already permitted services. Both the diagnostic model and the change impact model can be deployed on electronic devices, specifically expert models or deep learning models, without limitation.

[0101] In summary, the fault handling method of this application embodiment has at least one of the following technical effects:

[0102] Improved real-time performance: Kernel probes directly collect network status data in kernel mode, avoiding the intrusiveness and performance loss of traditional data collection methods, and ensuring real-time fault detection.

[0103] Precise root cause localization: Based on the ability to parse the context of network requests, fault diagnosis can be deepened from the "network layer" to the "application layer", accurately identifying missing or conflicting network firewall policies.

[0104] Refined repair operations: Tiered repair operations are performed based on the level of the target service and the risk-benefit ratio of rollback, avoiding high-risk repair operations while ensuring priority repair of core service failures.

[0105] Improved operational efficiency: Automated repair of non-operations (such as release and rollback) replaces manual operations, reducing response time and labor costs. At the same time, alarm mechanisms (such as silent processing in the test environment) avoid interfering with the non-production environment.

[0106] Enhanced risk controllability: By assessing the feasibility of rollback operations and the impact scope of target services, the scientific nature of repair decisions can be ensured, reducing the risk of secondary failures.

[0107] Figure 3 A schematic diagram of the fault handling device provided in this application is shown below. Figure 3As shown, the fault handling device provided in this embodiment includes: an identification module 310, a matching module 320, and a repair module 330.

[0108] The identification module 310 is used to collect network status data of the target service through the kernel probe and identify abnormal network data from the network status data. The host node where the target service is located includes the kernel probe.

[0109] Matching module 320 is used to match network firewall policies based on abnormal network data in order to identify network faults that match the abnormal network data, and the network firewall policies have been changed within a preset time period.

[0110] Repair module 330 is used to perform repair operations associated with network faults based on network faults.

[0111] In one possible embodiment, the network status data includes a network request return status code, which indicates the status returned by the target service after sending a network request; the identification module is specifically used to: identify abnormal network request return status codes from the network status data, whereby an abnormal network request return status code indicates that a network request was not received, or that a network request was received and an error was reported.

[0112] In one possible embodiment, the abnormal network data also includes at least one of the following associated with the abnormal network request return status code: the path, method, and header data of the network request; the path, method, and header data of the network request are detected by the observation platform of the target service.

[0113] In one possible embodiment, the header data of the network request includes an IP address and a port number, and the network firewall policy includes allow policies and security rules; the matching module is specifically used to: determine that the network failure is due to a missing allow policy when the IP address and port of the abnormal network data do not match any allow policy; and determine that the network failure is due to a conflict in the network firewall policy when the abnormal network data matches multiple allow policies and the security rules indicate that the multiple allow policies conflict.

[0114] In one possible embodiment, the repair module includes a repair submodule for performing repair operations based on the network failure, the level of the target service, and the frequency of the network failure.

[0115] In one possible embodiment, the repair submodule is specifically used to: perform automated repair operations when the target service is a core service and the frequency of network failures exceeds a preset threshold; the automated repair operations include allowing operations or rolling back operations, where the rollback operation is used to roll back the target network firewall policy that caused the network failure to the version before the change; and perform alarm operations when the target service is a non-core service, or when the frequency of network failures is less than a preset threshold; the alarm operations are used to alert relevant information about the network failure.

[0116] In one possible embodiment, if the network failure is due to a missing allowance policy, the repair operation includes an allowance operation; if the network failure is due to a conflict in network firewall policies and the benefit-risk ratio of a rollback operation is greater than a preset threshold, the repair operation includes a rollback operation; if the network failure is due to a conflict in network firewall policies and the benefit-risk ratio of a rollback operation is greater than a preset threshold, the repair operation includes a temporary allowance operation.

[0117] In one possible implementation, the benefit-risk ratio of a rollback operation is determined based on at least one of the following: the scope of impact on the target service, the feasibility of the rollback operation, and whether rolling back the target network firewall policy would affect services that have already been allowed.

[0118] The fault handling device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0119] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the electronic device further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0120] In the specific implementation process, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-mentioned fault handling method.

[0121] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0122] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0123] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0124] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0125] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described fault handling method.

[0126] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned fault handling method.

[0127] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0128] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0129] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0134] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A fault handling method, characterized in that, include: The kernel probe collects network status data of the target service and identifies abnormal network data from the network status data. The host node where the target service is located includes the kernel probe. Based on the abnormal network data, a network firewall policy is matched to determine the network fault that matches the abnormal network data. The network firewall policy has been changed within a preset time period. Based on the network fault, perform the repair operation associated with the network fault.

2. The method according to claim 1, characterized in that, The network status data includes a network request return status code, which indicates the status returned by the target service after sending the network request. The step of identifying abnormal network data from the network status data includes: Identify abnormal network request return status codes from the network status data. The abnormal network request return status codes indicate that the network request was not received, or that the network request was received and an error was reported.

3. The method according to claim 2, characterized in that, The abnormal network data also includes at least one of the following associated with the abnormal network request return status code: the path, method, and header data of the network request; the path, method, and header data of the network request are detected by the observation platform of the target service.

4. The method according to claim 2 or 3, characterized in that, The header data of the network request includes an IP address and port number; the network firewall policy includes allow policies and security rules; the process of determining network faults matching the abnormal network data by matching preset network policies includes: If the IP address and port of the abnormal network data do not match any of the aforementioned allowance policies, the network failure is determined to be due to a missing allowance policy. If the abnormal network data matches multiple allow policies and the security rule indicates a conflict between the multiple allow policies, the network failure is determined to be a network firewall policy conflict.

5. The method according to claim 4, characterized in that, The step of performing the network fault-related repair operation based on the network fault includes: Based on the network fault, the level of the target service, and the frequency of the network fault, perform a repair operation.

6. The method according to claim 5, characterized in that, The step of performing repair operations based on the network failure, the level of the target service, and the frequency of the network failure includes: If the target service is a core service and the frequency of network failures exceeds a preset threshold, an automated repair operation is performed. The automated repair operation includes a permission operation or a rollback operation. The rollback operation is used to roll back the target network firewall policy that caused the network failure to the version before the change. If the target service is a non-core service, or if the frequency of network failures is less than a preset threshold, an alarm operation is performed; the alarm operation is used to alert relevant information about the network failure.

7. The method according to claim 6, characterized in that, When the network failure is due to the absence of the allowed policy, the repair operation includes the allowed operation; If the network failure is a conflict in network firewall policies and the risk-benefit ratio of the rollback operation is greater than a preset threshold, the repair operation includes the rollback operation. If the network failure is a conflict in network firewall policies and the risk-benefit ratio of the rollback operation is greater than a preset threshold, the repair operation includes a temporary allow operation.

8. The method according to claim 7, characterized in that, The risk-benefit ratio of the rollback operation is determined based on at least one of the following: the scope of impact of the target service, the feasibility of the rollback operation, and whether rolling back the target network firewall policy would affect services that have already been allowed.

9. A fault handling device, characterized in that, include: The identification module is used to collect network status data of the target service through a kernel probe and identify abnormal network data from the network status data, wherein the host node where the target service is located includes the kernel probe; The matching module is used to match network firewall policies based on the abnormal network data in order to determine the network faults that match the abnormal network data, wherein the network firewall policies have been changed within a preset time period. The repair module is used to perform repair operations associated with the network fault based on the network fault.

10. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images