A closed-loop self-healing method and system for cloud cryptographic machine virtualization resources

Abstract

This invention discloses a closed-loop self-healing method and system for cloud cryptographic machine virtualization resources, belonging to the fields of cloud computing and cryptography technology. The method includes: in the entire lifecycle management process of cloud cryptographic machine virtualization resources, collecting and connecting all process operation logs to construct a task log chain; when an anomaly occurs in the management process, accurately locating the specific operational step and underlying cause of the failure based on the task log chain; matching and triggering differentiated intelligent repair strategies corresponding to specific failure reasons, and automatically re-verifying the status after repair, thereby achieving automated closed-loop self-healing of virtualization resource failures. This invention effectively avoids blind troubleshooting and retries, significantly shortening fault response time; when network configuration fails, IPs are reassigned instead of reusing potentially conflicting IP addresses; and the success rate of repairing cryptographic virtualization resources is improved.

Description

Technical Field

[0001] This invention belongs to the fields of cloud computing and cryptography, specifically relating to a management method for virtualized resources of a cloud hardware security module (CHSM), and particularly a closed-loop self-healing method and system for cloud hardware security module virtualized resources. Background Technology

[0002] With the rapid development of cloud computing technology, cryptographic services are gradually evolving from traditional hardware cryptographic devices towards cloudification and virtualization. Cloud cryptographic machines (CHSMs), as dedicated devices providing cryptographic computation services, can use virtualization technology to create multiple virtual security modules (VSMs) on a single physical device, providing independent cryptographic service capabilities for different tenants or businesses. However, existing cloud cryptographic machine virtualization resource management technologies have the following technical problems: Lack of precise fault localization in operation failures: VSM lifecycle management involves multiple steps such as creation, network configuration, data image import, and startup. When a step fails, existing systems struggle to pinpoint the exact problem, leading to difficulties in troubleshooting. For example, in some commercial cloud cryptographic machine management platforms, when VSM creation fails, it typically only returns a generic error code, requiring maintenance personnel to spend considerable time manually analyzing logs to determine the specific cause of the failure.

[0003] Lack of automated repair mechanisms: In traditional solutions, if VSM creation or configuration fails, operations and maintenance personnel need to manually intervene to troubleshoot and repair the problem. This is not only inefficient, but also extremely costly in large-scale deployment scenarios. This contradicts the automated operations and maintenance philosophy pursued by cloud computing.

[0004] The current system employs a simplistic retry mechanism, failing to differentiate between causes of failure and applying the same retry strategy to all failures. This results in a low success rate and may even exacerbate the problem by repeatedly executing erroneous operations. For example, when network configuration fails, a simple retrieval might lead to continued attempts to allocate already occupied IP addresses, resulting in persistent failures.

[0005] State consistency is difficult to guarantee: In a multi-step VSM operation process, if an intermediate step fails, the state changes generated by the executed steps (such as IP address occupancy, resource allocation, etc.) often cannot be automatically cleaned up, leading to resource leaks and state inconsistencies. This may result in subsequent resource allocation conflicts or system resource exhaustion.

[0006] Concurrent Repair Conflicts: In a distributed deployment environment, multiple management nodes may simultaneously repair a failed VSM. The lack of an effective concurrency control mechanism can easily lead to mutual interference between repair operations, resulting in unpredictable outcomes. For example, multiple nodes attempting to repair the network configuration of the same VSM at the same time may cause IP address allocation chaos.

[0007] Therefore, there is an urgent need for a closed-loop self-healing method and system for cloud cryptographic machine virtualization resources that can automatically diagnose the cause of failure, intelligently select repair strategies, and ensure state consistency. Summary of the Invention

[0008] The purpose of this invention is to provide a closed-loop self-healing method and system for cloud cryptographic machine virtualization resources to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a closed-loop self-healing method and system for cloud cryptographic machine virtualization resources, comprising the following steps: S101, Task Log Recording: Completely collect and connect all process operation logs in the full lifecycle management process of cloud cryptographic machine virtualization resources; S102. Fault detection and diagnosis: periodically scan abnormal task logs, locate faulty operation links, and query the cumulative number of repairs for the task. S103. Intelligent repair strategy selection: Match the corresponding repair strategy with the fault link and the cumulative number of repairs, and distinguish between thresholds to determine automatic repair or manual intervention. S104. Automatic retry with a limited number of attempts. Before performing a repair, the cumulative number of repair attempts is checked. If the threshold is not reached, the corresponding repair strategy is executed. The repair count is updated after each repair. S105. Ensure state consistency: Update task resource statistics and bind allocated resources based on repair results; automatically reclaim temporary resources when failure occurs due to exceeding limits. S106. Mark the task as permanently failed or require manual intervention. If the repair exceeds the limit or there is no matching repair strategy, mark the task as permanently failed, retain the log and push the alarm. S107. Distributed concurrency control, relying on distributed locks to isolate multi-node repair tasks, and automatically releasing locks upon timeout to avoid abnormal deadlocks of nodes. S108. Verification and feedback of repair results: Verify the actual operating status of virtualized resources and synchronize the repair results to each business subsystem through the message channel. S109. Log closed-loop update and manual notification of anomalies: If the repair verification is successful, the full-link log is updated to form a closed loop. If the verification fails or the failure is permanent, the fault data is retained and a manual processing notification is issued. S110. Retain failure logs or manual intervention. When the repair verification fails or the task is marked as permanently failed, retain complete fault records and push manual processing notifications to the business and operation and maintenance systems through a unified message channel.

[0010] Preferably, in step S101, during the lifecycle management of the cloud cryptographic machine virtualization resources, the task log module (721) performs detailed log recording for each operation step; this step is related to... Figure 2 Specifically, the steps include: Step S201, obtaining the operation type, which includes one of creating, configuring a network, importing data images, starting, stopping, and destroying; Step S202, obtaining the target resource identifier, used to uniquely identify the virtualized resource being operated on; Step S203, obtaining the execution status, which includes success, failure, and in progress; Step S204, obtaining the timestamp; Step S205, obtaining the operation details, which includes at least one of operation parameters, execution environment, error code, or error message; Step S206, generating a task log; and Step S207, appending the generated task log to the operation trajectory chain according to the operation execution order.

[0011] Preferably, in step S102, the fault detection module (722) periodically scans virtualization resource tasks in an abnormal state, identifies failed task logs, and the diagnostic engine module (723) performs fault diagnosis; this step is consistent with... Figure 3 Steps S301 to S304 correspond to the following steps: Step S301, extract the latest failure log; Step S302, identify the operation type in the latest failure log; Step S303, determine the faulty link based on the operation type; Step S304, query the cumulative number of repairs for the target task.

[0012] Preferably, in step S103, the strategy selection module (724) selects a repair strategy based on the faulty component and the cumulative number of repairs; in step S305, the repair strategy is selected based on the faulty component; when the number of repairs reaches or exceeds the maximum repair threshold, step S306 is executed to mark it as permanently failed; when the number of repairs is less than the maximum repair threshold and a matching strategy exists, step S307 is executed to execute the corresponding repair strategy; when no matching strategy exists, step S308 is executed to adopt a fallback strategy or switch to manual intervention.

[0013] The repair strategy includes: if the failed operation is network configuration, then reassign available IP addresses, subnet masks, and gateways from the IP address pool, call the network configuration interface of the cloud cryptographic machine (710) to reconfigure the virtualization resource network, and record the binding relationship between IP addresses and virtualization resources; if the failed operation is importing data images, then obtain the data image identifier associated with the target task, retrieve the corresponding data image file from the image storage system, and call the image import interface of the cloud cryptographic machine (710) to re-execute the image import; if the failed operation is startup, then call the startup interface of the cloud cryptographic machine (710) to re-execute the virtualization resource startup; if the failed operation is stop, then call the stop interface of the cloud cryptographic machine (710) to re-execute the virtualization resource stop; if the failed operation is creation, then clean up the allocated temporary resources and re-execute the virtualization resource creation.

[0014] Preferably, in step S104, the repair execution module (725) checks the cumulative number of repairs for the current task before each repair operation is performed; if the number of repairs is less than the preset maximum repair threshold, the repair strategy selected in step S103 is executed; if the number of repairs reaches or exceeds the maximum repair threshold, the process proceeds to step S106; after each repair operation is performed, regardless of success or failure, the number of repairs is incremented by 1 and updated in the task record.

[0015] Preferably, in step S105, the system state consistency is ensured during the repair operation through the state management module (726); step S401, determine whether the repair is successful; if the repair is successful, then step S402 is executed sequentially to increment the successful resource count; step S403, decrement the failed resource count; step S404, update the overall task state; step S405, determine whether the repair operation involves resource allocation; if resource allocation is involved, then step S406 is executed to establish the binding relationship between resources and virtualization resources; if resource allocation is not involved, then the state consistency processing ends; if step S401 determines that the repair is unsuccessful, then step S407 is executed to determine whether the maximum number of repairs has been reached; if the maximum number of repairs has been reached, then step S408 is executed to release the occupied temporary resources; if the maximum number of repairs has not been reached, then step S103 is returned to continue selecting and executing the repair strategy.

[0016] Preferably, in step S106, when the cumulative number of repairs reaches or exceeds the maximum repair threshold, or the fault type cannot match the preset repair strategy, the target task is marked as permanently failed by the status management module (726), and the failure log is retained and alarm information is issued by the verification feedback module (728) so that manual intervention can be carried out.

[0017] Preferably, in step S107, in a distributed deployment environment, the concurrent control module (727) interacts with the distributed lock service (730) to prevent multiple management nodes from concurrently repairing the same task; step S501, attempts to acquire the distributed lock, the lock key value is generated based on the task identifier; if the acquisition is successful, step S502 is executed to perform the repair operation, and step S503 is executed after the repair is completed to release the distributed lock; if the acquisition fails, step S504 is executed to skip the current task and wait for the next scheduling cycle; if the repair node is abnormal and does not actively release the lock, step S505 is executed, and the distributed lock automatically expires after the maximum holding time expires.

[0018] Preferably, in step S108, the verification feedback module (728) verifies the repair result after the repair operation is completed; step S601, queries the current status of virtualization resources; step S602, determines whether the current status is consistent with the expected status; if consistent, then step S603 is executed to mark the repair as successful, and step S604 is executed to update the task log to the success status; if inconsistent, then step S605 is executed to mark the repair as failed, and step S606 is executed to retain the failure log for subsequent analysis; then step S607 is executed to publish the repair result to the unified message channel (740) to notify other business subsystems (750) to update the relevant status.

[0019] Preferably, in step S109, when step S108 is successfully verified, the corresponding failure log or repair log is updated to a successful state through the task log module (721), and the repair completion time, repair strategy identifier, execution node and verification result are recorded, so that the operation trajectory chain forms a closed loop record from failure, repair to success. Preferably, in step S110, when the verification in step S108 fails, or when step S106 has been marked as permanently failed, the failure log, number of repairs, error code, error information and resource release record are retained, and a manual intervention notification is sent to other business subsystems (750) or operation and maintenance system through the unified message channel (740).

[0020] Compared with the prior art, the beneficial effects of the present invention are: 1. Precise fault location: Through a complete task log chain, the specific operation step of the failure can be accurately located, avoiding blind troubleshooting and improving the efficiency of fault diagnosis; it can accurately point out network configuration failure, thereby effectively shortening the troubleshooting time. 2. Intelligent Repair: Targeted repair strategies are selected based on different reasons for failure, rather than simple retries, significantly improving the repair success rate; for example, when network configuration fails, IP addresses are reassigned instead of reusing potentially conflicting IP addresses, thus increasing the repair success rate. 3. High degree of automation: The entire repair process requires no manual intervention. The system can automatically detect faults, diagnose causes, select strategies, execute repairs, and verify results, which greatly reduces operation and maintenance costs. According to statistics, it can reduce a lot of manual handling time for VSM faults.

[0021] 4. Good state consistency: Through resource binding and release mechanisms, the state consistency during the repair process is ensured, avoiding resource leakage and state conflicts; this effectively prevents the long-term occupation of resources such as IP addresses and storage space due to failure of intermediate steps.

[0022] 5. Strong concurrency security: Through the distributed lock mechanism, conflicts caused by concurrent repairs of multiple nodes are effectively prevented, ensuring the stability and reliability of the system; in a distributed cluster environment, it can ensure that repair tasks of the same VSM will not be executed repeatedly or interfere with each other.

[0023] 6. Controllable retry mechanism: By limiting the maximum number of repairs, the waste of resources and excessive system load caused by infinite retries are avoided, while also providing opportunities for manual intervention in serious faults; this ensures the efficiency of automated repair while also taking into account system stability.

[0024] 7. High scalability: This method is not only applicable to the virtualization resource management of cloud cryptographic machines, but can also be extended to other types of cloud computing resource management scenarios, and has good versatility and scalability. Attached Figure Description

[0025] Figure 1 A flowchart of a closed-loop self-healing method for cloud cryptographic machine virtualization resources; Figure 2 A schematic diagram of the task log recording process for a closed-loop self-healing method for cloud cryptographic machine virtualization resources; Figure 3 A flowchart for fault diagnosis and repair strategy selection in a closed-loop self-healing method for cloud cryptographic machine virtualization resources; Figure 4 A schematic diagram illustrating the state consistency guarantee mechanism of a closed-loop self-healing method for cloud cryptographic machine virtualization resources; Figure 5 A closed-loop self-healing method for cloud cryptographic machine virtualization resources and its distributed concurrent control flowchart; Figure 6 A flowchart illustrating the verification and feedback process of a closed-loop self-healing method for cloud cryptographic machine virtualization resources; Figure 7 This is an architectural block diagram of a closed-loop self-healing system for cloud cryptographic machine virtualization resources. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example Please see Figures 1-7 As shown, the present invention provides a technical solution comprising the following steps: S101, Task Log Recording: Completely collect and connect all process operation logs in the full lifecycle management process of cloud cryptographic machine virtualization resources; S102. Fault detection and diagnosis: periodically scan abnormal task logs, locate faulty operation links, and query the cumulative number of repairs for the task. S103. Intelligent repair strategy selection: Match the corresponding repair strategy with the fault link and the cumulative number of repairs, and distinguish between thresholds to determine automatic repair or manual intervention. S104. Automatic retry with a limited number of attempts. Before performing a repair, the cumulative number of repair attempts is checked. If the threshold is not reached, the corresponding repair strategy is executed. The repair count is updated after each repair. S105. Ensure state consistency: Update task resource statistics and bind allocated resources based on repair results; automatically reclaim temporary resources when failure occurs due to exceeding limits. S106. Mark the task as permanently failed or require manual intervention. If the repair exceeds the limit or there is no matching repair strategy, mark the task as permanently failed, retain the log and push the alarm. S107. Distributed concurrency control, relying on distributed locks to isolate multi-node repair tasks, and automatically releasing locks upon timeout to avoid abnormal deadlocks of nodes. S108. Verification and feedback of repair results: Verify the actual operating status of virtualized resources and synchronize the repair results to each business subsystem through the message channel. S109. Log closed-loop update and manual notification of anomalies: If the repair verification is successful, the full-link log is updated to form a closed loop. If the verification fails or the failure is permanent, the fault data is retained and a manual processing notification is issued. S110. Retain failure logs or manual intervention. When the repair verification fails or the task is marked as permanently failed, retain complete fault records and push manual processing notifications to the business and operation and maintenance systems through a unified message channel.

[0028] In this embodiment, step S101 involves performing detailed log recording for each operation step during the lifecycle management of the cloud cryptographic machine virtualization resources via the task log module (721); this step is related to... Figure 2Specifically, the steps include: Step S201, obtaining the operation type, which includes one of creating, configuring a network, importing data images, starting, stopping, and destroying; Step S202, obtaining the target resource identifier, used to uniquely identify the virtualized resource being operated on; Step S203, obtaining the execution status, which includes success, failure, and in progress; Step S204, obtaining the timestamp; Step S205, obtaining the operation details, which includes at least one of operation parameters, execution environment, error code, or error message; Step S206, generating a task log; and Step S207, appending the generated task log to the operation trajectory chain according to the operation execution order.

[0029] In this embodiment, step S102 involves the fault detection module (722) periodically scanning virtualized resource tasks in abnormal states, identifying failed task logs, and the diagnostic engine module (723) performing fault diagnosis; this step is related to... Figure 3 Steps S301 to S304 correspond to the following steps: Step S301, extract the latest failure log; Step S302, identify the operation type in the latest failure log; Step S303, determine the faulty link based on the operation type; Step S304, query the cumulative number of repairs for the target task.

[0030] In this embodiment, step S103 involves the strategy selection module (724) selecting a repair strategy based on the faulty component and the cumulative number of repairs; this step is related to... Figure 3 Steps S305 to S308 correspond to the following steps: Step S305: Select a repair strategy based on the faulty component; when the number of repairs reaches or exceeds the maximum repair threshold, execute step S306 to mark it as permanently failed; when the number of repairs is less than the maximum repair threshold and a matching strategy exists, execute step S307 to execute the corresponding repair strategy; when no matching strategy exists, execute step S308 to adopt a fallback strategy or switch to manual intervention.

[0031] The repair strategy includes: if the failed operation is network configuration, then reassign available IP addresses, subnet masks, and gateways from the IP address pool, call the network configuration interface of the cloud cryptographic machine (710) to reconfigure the virtualization resource network, and record the binding relationship between IP addresses and virtualization resources; if the failed operation is importing data images, then obtain the data image identifier associated with the target task, retrieve the corresponding data image file from the image storage system, and call the image import interface of the cloud cryptographic machine (710) to re-execute the image import; if the failed operation is startup, then call the startup interface of the cloud cryptographic machine (710) to re-execute the virtualization resource startup; if the failed operation is stop, then call the stop interface of the cloud cryptographic machine (710) to re-execute the virtualization resource stop; if the failed operation is creation, then clean up the allocated temporary resources and re-execute the virtualization resource creation.

[0032] In this embodiment, in step S104, the repair execution module (725) checks the cumulative number of repairs for the current task before each repair operation is performed; if the number of repairs is less than the preset maximum repair threshold, the repair strategy selected in step S103 is executed; if the number of repairs reaches or exceeds the maximum repair threshold, the process proceeds to step S106; after each repair operation is performed, regardless of success or failure, the number of repairs is incremented by 1 and updated in the task record.

[0033] In this embodiment, step S105 ensures the consistency of the system state during the repair operation through the state management module (726); this step is related to... Figure 4 Specifically, the steps are as follows: Step S401, determine whether the repair was successful; if the repair was successful, proceed to Step S402, increment the successful resource count; Step S403, decrement the failed resource count; Step S404, update the overall task status; Step S405, determine whether the repair operation involves resource allocation; if resource allocation is involved, proceed to Step S406, establish the binding relationship between the resource and the virtualization resource; if resource allocation is not involved, end the state consistency processing; if Step S401 determines that the repair was unsuccessful, proceed to Step S407, determine whether the maximum number of repairs has been reached; if the maximum number of repairs has been reached, proceed to Step S408, release the occupied temporary resources; if the maximum number of repairs has not been reached, return to Step S103 to continue selecting and executing the repair strategy.

[0034] In this embodiment, in step S106, when the cumulative number of repairs reaches or exceeds the maximum repair threshold, or when the fault type cannot match the preset repair strategy, the target task is marked as permanently failed by the status management module (726), and the failure log is retained and alarm information is issued by the verification feedback module (728) so that manual intervention can be carried out.

[0035] In this embodiment, step S107, in a distributed deployment environment, interacts with the distributed lock service (730) through the concurrency control module (727) to prevent multiple management nodes from concurrently repairing the same task; this step is related to... Figure 5 Specifically, the steps are as follows: Step S501, attempt to acquire the distributed lock, the lock key value is generated based on the task identifier; if the acquisition is successful, proceed to step S502, perform the repair operation, and after the repair is completed, proceed to step S503, release the distributed lock; if the acquisition fails, proceed to step S504, skip the current task and wait for the next scheduling cycle; if the repair node is abnormal and does not actively release the lock, proceed to step S505, the distributed lock automatically expires after the maximum holding time expires.

[0036] In this embodiment, step S108 verifies the repair result after the repair operation is completed by the verification feedback module (728); this step is related to... Figure 6 Specifically, the steps are as follows: Step S601, query the current status of the virtualization resources; Step S602, determine whether the current status is consistent with the expected status; if consistent, execute Step S603, mark the repair as successful, and execute Step S604, update the task log to the success status; if inconsistent, execute Step S605, mark the repair as failed, and execute Step S606, retain the failure log for subsequent analysis; then execute Step S607, publish the repair result to the unified message channel (740), and notify other business subsystems (750) to update the relevant status.

[0037] In this embodiment, in step S109, when step S108 is successfully verified, the corresponding failure log or repair log is updated to a successful state through the task log module (721), and the repair completion time, repair strategy identifier, execution node and verification result are recorded, so that the operation trajectory chain forms a closed loop record from failure, repair to success. In this embodiment, when step S108 fails to verify or step S106 has been marked as permanently failed, the failure log, number of repairs, error code, error information and resource release record are retained in step S110, and a manual intervention notification is sent to other business subsystems (750) or operation and maintenance system through the unified message channel (740).

[0038] Example 1: Repairing network configuration after VSM creation failure: Suppose that in a cloud cryptography machine cluster, 10 VSM instances need to be created for a cryptographic service. During the execution process, the 5th VSM fails at the network configuration step.

[0039] Step 1: Fault Detection The system performs a repair task scan every 2 minutes via the fault detection module (722). In this scan, a failure log was found for the VSM creation task with task ID TASK-20260511-005. The operation log for this task was queried via the diagnostic engine module (723), yielding the following record: Step 2: Fault Diagnosis: The diagnostic engine module (723) extracts the latest failure log LOG-002 and identifies the failure operation type as "configure network". The cumulative number of repairs for this task is 0, which is less than the maximum repair threshold of 3, so repair can be performed.

[0040] Step 3: Repair Strategy Selection: The strategy selection module (724) selects a network reconfiguration repair strategy based on the failed operation type "Configure Network": Retrieve available IP addresses from the IP address pool for the resource group to which the password service belongs; Assign a new IP address: 192.168.10.50, subnet mask: 255.255.255.0, gateway: 192.168.10.1; Call the network configuration interface of the cloud cryptographic machine (710), and pass in the VSM identifier vsm_abc123 and the newly assigned IP parameters.

[0041] Step 4: Perform the repair: The repair execution module (725) calls the network configuration API of the cloud cryptographic machine (710) to perform a network reconfiguration operation. After successful configuration, the status management module (726) records the binding relationship between IP address 192.168.10.50 and VSM-005 in the IP address allocation table.

[0042] Step 5: Status Update: After a successful repair, the status management module (726) performs the following status update operation: For batch creation task TASK-20260511-005, its successful VSM count has been increased from 4 to 5. For batch creation task TASK-20260511-005, its failure VSM count has been reduced from 6 to 5; Update the status of log LOG-002 from "failure" to "success"; Increase the number of repairs from 0 to 1.

[0043] Step 6: Result Verification and Feedback The verification feedback module (728) queries the current status of VSM-005 to confirm that its network configuration has been successfully applied, the IP address is 192.168.10.50, and the status is "normal operation". After the verification is successful, the verification feedback module (728) publishes the repair result to the unified message channel (740) to notify other business subsystems (750) such as the gateway service to update the routing configuration.

[0044] Example 2: Automatic repair of VSM startup failure: Suppose that during a batch creation of VSMs, the 8th VSM fails during the startup step.

[0045] Step 1: Fault Detection The fault detection module (722) scanned and found that the VSM creation task with task ID TASK-20260511-008 had a failure log. The diagnostic engine module (723) queried the operation log: Step 2: Fault Diagnosis: The diagnostic engine module (723) extracts the latest failure log LOG-004 and identifies the failed operation type as "startup". The cumulative number of repairs for this task is 0, indicating that repair can be performed.

[0046] Step 3: Repair strategy selection: The strategy selection module (724) selects the restart repair strategy according to the failure operation type "start": calls the start interface of the cloud cryptographic machine (710) and passes in the VSM identifier vsm_def456; Waiting for the startup process to complete; timeout period is 60 seconds.

[0047] Step 4: Perform the repair: The repair execution module (725) calls the startup API of the cloud cryptographic machine (710) to perform a VSM restart operation. After successful startup, the VSM status changes to "normal operation".

[0048] Step 5: Status Update: After a successful repair, the status management module (726) updates the task status: For batch creation task TASK-20260511-008, its successful VSM count is increased by 1; For batch creation task TASK-20260511-008, its failure VSM count is reduced by 1; The number of repair attempts increased from 0 to 1.

[0049] Step 6: Result Verification and Feedback The verification feedback module (728) queries the status of VSM-008 to confirm that it has been successfully started and is in normal operation. After verification, a repair success message is published to the unified message channel (740).

[0050] Example 3: Processing to reach the maximum number of repairs: Suppose a VSM fails three times in a row during the network configuration step.

[0051] Step 1: First Repair: The fault detection module (722) detected a network configuration failure, and the diagnostic engine module (723) found that the number of times it had been queried for repair was 0. The strategy selection module (724) selects network reconfiguration, and the repair execution module (725) assigns IP address 192.168.10.60; If the repair still fails, the status management module (726) increases the number of repair attempts to 1.

[0052] Step 2: Second repair: The fault detection module (722) detected a network configuration failure, and the diagnostic engine module (723) checked the repair count and found it to be 1. The policy selection module (724) selects network reconfiguration, and the repair execution module (725) assigns a new IP address 192.168.10.61; The repair still failed, and the status management module (726) increased the number of repair attempts to 2.

[0053] Step 3: Third repair; The fault detection module (722) detected a network configuration failure, and the diagnostic engine module (723) checked the number of repair attempts and found it to be 2. The strategy selection module (724) selects network reconfiguration, and the repair execution module (725) assigns a new IP address 192.168.10.62; The repair still failed, and the status management module (726) increased the number of repair attempts to 3.

[0054] Step 4: Mark permanent failures and clean up resources; During the fourth scan, the diagnostic engine module (723) detected that the number of repairs had reached the maximum threshold of 3; The status management module (726) marks the task as a permanent failure and will no longer perform automatic repair; The status management module (726) completely releases the occupied IP address 192.168.10.62 to prevent resource leakage; The verification feedback module (728) sends an alarm notification to the unified message channel (740), prompting the operation and maintenance personnel to manually intervene. (At the same time, the system will record the specific reasons for this permanent failure and historical repair attempts, so that the operation and maintenance personnel can perform manual analysis and optimize the repair strategy).

[0055] Example 4: Concurrency Control in a Distributed Environment Suppose the system has three management nodes, Node-A, Node-B, and Node-C, and they all detect the same failed task, TASK-20260511-010.

[0056] Step 1: Lock contention 0; The concurrency control module (727) of Node-A first attempts to acquire the distributed lock provided by the distributed lock service (730), with the key value of the lock being "repair_lock:TASK-20260511-010"; Node-A successfully acquired the lock. The minimum holding time for the lock is 2 minutes, and the maximum holding time is 4 minutes. The concurrency control module (727) of Node-B and Node-C attempted to acquire the same lock, but failed.

[0057] Step 2: Perform the repair; The repair execution module (725) of Node-A performs the repair operation, which takes 30 seconds to complete; Node-A's concurrency control module (727) releases the distributed lock; When Node-B and Node-C scan again in the next scheduling cycle (2 minutes later), they find that the task has been successfully repaired and skip processing.

[0058] Step 3: Prevent deadlock; If Node-A crashes abnormally during the repair process and does not actively release the lock; The distributed lock service (730) automatically expires after the maximum holding time of 4 minutes; Node-B or Node-C can reacquire the lock and perform repairs on the next scan.

[0059] Example 5: System Architecture Implementation; The system architecture of this method is as follows: Figure 7 As shown, it mainly includes the following modules: The Task Log module (721) is responsible for recording all operation logs during the VSM lifecycle management process, including operation type, target resource, execution status, timestamp, and other information. Logs are stored in a relational database, supporting fast querying and sorting. This module provides a log writing interface to the upper layer and a log query interface to the lower layer, ensuring the integrity and traceability of log data.

[0060] Fault Detection Module (722): Periodically (e.g., every 2 minutes) scans all task logs to identify tasks in a failed state. This module is implemented using a scheduled task framework and supports distributed deployment. Its main function is to trigger the diagnostic engine module to analyze abnormal tasks.

[0061] Diagnostic Engine Module (723): Analyzes failure logs, extracts failure operation types, queries repair counts, and determines whether repair can be performed. This module implements the core logic of fault diagnosis. It receives trigger signals from the fault detection module, interacts with the task log module to obtain log data, and passes the diagnostic results to the strategy selection module.

[0062] Strategy Selection Module (724): Based on the fault diagnosis results, selects the corresponding repair strategy from the strategy library. The strategy library supports dynamic expansion, allowing the addition of new repair strategies without modifying the core code. This module receives the diagnostic results from the diagnostic engine module and selects the most suitable repair solution according to preset strategy matching rules, then sends the selected strategy instruction to the repair execution module.

[0063] Repair Execution Module (725): This module calls the management interface of the cloud cryptographic machine (710) to perform specific repair operations, such as network configuration, image import, startup, and shutdown of the virtual security modules (711, 712, 713). This module encapsulates the communication protocol with the cloud cryptographic machine and supports devices from various manufacturers. It receives repair instructions from the policy selection module and is responsible for the actual interaction with the underlying cloud cryptographic machine virtualization platform.

[0064] Status Management Module (726): Maintains information such as task status, resource binding relationships, and repair counts. This module uses a transaction mechanism to ensure data consistency. It interacts with the task log module and repair execution module to ensure real-time updates and accuracy of system status.

[0065] Concurrency control module (727): Implements concurrency control based on distributed lock service (730) to prevent multiple nodes from repairing the same task simultaneously. This module integrates the client of distributed lock service, providing lock acquisition and release functions before the repair execution module attempts to perform repair operations, ensuring the atomicity of operations in a distributed environment.

[0066] Verification Feedback Module (728): Verifies the repair results, updates the system status, and publishes the repair result message through the unified message channel (740). This module implements decoupled communication with other business subsystems (750). It receives the repair results from the repair execution module, interacts with the task log module and the status management module to update relevant information, and finally notifies other dependent services of the repair results through the message middleware.

[0067] These modules work together to achieve a closed-loop self-healing function for the cloud cryptographic machine's virtualized resources. The repair execution module interacts with the virtual security module in the cloud cryptographic machine through interface calls, the concurrency control module communicates with the distributed lock service, and the verification feedback module transmits messages and synchronizes states with other external business subsystems through a unified message channel.

[0068] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their likenesses.

[0069] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A closed-loop self-healing method and system for cloud cryptographic machine virtualization resources, characterized in that, Includes the following steps: S101, Task Log Recording: Completely collect and connect all process operation logs in the full lifecycle management process of cloud cryptographic machine virtualization resources; S102. Fault detection and diagnosis: periodically scan abnormal task logs, locate faulty operation links, and query the cumulative number of repairs for the task. S103. Intelligent repair strategy selection: Match the corresponding repair strategy with the fault link and the cumulative number of repairs, and distinguish between thresholds to determine automatic repair or manual intervention. S104. Automatic retry with a limited number of attempts. Before performing a repair, check the cumulative number of repair attempts. If the threshold is not reached, execute the corresponding repair strategy. Update the repair count after each repair. S105. Ensure state consistency: Update task resource statistics and bind allocated resources based on repair results; automatically reclaim temporary resources when failure occurs due to exceeding limits. S106. Mark the task as permanently failed or require manual intervention. If the repair exceeds the limit or there is no matching repair strategy, mark the task as permanently failed, retain the log and push the alarm. S107. Distributed concurrency control, relying on distributed locks to isolate multi-node repair tasks, and automatically releasing locks upon timeout to avoid abnormal deadlocks of nodes. S108. Verification and feedback of repair results: Verify the actual operating status of virtualized resources and synchronize the repair results to each business subsystem through the message channel. S109. Log closed-loop update and manual notification of anomalies: If the repair verification is successful, the full-link log is updated to form a closed loop. If the verification fails or the failure is permanent, the fault data is retained and a manual processing notification is issued. S110. Retain failure logs or manual intervention. When the repair verification fails or the task is marked as permanently failed, retain complete fault records and push manual processing notifications to the business and operation and maintenance systems through a unified message channel.

2. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S101, during the lifecycle management of the cloud cryptographic machine virtualization resources, the task log module (721) performs detailed log recording for each operation step; step S201, obtain the operation type, which includes one of creation, network configuration, data image import, start, stop, and destruction; step S202, obtain the target resource identifier, used to uniquely identify the virtualization resource being operated on; step S203, obtain the execution status, which includes success, failure, and in progress; step S204, obtain the timestamp; step S205, obtain the operation details, which include at least one of operation parameters, execution environment, error code, or error message; step S206, generate the task log; Step S207: Append the generated task log to the operation trajectory chain in the order of operation execution.

3. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S102, the fault detection module (722) periodically scans virtualized resource tasks in an abnormal state, identifies failed task logs, and the diagnostic engine module (723) performs fault diagnosis; step S301, extracts the latest failure log; step S302, identifies the operation type in the latest failure log; step S303, determines the faulty link according to the operation type; step S304, queries the cumulative number of repairs for the target task.

4. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S103, the strategy selection module (724) selects a repair strategy based on the faulty component and the cumulative number of repairs; in step S305, the repair strategy is selected based on the faulty component; when the number of repairs reaches or exceeds the maximum repair threshold, step S306 is executed to mark it as permanently failed; when the number of repairs is less than the maximum repair threshold and a matching strategy exists, step S307 is executed to execute the corresponding repair strategy; when no matching strategy exists, step S308 is executed to adopt a fallback strategy or switch to manual intervention.

5. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S104, the repair execution module (725) checks the cumulative number of repairs for the current task before each repair operation is performed; if the number of repairs is less than the preset maximum repair threshold, the repair strategy selected in step S103 is executed. If the number of repairs reaches or exceeds the maximum repair threshold, proceed to step S106; after each repair operation is performed, regardless of success or failure, the number of repairs is incremented by 1 and updated in the task record.

6. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S105, the consistency of the system state is ensured during the repair operation through the state management module (726); in step S401, it is determined whether the repair is successful; if the repair is successful, step S402 is executed sequentially to increment the successful resource count; in step S403, the failed resource count is decremented. Step S404: Update the overall status of the task; Step S405: Determine whether the repair operation involves resource allocation; If it involves resource allocation, proceed to step S406: Establish the binding relationship between the resource and the virtualization resource. If resource allocation is not involved, then end the state consistency process; If step S401 determines that the repair was unsuccessful, then step S407 is executed to determine whether the maximum number of repairs has been reached. If the maximum number of repairs is reached, proceed to step S408 to release the occupied temporary resources; If the maximum number of repairs has not been reached, return to step S103 to continue selecting and executing a repair strategy.

7. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S106, when the cumulative number of repairs reaches or exceeds the maximum repair threshold, the target task is marked as a permanent failure by the status management module (726), the failure log is retained and an alarm message is issued by the verification feedback module (728), and manual intervention is required.

8. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S107, in a distributed deployment environment, the concurrent control module (727) interacts with the distributed lock service (730) to prevent multiple management nodes from concurrently repairing the same task; in step S501, an attempt is made to acquire the distributed lock, the key value of which is generated based on the task identifier; if the acquisition is successful, step S502 is executed to perform the repair operation, and after the repair is completed, step S503 is executed to release the distributed lock; If acquisition fails, proceed to step S504, skip the current task and wait for the next scheduling cycle; if the node is repaired and the lock is not actively released, proceed to step S505, and the distributed lock will automatically expire after the maximum holding time expires.

9. The closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S108, the verification feedback module (728) verifies the repair result after the repair operation is completed; in step S601, the current status of the virtualization resources is queried. Step S602: Determine whether the current state is consistent with the expected state; If they match, proceed to step S603 to mark the repair as successful, and proceed to step S604 to update the task log to a successful status; if they do not match, proceed to step S605 to mark the repair as failed, and proceed to step S606 to retain the failure log for subsequent analysis; then proceed to step S607 to publish the repair result to the unified message channel (740) and notify other business subsystems (750) to update the relevant status.

10. A closed-loop self-healing method and system for cloud cryptographic machine virtualization resources according to claim 1, characterized in that: In step S109, when step S108 is successfully verified, the corresponding failure log or repair log is updated to a successful status through the task log module (721), and the repair completion time, repair strategy identifier, execution node and verification result are recorded, so that the operation trajectory chain forms a closed loop record from failure to repair to success.

Images