Spacecraft modular computing cloud platform and OpenStack reconfigurable method

Abstract

The invention provides a spacecraft modular computing cloud platform and an OpenStack reconfigurable method.According to the platform, a plurality of computing nodes are interconnected through a high-speed bus and matched with a dual-redundancy PMC board card to achieve hardware monitoring and fault control, a system deconstructs OpenStack into an SGC containerized service unit, and on-demand cutting deployment and resource scheduling are achieved in combination with an assembly bus; and designing a multi-stage operation process, such as a platform power-up management domain, computing node power-up diagnosis, software starting registration, cloud service initialization, business virtual machine deployment and a second-level fault detection and automatic recovery mechanism. Compared with the prior art, the deployment and maintenance cost is greatly reduced, the reliability and flexibility of the system are improved, the problems that the original OpenStack fault processing is time-consuming and depends on manpower are solved, and the method is suitable for high-reliability demand scenes such as spacecraft on-board data processing and edge calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of high-performance computing systems, distributed system architecture and cloud computing platform design, in particular to a spacecraft modular computing cloud platform and an OpenStack reconfigurable method. BACKGROUND

[0002] The current spacecraft types are mainly communication, navigation, remote sensing and scientific experiment satellites. Due to the progress of science and technology, the types of tasks on the satellite are becoming more and more rich and complex. For example, the radar remote sensing satellite needs to process radar data on the satellite and identify the preset target. Only the target specific information is downloaded to the ground through the telemetry channel to enable the ground unit to respond quickly. Such a complex processing process puts higher requirements on the computer resources on the satellite. The existing single computer board card is already difficult to meet the future satellite computing needs. At the same time, cloud computing platforms such as OpenStack are widely used in data centers and enterprise-level application environments, and have good modular architecture and virtualization capabilities. However, OpenStack itself has many modules, complex dependencies, and high coupling. It is usually deployed on high-performance dedicated servers. This way results in high deployment cost and poor flexibility, which is not suitable for edge computing, task-driven small computing environments or high-reliability embedded systems in special scenarios. When the computing node of the original OpenStack fails, the administrator needs to read the log, analyze the cause, and then handle the failure.

[0003] On the other hand, with the continuous improvement of the performance of COTS devices, it has become a development trend to use general-purpose computing nodes to build customized computing platforms. However, there is still a lack of an efficient, reliable and flexible way to build these devices into a unified, highly reliable and high-performance system platform in the existing system. At the same time, there is a lack of decoupled reconfiguration mechanism for OpenStack to achieve tailored deployment, which hinders its widespread application in lightweight and multi-scenario deployment.

[0004] In summary, this paper proposes a cloud system built on high-performance devices on the satellite. This system can provide sufficient or even exceed the demand for computing resources for the satellite. At the same time, by adjusting the OpenStack system, it can meet the high reliability requirements of spacecraft and flexibly adapt to the demand for computing resources in different task stages. SUMMARY

[0005] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art. A spacecraft modular computing cloud platform and an OpenStack reconfigurable method are proposed. Combined with the system-level assembly bus and self-management mechanism, supplemented by the service granularity reconfiguration technology of OpenStack, a cloud platform with reconfigurable, tailorable, fault monitoring, fault tolerance recovery and elastic deployment capabilities is realized.

[0006] To achieve the above object, the technical solution adopted by the present application is: The present application proposes a spacecraft modular computing cloud platform, comprising: A plurality of computing nodes, each node carrying a spacecraft on-board computing task; A high-speed bus for realizing the communication interconnection of the computing nodes and the PMC board card; A PMC board card for monitoring and managing the health status of each computing node, including a main PMC board card and a standby PMC board card, the dual-redundancy PMC board card using a hardware watchdog, the hardware watchdog being only one of the triggering conditions, and the hardware watchdog being able to independently trigger a reset signal when the software crashes; A power distribution unit for supplying power to the PMC board card and the computing node; An assembly bus logical structure deployed in the computing node for realizing node registration, resource scheduling, fault transfer and task distribution, and automatically forming a disaster recovery copy in other nodes after being started by an initial control node; An OpenStack deployment module for decomposing OpenStack into a plurality of minimum service units of control type, network type, storage type and management type, each SGC encapsulating complete running dependencies and communication protocols, and being dynamically issued to the computing node in the form of a container image; A deployment and running mechanism, the computing node reporting hardware resource information to a system management bus, the control node analyzing a function map to generate a minimum deployment topology, and the SGC container being automatically deployed according to node load and recorded to a system state map; A system fault tolerance and reliability mechanism, the control type SGC service being deployed in double copies and synchronizing states in real time, and a node running a lightweight Agent for monitoring the health of the container and reporting to the assembly bus.

[0007] Further, the computing node comprises a computer board card and a graphics computer board card, connected with the connection port, the high-speed bus and the power distribution unit; The high-speed bus comprises an Ethernet, a vpx bus or an optical fiber; The PMC board card uses an independent low-power high-reliability processor, a commercial memory with error detection and correction function and a hardware watchdog.

[0008] Further, the assembly bus logical structure further comprises a disaster recovery copy mechanism: after being started by an initial control node, the assembly bus automatically generates a disaster recovery copy in other computing nodes, and when the initial control node fails, the disaster recovery copy immediately takes over the resource scheduling function.

[0009] Further, the deployment and running mechanism registers and reports CPU, memory, storage, architecture type resources to the assembly bus in the starting stage, and when a node is down or the network is abnormal, the system immediately performs service migration or hot switching to ensure continuous operation of critical services.

[0010] Further, the system fault tolerance and reliability mechanism: the assembly bus periodically performs heartbeat detection to exclude failed nodes from the state diagram, supports a service backfill mechanism based on priority, and after a new node is added, the migrated service can be restored to the original deployment strategy.

[0011] The application provides a reconfigurable method of a spacecraft modular computing cloud platform OpenStack, which comprises the following steps: Step 1: platform power-on and management domain establishment, the power distribution unit preferentially powers on the main PMC board card, the main PMC board card completes self-checking and firmware initialization, then powers on the standby PMC board card, the standby PMC board card also performs self-checking and firmware initialization, the dual PMC board cards synchronize states through a dedicated heartbeat link, and a management domain is formed. Step 2: power on of the computing node and health diagnosis, the main PMC board card orderly or in parallel powers on all computing nodes through the power distribution unit, the PMC board card polls each computing node through an independent management network bus, generates a hardware resource map, and isolates a fault node. Step 3: software startup of the computing node and registration of resource information to the control node to obtain network configuration. Step 4: cloud platform service initialization, the control node starts core OpenStack services, available resources in the computing node are reported to the OpenStack control node, and the main PMC board card synchronizes information with the OpenStack control node through an API. Step 5: business virtual machine deployment and system readiness process, the OpenStack NovaApi receives a workload request from a ground station or other subsystems on the satellite, the nova-scheduler schedules a virtual machine instance to a healthy board card according to a preset strategy, the nova-compute driver KVM creates a virtual machine, and business applications in the virtual machine are automatically started.

[0012] Step 6: fault detection and processing process, the main PMC board card monitors the board card state through a current sensor and a hardware watchdog, when the board card current suddenly exceeds a threshold or the watchdog times out, the health management and reconfiguration engine determines that it is a hardware fault, the health management and reconfiguration engine instructs the main PMC board card to cut off the power supply or reset of the fault board card, and then queries a resource map to determine a standby board card, and the business is automatically reconnected to the standby board card to complete recovery.

[0013] Further, in step 3, the computing node registers resource information to the control node, specifically, the nova-compute registers to the control node to report the resource of the node; the neutron-openvswitch-agent connects the neutron-server of the control node to obtain the network configuration.

[0014] Further, in step 4, the Nova-Compute agent in the computing node discovers and connects to the message bus and API endpoint of the control node, and the Nova-Compute agent reports the available resource of the board card to the OpenStack control node.

[0015] Further, in step 6, the fault detection further includes that the health agent process on the board card of the computing node periodically sends a heartbeat packet to the health management and reconstruction engine, and when the health management and reconstruction engine continuously loses the heartbeat packet, the corresponding board card is listed as a suspicious node; the OpenStack Nova service monitors the communication state of the nova-compute agent on the board card of the computing node in real time, and when the communication is interrupted, the corresponding board card of the computing node is listed as a suspicious node.

[0016] Further, in step 6, the business automatic reconnection of the standby board card reestablishes the virtual machine on the standby board card through the OpenStack Nova API calling command, associates the original network configuration and security certificate and is associated to the new instance, mounts the original disk to the new instance by using the shared storage, and the business application automatically reconnects the new instance to complete the recovery.

[0017] Compared with the prior art, the present application achieves the following technical effects: One, using devices to build a computing cloud platform greatly reduces the deployment and maintenance costs caused by using anti-radiation devices in the traditional aerospace field; Two, by decomposing OpenStack into SGC, on-demand tailoring deployment is realized, resource occupation is reduced, and heterogeneous architectures such as X86 and ARM are supported to adapt to diversified application scenarios; Three, by means of the double-redundancy design of the PMC board card, the assembly bus, the service copy and the fault-tolerant scheduling mechanism, combined with the second-level fault detection and automatic isolation and recovery capability, the problem that the original OpenStack fault needs to be handled manually and takes a long time is solved, the system reliability and fault response efficiency are significantly improved, and the introduction of the reliable infrastructure domain (including the PMC, the isolation circuit and the independent management network) enables the system to have self-perception and management capabilities, and is more suitable for spacecraft and other resource-limited and high-reliability demand scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to facilitate the description, the present application is described in detail by the following specific embodiments and drawings.

[0019] Figure 1 Structure diagram of a modular high-reliability computing platform; Figure 2 Process diagram of platform power-on and management domain establishment; Figure 3 Process diagram of computing node power-on and health diagnosis; Figure 4 Process diagram of computing node software startup and bus registration; Figure 5 Process diagram of cloud platform service initialization; Figure 6 Process diagram of business virtual machine deployment and system readiness; Figure 7 Process diagram for handling failures caused by space environment. DETAILED DESCRIPTION

[0020] The following is a specific embodiment of the present application and further describes the technical solutions of the present application in conjunction with the accompanying drawings, but the present application is not limited to these embodiments; in the following description, specific details such as specific configurations are provided only to help a comprehensive understanding of the embodiments of the present application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application.

[0021] The spacecraft modular computing cloud platform provided by the present application comprises: A plurality of computing nodes, the computing node type comprising at least one of an X86 server, an ARM embedded board card, and a lightweight GPU workstation, the inside comprising a computer board card and a graphic computer board card, connected with a connection port, a high-speed bus, and a power distribution unit, each node running a Linux operating system, built-in lightweight management agent, carrying spacecraft on-board computing tasks; A high-speed bus, the high-speed bus comprising an Ethernet, a vpx bus, or an optical fiber, used for realizing communication interconnection of each computing node and the PMC board card; A power distribution unit, used for powering the PMC board card and the computing node; A PMC board card, comprising a main PMC board card and a backup PMC board card, the PMC board card adopting a commercial memory with a low power consumption and a high correctable rate, and a hardware watchdog, connected with each computing node, a switching node, and a power module through an independent management network bus, and powered through an independent power conversion module, the hardware watchdog being capable of triggering a reset independently when software crashes; An assembly bus logic structure, deployed in the computing node, responsible for node registration, resource scheduling, fault transfer, and task distribution, and capable of automatically generating a disaster recovery copy in other nodes after an initial control node is started, and capable of immediately taking over resource scheduling when the initial control node fails. OpenStack deployment module, for decomposing OpenStack into several control class, network class, storage class and management class minimum service units, each SGC encapsulates complete running dependency and communication protocol, and is dynamically issued to a computing node in the form of a container image; Wherein the SGC includes: Control class SGC: NovaAPI, Keystone, Glance; Network class SGC: NeutronServer, L3Agent, MetadataAgent; Storage class SGC: Cinder, Swift; Management class SGC: Horizon, Heat, Placement; The SGC deployment module supports selecting a single class or a combination of multiple classes of SGC according to task requirements to form a minimum service subset; Deployment and running mechanism, the system adopts a "function graph + template cutting" way to define the required function set, constructs a minimum service set, and registers and reports resources to the assembly bus when starting; the control node parses the graph to generate a deployment topology, and the SGC is automatically deployed according to the node load, and is recorded to the system state graph; when the node is down or the network is abnormal, the system will timely migrate services or hot switch to ensure the running of key services; System fault tolerance and reliability mechanism, each control class SGC service is deployed in double copies, and the state information is synchronized in real time; a lightweight Agent is run on each node to monitor the health of the container and report it; the assembly bus periodically detects the heartbeats and removes the failed nodes; a priority service backfill mechanism is supported, and the migrated services can be restored to the original deployment strategy when a new node is added.

[0022] Implementation configuration based on the hardware structure of the platform, as shown in Figure 1 The basic part of the computing platform of the present application is composed of three computer board cards and three graphic computer board cards, each board card is connected with a power distribution unit and a PMC board card through a high-speed bus (ethernet is selected, and VPX form is adopted in physical form), in addition, the PMC board card is used as a special management board card to monitor and manage the health status of the computer board card, and simultaneously sends control instructions such as power-off, reset and power-on when a fault occurs.

[0023] The computer board card 1-3 corresponds to the port 1-3, the graphic computer 1-3 corresponds to the port 4-6 respectively, the board card and the bus port are matched one by one, and stable data transmission is realized. The PMC board card sets two management board cards of main and standby, the two are connected through a high-speed heartbeat and state synchronization link, real-time synchronization hardware monitoring data and management instructions are realized; the main PMC board card is preferentially accessed to the power distribution unit for power supply, the standby PMC board card is used as backup, only when the main PMC board card fails, the management function is detected and quickly taken over through the heartbeat link, the hardware management is ensured not to be interrupted, and the voltage, current and temperature sensors are configured for each board card, the sensor signals are all accessed to the PMC board card, and data support is provided for hardware state monitoring.

[0024] The spacecraft modular computing cloud platform and the OpenStack reconfigurable method provided by the embodiment of the application are described below, and the reconfigurable method of the spacecraft modular computing cloud platform OpenStack described below can be correspondingly referred to the modular computing cloud platform for spacecraft described above.

[0025] The implementation steps of the platform power-on and management domain establishment process are as shown in the following Figure 2 The spacecraft is powered by a primary power supply, the primary power supply inputs power to a power distribution unit (PDU), the PDU preferentially powers on the active PMC board card (Active PMC), the active PMC board card automatically performs a hardware self-check after starting, checks whether the processor, memory and communication interface are normal, confirms that there is no exception, and then runs the on-board firmware to initialize the hardware environment; then the active PMC board card sends a power-on instruction to the standby management board card (Standby PMC) through the PDU, the standby PMC board card repeats the self-check and firmware startup process; after the two board cards are initialized, the two are connected through a special heartbeat link, when the active PMC board card state is synchronized to the standby PMC board card and there is no data conflict, the management domain is marked as ready.

[0026] The node initialization based on the computing node power-on and health diagnosis process, the starting and health check of the computing node are completed, as shown in the following Figure 3As shown, after the management domain is ready, the main PMC board sends a power-on instruction to the power distribution unit, controls the power distribution module to orderly power on the target board according to the task requirements (such as starting 1 computer board and 1 graphics computer board this time), and avoids excessive instantaneous current by starting 1 node every 1 second, that is, the control logic for powering on the computing node cluster in the schematic diagram; after power-on is completed, the main management board polls the voltage, current and temperature sensors of the powered-on nodes through the CAN bus, and feeds back the collected state data to the main PMC board in real time; the main PMC board marks the node state (healthy / alarm / fault) according to the data, initializes the hardware resource map in the memory, and for the board marked as fault, the main PMC board immediately sends a power-off instruction to the power distribution module to isolate and mark it as a fault node, and no longer supplies power to it, so as to ensure that the fault node does not affect the overall system, and the hardware resource map construction is completed in this stage.

[0027] Based on the software deployment of the computing node software startup and bus registration process, the node software is ready and registered, such as Figure 4 As shown, the healthy computing nodes are started in parallel, and a slim Linux operating system (cutting useless services, retaining only core drivers, and shortening startup time) is booted from local storage or network; after the OS is started, the OpenStack agent is automatically run, wherein the nova-compute agent initiates a registration request to the nova-conductor of the control node, and reports the node CPU, memory and storage resource information; the neutron-openvswitch-agent agent actively connects the neutron-server of the control node to obtain network configuration (such as IP address and gateway); after both agents are successfully registered and feedback normal signals, the node sends a software ready notification to the main PMC, and the node enters the software ready state.

[0028] Based on the service initialization process of the cloud platform, the service is built, and the OpenStack service initialization and state synchronization are completed, such as Figure 5 As shown, the core OpenStack service startup and the management domain establishment are performed in parallel, the control node starts the core services such as nova-api, nova-scheduler, neutron-server and glance-api, the nova-compute agent of the computing node discovers the API endpoint of the control node through the message bus, establishes a stable connection and reports the available node resources to the OpenStack control node; the main PMC queries the node state in the OpenStack database through the API interface, compares it with the local resource map, and if there is a difference, synchronously corrects it to ensure that the states are consistent, completes the "initialization service coordination-synchronize global state" step, and until the cloud platform control plane is ready in the schematic diagram.

[0029] Based on the implementation operation of the service virtual machine deployment and system readiness process, the service deployment and system enablement are realized, such as Figure 6 As shown in the figure, after the master PMC confirms that the cloud platform control plane and all nodes are ready, the resource map is updated to platform ready, and the OpenStack NovaApi starts to receive workload requests (such as remote sensing data processing tasks) from the ground station or other subsystems on the satellite; the nova-scheduler schedules the virtual machine instance to the healthy computer board card according to the preset strategy, and the nova-compute driver KVM creates a virtual machine; the virtual machine automatically loads the remote sensing data processing application program after starting, at which time the service application service is online, and the “start service virtual machine-service application service online” step is completed, and the system is marked as fully put into formal use.

[0030] Based on the fault response of the space environment caused fault handling process, the fault detection and repair are executed, such as Figure 7 As shown in the figure, when the computer board card 1 fails due to space single event latch-up, the current sensor of the PMC detects that its current far exceeds the normal threshold within milliseconds, or the hardware watchdog times out because the OS is locked and not fed, both signals can trigger the health management and reconstruction engine, and the health management and reconstruction engine receives any one of the two signals, and determines it as a serious hardware failure within seconds, and immediately triggers the processing flow, wherein the health agent process on the computer board card periodically sends a heartbeat packet to the health management and reconstruction engine, and when the health management and reconstruction engine loses 2-3 consecutive heartbeat packets, the corresponding board card is listed as a suspicious node; the OpenStack NovaApi service monitors the communication state of the nova-compute agent on each board card in real time, and when the communication is interrupted, the corresponding board card is listed as a suspicious node; The health management and reconstruction engine sends a highest priority instruction to the PMC, and the PMC immediately controls the power distribution module to cut off the power supply of the computer board card 1 to avoid burning and bus interference of the latch device due to overcurrent, and then the health management and reconstruction engine queries the resource map, selects an idle computer board card 2 as a standby node, calls the novaevacuate command through the OpenStack NovaAPI, and rebuilds the instance on the board card 2 according to the original virtual machine configuration, mounts the original Cinder volume to ensure data consistency, and synchronizes the IP, security group configuration and security certificate; the user application automatically reconnects the new virtual machine through the floating IP, completes the service recovery, and realizes fault isolation and service recovery without human intervention.

[0031] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0032] Those skilled in the art can make various modifications or supplements to the specific embodiments described or adopt similar ways to replace, but will not deviate from the inventive concept of the present application or exceed the scope defined by the appended claims.

Claims

1. A spacecraft modular computing cloud platform, characterized by, Comprise: a plurality of computing nodes, each node carrying a spacecraft on-board computing task; a high-speed bus for realizing the communication interconnection of the computing nodes and the PMC board card; a PMC board card for monitoring and managing the health status of each computing node, including a primary PMC board card and a backup PMC board card, the dual-redundant PMC board card uses a hardware watchdog, and the hardware watchdog is only one of the triggering conditions, when the software crashes, the hardware watchdog can still independently trigger a reset signal; a power distribution unit for powering the PMC board card and the computing node; an assembly bus logical structure deployed in the computing node for realizing node registration, resource scheduling, fault transfer and task distribution, and automatically forming a disaster recovery copy in other nodes after being started by an initial control node; an OpenStack deployment module for decomposing OpenStack into a plurality of minimum service units of control type, network type, storage type and management type, each SGC encapsulating complete running dependencies and communication protocols, and being dynamically issued to the computing node in the form of a container image; a deployment and running mechanism in which the computing node reports hardware resource information to the system management bus, the control node analyzes a function map to generate a minimum deployment topology, and the SGC container is automatically deployed according to node load and recorded to a system state map; a system fault tolerance and reliability mechanism in which the control type SGC service is deployed in double copies and the state is synchronized in real time, and a node runs a lightweight Agent to monitor the health of the container and report to the assembly bus.

2. The modular computing cloud platform for spacecraft of claim 1, wherein, The computing node comprises a computer board card and a graphics computer board card, which are connected with the connection port, the high-speed bus and the power distribution unit; The high-speed bus comprises an Ethernet, a vpx bus or an optical fiber; The PMC board card uses an independent low-power high-reliability processor, a commercial memory with error detection and correction function and a hardware watchdog.

3. The modular computing cloud platform for spacecraft of claim 1, wherein, The assembly bus logical structure further comprises a disaster recovery copy mechanism: after the initial control node is started, a disaster recovery copy of the assembly bus is automatically generated in other computing nodes, and when the initial control node fails, the disaster recovery copy immediately takes over the resource scheduling function.

4. The modular computing cloud platform of spacecraft of claim 1, wherein, The deployment and running mechanism registers and reports CPU, memory, storage and architecture type resources to the assembly bus in the starting stage, and when a node is down or the network is abnormal, the system immediately performs service migration or hot switching to ensure the continuous running of key services.

5. The modular computing cloud platform of spacecraft of claim 1, wherein, The system fault tolerance and reliability mechanism: the assembly bus periodically performs heartbeat detection to exclude the failed node from the state map, supports a service backfill mechanism based on priority, and after a new node is added, the migrated service can be restored to the original deployment strategy.

6. A reconfigurable method of a spacecraft modular computing cloud platform OpenStack, applied to the computing cloud platform of any one of claims 1-5, the method comprising the following steps: Step 1: platform power-on and management domain establishment, the power distribution unit preferentially powers on the primary PMC board card, the primary PMC board card completes self-checking and firmware initialization, then powers on the backup PMC board card, the backup PMC board card also performs self-checking and firmware initialization, the dual PMC board cards synchronize the state through a dedicated heartbeat link to form a management domain; Step 2: Power on and health diagnosis of computing nodes, the primary PMC board card powers on all computing nodes in order or in parallel through the power distribution unit, the PMC board card polls each computing node through an independent management network bus, generates a hardware resource map, and isolates faulty nodes; Step 3: Software startup of computing nodes and registration of resource information to the control node to obtain network configuration; Step 4: Cloud platform service initialization, the control node starts the core OpenStack service, and the available resources in the computing nodes are reported to the OpenStack control node, and the primary PMC board card synchronizes information with the OpenStack control node through API; Step 5: Business virtual machine deployment and system readiness process, the OpenStack NovaApi receives workload requests from the ground station or other subsystems on the satellite, the nova-scheduler schedules virtual machine instances to healthy board cards according to the preset strategy, the nova-compute driver KVM creates virtual machines, and the business application in the virtual machine automatically starts Step 6: Fault detection and processing process, the primary PMC board card monitors the board card state through current sensors and hardware watchdogs, when the board card current suddenly exceeds the threshold or the watchdog times out, the health management and reconstruction engine determines it as a hardware failure, the health management and reconstruction engine instructs the primary PMC board card to cut off the power supply or reset the faulty board card, and then queries the resource map to determine the standby board card, and the business automatically reconnects the standby board card to complete recovery.

7. The method of claim 6, wherein the method further comprises: In step 3, the computing node registers resource information to the control node, specifically, the nova-compute registers to the control node and reports the node resources; the neutron-openvswitch-agent connects the control node's neutron-server to obtain network configuration. 8.The method of claim 6, wherein, In step 4, the Nova-Compute agent in the computing node discovers and connects to the message bus and API endpoint of the control node, and the Nova-Compute agent reports the available resources of the board card to the OpenStack control node. 9.The method of claim 6, wherein, In step 6, fault detection also includes the health agent process on the board card of the computing node sending heartbeat packets to the health management and reconstruction engine regularly, when the health management and reconstruction engine loses a heartbeat packet continuously, the corresponding board card is listed as a suspicious node; the OpenStack Nova service monitors the communication state of the nova-compute agent on each computing node in real time, when the communication is interrupted, the board card of the corresponding computing node is listed as a suspicious node. 10.The method of claim 6, wherein, In step 6, the business automatically reconnects the standby board card by calling the OpenStack NovaAPI command to rebuild the virtual machine on the standby board card, associates the original network configuration and security certificate, and associates to the new instance, mounts the new instance to the original disk using shared storage, and the business application automatically reconnects the new instance to complete recovery.

Images