A micro-service container deployment method and system of a flight simulator data disaster recovery system

Abstract

The application relates to the technical fields of cloud computing, container deployment and data disaster recovery, and discloses a micro-service container deployment method and system for a flight simulator data disaster recovery system, which splits a monolithic disaster recovery application into multiple micro-services, configures hierarchical ready tokens based on service dependency topology, generates stateless container images for transmitting and executing the micro-services, writes backup task states into a coordination cache, schedules tasks based on link quality, storage capacity, container instance load and a deadline, performs disk brushing, checking and confirmation on chunked data, atomically renames and submits a file index after completing file checking, and performs truncation or continuous transmission by a takeover container instance that has obtained a distributed lock when a lease expires and a heartbeat stops. The application improves the reliability of multi-base concurrent backup, fault takeover and data state consistency.

Description

Technical Field

[0001] This invention relates to the fields of cloud computing, container deployment, and data disaster recovery technology, specifically to a microservice-based container deployment method and system for a flight simulator data disaster recovery system. Background Technology

[0002] Flight simulators generate a large amount of flight data, configuration data, mission records, and documents during operation, training, and maintenance. To prevent data loss due to equipment failure, base failure, or storage failure, a data disaster recovery system is typically configured for the flight simulator to back up, index, and restore simulator-side files and mission metadata.

[0003] Existing flight simulator data disaster recovery systems mostly adopt a monolithic architecture, centrally deploying the web front-end, back-end interface, device connection, task scheduling, file transfer, and metadata management. This type of architecture, as the number of devices increases, exhibits high coupling between functional modules. Transfer tasks, device connections, and task log writing are prone to interfering with each other, making independent expansion by functional module difficult. For scenarios involving concurrent access from multiple flight simulators and large file backups, the monolithic architecture is also prone to resource bottlenecks in connection pooling, task scheduling, and file transfer.

[0004] In multi-site deployment scenarios, there are differences in link bandwidth, latency, packet loss rate, and storage capacity between different sites. When backup tasks are issued according to fixed configurations or manually in sequence, task scheduling cannot reflect changes in link quality, storage capacity, and container instance load in a timely manner, which can easily lead to some tasks failing to complete within the limited backup window. When the transmission execution node fails midway through file transmission, there is a lack of consistent recovery criteria between task progress, metadata records, and files already written to disk on the target end. This may result in issues such as cached progress exceeding the actual data written to disk, residual temporary files on the target end, duplicate writing of file indexes, or duplicate transmission by the takeover instance.

[0005] Therefore, this application provides a microservice-based container deployment method and system for a flight simulator data disaster recovery system to solve the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide a microservice-based container deployment method and system for flight simulator data disaster recovery, in order to solve the technical problems of high coupling of monolithic architecture, difficulty in concurrent backup expansion, insufficient multi-base resource scheduling, and difficulty in maintaining consistency of task status, file disk status and file index status after transmission failure in existing flight simulator data disaster recovery systems.

[0007] To address the aforementioned technical problems, this invention provides a microservice-based container deployment method for a flight simulator data disaster recovery system, comprising: The monolithic disaster recovery application is split into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice. Layered ready tokens are configured based on service dependency topology, and downstream service startup permits are released after the basic service health probe passes. To generate stateless container images for microservices to be executed during transport, the source path, target path, file size, task version, chunk confirmation bitmap, and confirmation offset of the backup task are written to the coordination cache. Based on link quality, storage capacity, container instance load, and deadline, unschedulable instances are filtered out and sorted, and backup tasks are assigned to the container instances that are ranked higher. The container instance writes block data to the target temporary file. When a single block is flushed to disk and passes verification, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. When the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset, and the accompanying verification file, truncates or resumes the transmission of the target temporary file, and outputs the file index after submission.

[0008] In some specific embodiments, configuring layered ready tokens based on service dependency topology and releasing downstream service startup permits after basic service health checks pass further includes: The basic services include device access microservices and metadata microservices; the downstream services include core services and gateway microservices; the core services include task scheduling microservices and transmission execution microservices; and the layered ready tokens include basic layer ready tokens and core layer ready tokens. Save the service dependency topology; The initialization container of the basic service performs health probes on metadata connection, device connection pool preheating and instance address registration, and writes the basic layer ready token after the probe passes. The core service listens to the base layer ready token and reads the base service instance address before starting. After starting, it writes the core layer ready token. The gateway microservice starts after reading the core layer ready token.

[0009] In some specific embodiments, after saving the service dependency topology, the method further includes: When a failure of a basic service is detected, a dependency interruption signal is sent to the core services that depend on the basic service. After receiving the dependency interruption signal, the core service stops accepting new backup tasks and closes the task delivery channel, while retaining the backup tasks that have already obtained execution rights in the coordination cache. After the basic service rewrites the base layer ready token, it notifies the core service to resume task processing based on the service dependency topology, and notifies the gateway microservice to resume interface forwarding.

[0010] In some specific embodiments, writing the source path, target path, file size, task version, chunk acknowledgment bitmap, and acknowledgment offset of the backup task into the coordination cache further includes: The task identifier records the source path, target path, file size, task version, currently executing container instance, and lease expiration time. A block confirmation bitmap and a distributed lock are established based on the task identifier, and the verification value of each block is recorded. The block confirmation bitmap marks the blocks that have been flushed to disk and passed the verification based on the block sequence number. The confirmation offset is updated with the end position of the consecutive confirmed blocks.

[0011] In some specific embodiments, the container instance writes chunked data to a temporary file on the target end. When a single chunk is flushed to disk and verification passes, the chunk confirmation bitmap and the confirmation offset are updated, further including: The container instance writes chunked data to a target temporary file with a temporary suffix via a secure file transfer channel. Forced flushing and cyclic redundancy check are performed on individual blocks that have been written. When the cyclic redundancy check result is consistent with the source block check value, the bit corresponding to the block number in the block confirmation bitmap is marked as confirmed, and the confirmation offset is updated to the maximum end position of the consecutive confirmed blocks.

[0012] In some specific embodiments, after file segmentation is completed, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file, and the file index is submitted using the task identifier and task version as unique keys. Further steps include: When all blocks are in the confirmed state, generate an accompanying verification file containing a block verification list, file size, overall hash, task identifier, and task version; Read the accompanying verification file to verify the target end temporary file; After verification, the target temporary file is renamed to a backup file. The task identifier and task version are used as unique keys to write the file index to the metadata microservice in an idempotent manner. After a successful write, the task status in the coordination cache is advanced to "committed".

[0013] In some specific embodiments, when the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset, and the accompanying verification file, truncates or resumes the transmission of the target-end temporary file, and outputs the file index after submission, further including: Move backup tasks with expired leases and stopped heartbeats to the interrupt queue; The waiting container instances compete for the distributed lock through atomic writes; The takeover container instance of the distributed lock reads the block confirmation bitmap, the confirmation offset, the size of the target temporary file, and the confirmation size in the accompanying verification file. It takes the minimum recoverable offset to truncate the target temporary file and continues to write block data from the minimum recoverable offset.

[0014] In some specific embodiments, the process of filtering and sorting unschedulable instances based on link quality, storage capacity, container instance load, and deadline further includes: The link quality includes available bandwidth, round-trip latency, and packet loss rate of the cross-base link; the storage capacity includes the remaining capacity of the target end and disk write capability; the container instance load includes the number of concurrent tasks of the container instance; and the deadline is the deadline of the backup task. The task scheduling microservice reads the available bandwidth of the cross-base link, the round-trip latency, the packet loss rate, the remaining capacity of the target end, the disk write capacity, the number of concurrent tasks of the container instance, and the base to which the source simulator belongs; Remove container instances that do not meet the storage reservation conditions, concurrency limit conditions, link availability conditions, or connection pool warm-up conditions; The link margin is determined based on the link quality, and the urgency of the deadline is determined based on the deadline. For the remaining container instances, a ranking score is calculated based on the remaining link capacity, the remaining storage capacity, the affinity with the same base, the urgency of the deadline, historical transmission performance, and the container instance load. Backup tasks are then assigned to container instances with higher ranking scores.

[0015] In some specific embodiments, when the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset, and the accompanying verification file, truncates or resumes the transmission of the target-end temporary file, and outputs the file index after submission, further including: Read the block check list and confirmation size in the accompanying check file, and compare the confirmation size with the maximum consecutive block end offset indicated by the block confirmation bitmap; When the size of the target temporary file is greater than the confirmed size or greater than the maximum continuous block end offset, the target temporary file is truncated to a smaller offset; When the block confirmation bitmap is lost, delete the target temporary file and move the backup task back into the task scheduling queue.

[0016] Based on the same concept, the present invention also provides a microservice-based container deployment system for a flight simulator data disaster recovery system, comprising: The service splitting and gating module is configured to split the monolithic disaster recovery application into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice. It configures layered ready tokens based on service dependency topology and releases downstream service startup permits after the basic service health probe passes. The image and cache module is configured to generate stateless container images for transport execution microservices and write the source path, target path, file size, task version, chunk confirmation bitmap and confirmation offset of the backup task to the coordination cache. The resource scheduling module is configured to filter out unschedulable instances and sort them based on link quality, storage capacity, container instance load and deadline, and then assign backup tasks to the container instances that are ranked higher. The transmission confirmation and submission module is configured to have the container instance write block data to the target temporary file. When a single block is flushed to disk and the verification is successful, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. The takeover module is configured to, when the lease expires and the heartbeat stops, acquire the takeover container instance of the distributed lock, read the block confirmation bitmap, the confirmation offset and the accompanying verification file, truncate or resume the transmission of the target end temporary file and output the file index after submission.

[0017] Compared with existing technologies, its advantages are as follows: This invention discloses a microservice-based container deployment method and system for a flight simulator data disaster recovery system. The monolithic disaster recovery application is decomposed into device access microservices, task scheduling microservices, transmission execution microservices, metadata microservices, and gateway microservices. Layered ready tokens are configured based on service dependency topology, enabling the basic services, core services, and gateway microservices to start in an orderly manner according to their dependencies. Downstream services receive or issue backup tasks after the basic service passes health checks. When the basic service fails, the core service can stop accepting new backup tasks and retain the state of backup tasks that have already obtained execution rights, thereby reducing state conflicts between service startup, fault recovery, and task issuance.

[0018] The source path, destination path, file size, task version, chunked acknowledgment bitmap, and acknowledgment offset of the backup task are written to the coordination cache. The transport execution microservice is designed as a stateless container image. If the transport execution container instance fails, the takeover container instance can restore the transport context based on the task status in the coordination cache. The chunked acknowledgment bitmap and acknowledgment offset together define the range of consecutive files that have been flushed to disk and verified, enabling the takeover process to locate the start point of the resume transfer and reduce duplicate transfers and error overwriting.

[0019] When a single block is flushed to disk and passes verification, the block confirmation bitmap and confirmation offset are updated to coordinate the transmission progress in the cache and establish an order constraint between the actual file written to disk on the target end. The confirmed state corresponds to the block that has been flushed to disk and passed verification. If a container instance fails during transmission, the takeover container instance can truncate the temporary file on the target end and continue writing from the recoverable offset, thereby reducing the risk that the cache progress will be ahead of the target file's disk state.

[0020] After file segmentation, an accompanying verification file is generated. After verifying the file size and overall hash of the temporary file on the target end, an atomic renaming is performed. The file index is committed using the task identifier and task version as unique keys. The target end file status, accompanying verification status, and metadata file index status can form a closed loop. When a failure occurs between verification, renaming, or index commit, the takeover container instance can still perform idempotent commits or state synchronization through the accompanying verification file and metadata microservice, reducing duplicate indexing and residual temporary files, and lowering the risk of verification-failed files entering the formal backup state.

[0021] Based on link quality, storage capacity, container instance load, and deadline, unschedulable instances are filtered and sorted. Backup task allocation can simultaneously consider cross-base links, target-end storage, container instance load, and task time constraints. Unschedulable instances are removed before task issuance, and the remaining instances are assigned execution positions according to their sorting scores, making task allocation during concurrent backups of multiple flight simulators more suitable for the current operating resource status.

[0022] When the lease expires and the heartbeat stops, the takeover container instance that acquires the distributed lock reads the block confirmation bitmap, confirmation offset, target temporary file size and confirmation size in the accompanying check file, and determines the recoverable offset to perform truncation or resumption. This can reduce the risk of multiple container instances repeatedly taking over the same task, and make the resumption based on both the coordination cache and the target file state. Attached Figure Description

[0023] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1This is a flowchart illustrating some specific embodiments of a microservice-based container deployment method for a flight simulator data disaster recovery system according to the present invention; Figure 2 This is a schematic diagram of the microservice-based container deployment system of a flight simulator data disaster recovery system according to some specific embodiments of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application are also intended to include the plural forms, unless the context clearly indicates otherwise, and "multiple" generally includes at least two.

[0026] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0027] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0028] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0029] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0030] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0031] Reference Figure 1 A microservice-based container deployment method for a flight simulator data disaster recovery system, comprising: S101 breaks down the monolithic disaster recovery application into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice, configures layered ready tokens based on service dependency topology, and releases downstream service startup permits after the basic service health probe passes. S102, generate a stateless container image for the microservice to be executed for transmission, and write the source path, target path, file size, task version, block confirmation bitmap and confirmation offset of the backup task to the coordination cache. S103, based on link quality, storage capacity, container instance load and deadline, filters out unschedulable instances and sorts them, and assigns backup tasks to the container instances that are ranked higher. S104, the container instance writes block data to the target temporary file. When a single block is flushed to disk and the verification is successful, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. S105, when the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset and the accompanying verification file, truncates or resumes the transmission of the target end temporary file and outputs the file index after submission.

[0032] To provide a clearer explanation, the steps in the embodiments of the present invention are described in detail below: S101 breaks down the monolithic disaster recovery application into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice, configures layered ready tokens based on service dependency topology, and releases downstream service startup permits after the basic service health probe passes. Furthermore, the service dependency topology is saved; The initialization container of the basic service performs health probes on metadata connection, device connection pool preheating and instance address registration, and writes the basic layer ready token after the probe passes. The core service listens to the base layer ready token and reads the base service instance address before starting. After starting, it writes the core layer ready token. The gateway microservice starts after reading the core layer ready token.

[0033] The basic services include device access microservices and metadata microservices; the downstream services include core services and gateway microservices; the core services include task scheduling microservices and transmission execution microservices; and the layered ready tokens include basic layer ready tokens and core layer ready tokens.

[0034] In this embodiment, the original monolithic disaster recovery application included a web front-end, back-end interface, metadata database, and file transfer functionality. During the split, the device access microservice is responsible for simulator connection, authentication, heartbeat detection, and connection pool maintenance; the task scheduling microservice is responsible for receiving, queuing, allocating, and monitoring the status of scheduled and manual backup tasks; the transmission execution microservice is responsible for moving backup files through a secure file transfer channel; the metadata microservice is responsible for device inventory, task logs, file indexes, and audit records; and the gateway microservice serves as a unified interface entry point, receiving front-end access requests and forwarding them to the corresponding microservices. Each microservice runs in a containerized manner, and their startup relationships are determined through service dependency topology.

[0035] In one specific implementation, the coordinator can store the service dependency topology. The coordinator can be implemented using a key-value store component, a custom resource in a container orchestration component, or a dedicated coordination service. The service dependency topology stored by the coordinator includes at least the service name, service level, direct dependent services, ready token name, instance address list, token write time, and token expiration date.

[0036] Before basic services start, the initialization container performs business-level health probes. Metadata connection probes include establishing database connections, writing test records, reading test records, and deleting test records; device connection pool warm-up includes reading simulator addresses, authentication methods, ports, and base station affiliations from the device list, and establishing control and data channels for each simulator; instance address registration includes writing service instance addresses, ports, protocols, and health status to the coordinator. After all probes pass, the initialization container writes a base layer ready token to the coordinator. This base layer ready token signifies that the basic service has the ability to access instance addresses and execute business operations.

[0037] Furthermore, the detection items include metadata connection, device connection pool preheating, and instance address registration; If any of the aforementioned probes fails, the failed probe and the failure time are recorded, and the probe is re-probeed according to a preset backoff interval. When the metadata connection returns to a writable state, the device connection pool completes the preheating of the control channel and data channel, and the instance address completes service registration, the base layer ready token is written. When the base layer ready token is revoked, the core service closes the task delivery channel and returns unstarted tasks to the scheduling queue.

[0038] The initialization container generates probe records for metadata connection, device connection pool preheating, and instance address registration. Probe records can include probe name, target address, return code, error type, start time, end time, and number of retries. The backoff interval can be set to... , , , Incremental, the maximum backoff interval can be set to If any probe fails, the initialization container remains in a pre-started state and the failed probe is written to the coordinator's startup diagnostic record. This startup diagnostic record is used by core services to identify the reasons why the infrastructure layer is not ready, and also to locate the corresponding fault point.

[0039] The metadata connection returning a writable status indicates that the metadata microservice is capable of performing atomic writes to the database; the device connection pool completing control and data channel warm-up indicates that the device access microservice has established command execution and file transfer channels for the simulator; and the instance address completing service registration indicates that the basic service can be discovered by the core service. After the base layer ready token is written, the core service reads the list of basic service instance addresses from the coordinator and performs connection verification. The task scheduling microservice establishes a task log writing connection after reading the metadata microservice instance address, and the transport execution microservice requests a data channel handle after reading the device access microservice instance address.

[0040] The revocation of a base layer ready token can be triggered by a failure of a base service health probe, all instance addresses becoming unreachable, a large-scale disconnection of the device connection pool, or a metadata connection becoming unwritable. After the core service closes the task distribution channel, backup tasks already in the execution state are retained in the coordination cache, while backup tasks that have not obtained execution rights are returned to the scheduling queue. The core service retains its process and local runtime context, and reads the new base service instance address and resumes scheduling after the base layer ready token is rewritten. This process ensures that unstarted tasks and executed tasks enter a recognizable state, reducing task state loss.

[0041] In this embodiment, the device accesses a dual-channel connection pool of a microservice maintenance simulator. Each simulator is independently configured with one control channel and one data channel. The control channel handles command execution, heartbeat detection, and disk status reading, while the data channel handles secure file transfer of backup files. During the connection establishment phase, the device accesses the microservice and asynchronously initializes the dual-channel connection based on the device list in the metadata microservice. If the first connection fails, the device status is marked as pending initialization, and reconnection is performed at exponential backoff intervals. If the control channel remains idle for more than [a certain period], [the connection is considered complete]. The control session will be closed after a certain number of minutes, and the data channel will remain idle for more than [number] minutes. The data session is closed after a certain number of minutes; the session is destroyed and the connection handle is released when the channel is abnormal.

[0042] Device access to microservices affects each control channel Send lightweight commands, which can be echo commands or disk space query commands; continuously If there is no response or a non-zero code is returned, the control channel will be marked as abnormal. The device access microservice performs data channel checks every time... Perform file status detection; continuous Upon timeout, the data channel is marked as abnormal. When both the control channel and data channel are abnormal, or when the control channel is abnormal and reconstruction fails, the device status is changed from healthy to isolated. During isolation, the device access microservice closes all connection sessions to the corresponding simulator, writes the device identifier to the isolated device set in the coordination cache, and publishes an isolation event through the device status channel. After subscribing to the isolation event, the task scheduling microservice removes the corresponding simulator from the list of allocatable devices. For ongoing transmission tasks, the transmission execution microservice releases the distributed lock and enters the interrupt queue; for tasks not yet started, the task scheduling microservice withdraws them and reassigns them to healthy devices.

[0043] In this embodiment, connection pools can also be managed at the base level. The device access microservice divides simulators from the same base into base device groups. Devices within the same base device group share the local network heartbeat threshold, while devices across base device groups use longer timeout tolerance parameters.

[0044] When the device recovers, the device access microservice attempts to rebuild the control and data channels at exponential backoff intervals. The control channel remains continuous. A total of 1 cycle Detection normal, data channel continuous A total of 1 cycle If the detection is normal, and the disk space, authentication fingerprint, and device identifier match the records before isolation, the device status is restored to healthy, the device identifier is removed from the isolated device set, and a recovery notification is issued. In concurrent scenarios with one or more simulators, devices accessing microservices can use a single coroutine to poll the control channel in batches, generate heartbeat detection records in batches, and distribute the detection results to the corresponding device status processing flow.

[0045] After saving the service dependency topology, the following is also included: When a failure of a basic service is detected, a dependency interruption signal is sent to the core services that depend on the basic service. After receiving the dependency interruption signal, the core service stops accepting new backup tasks and closes the task delivery channel, while retaining the backup tasks that have already obtained execution rights in the coordination cache. After the basic service rewrites the base layer ready token, it notifies the core service to resume task processing based on the service dependency topology, and notifies the gateway microservice to resume interface forwarding.

[0046] The coordinator can perform reverse cascading control based on service dependency topology. When a basic service fails, the coordinator reads the list of core services that depend on the basic service and sends a dependency interruption signal to the task scheduling microservice and the transport execution microservice. Upon receiving the dependency interruption signal, the core service stops accepting new backup tasks from external entry points and stops issuing new task execution instructions to the transport execution microservice. Backup tasks that have already obtained execution rights are retained in the coordination cache, which continues to store the task identifier, task version, chunk acknowledgment bitmap, acknowledgment offset, current execution container instance, and lease expiration time. After the basic service re-registers and writes the basic layer ready token, the coordinator verifies the basic layer ready token, notifies the core service to restore service discovery and task processing, and notifies the gateway microservice to restore interface forwarding.

[0047] S102, generate a stateless container image for the microservice to be executed for transmission, and write the source path, target path, file size, task version, block confirmation bitmap and confirmation offset of the backup task to the coordination cache. Furthermore, the source path, target path, file size, task version, current execution container instance, and lease expiration time are recorded using the task identifier; A block confirmation bitmap and a distributed lock are established based on the task identifier, and the verification value of each block is recorded. The block confirmation bitmap marks the blocks that have been flushed to disk and passed the verification based on the block sequence number. The confirmation offset is updated with the end position of the consecutive confirmed blocks.

[0048] In this embodiment, the container image for transmitting the microservice adopts a stateless design. The container instance does not store the unique state that can be used as the basis for task recovery locally. The execution status of the backup task is written to the coordination cache, the file write-to-disk status is written to the target temporary file and the accompanying verification file, and the file index commit status is written to the metadata microservice.

[0049] The transport execution microservice container image can be generated through a multi-stage build process. The build stage installs secure file transfer tools, breakpoint resume tools, and verification tools. The runtime stage retains the commands, certificates, scripts, and runtime dependencies required for transport execution. The device access microservice image can use a lightweight operating system image with an integrated secure command client. The transport execution microservice image can integrate a transport toolchain that supports secure replication, synchronous transport, and compatibility with legacy devices. The metadata microservice can use a database base image with added initialization scripts. The image testing phase verifies command availability, certificate mounting, target path access, temporary file writing, and command execution verification. After passing the tests, the image is pushed to the image repository for the container orchestration system to pull.

[0050] Upon receiving a backup task, the task scheduling microservice generates a task identifier. The task identifier can be a combination of the simulator identifier, backup plan identifier, file path summary, and task creation time. In one specific implementation, a main cache record can be established using the task identifier. This main cache record stores the source path, target path, file size, task version, current execution container instance, and lease expiration time.

[0051] The main cache record can be stored using a key-value structure, for example, a main hash record can be created using the task identifier as part of the key. The source path is the path to the file to be backed up on the simulator side, and the target path is the directory and filename of the target disaster recovery endpoint. The file size is read by the device access microservice through the control channel. The task version is set when the task is first created. The cache size increments after the container instance acquires new execution rights. The currently executing container instance records the container instance identifier and its location, while the lease expiration time records the effective expiration time of the current execution rights. The main cache record can be read jointly by the transport execution microservice, the task scheduling microservice, and the coordinator for task allocation, execution, lease renewal, and takeover.

[0052] The coordinated cache can establish a block confirmation bitmap, a block verification record, and a distributed lock record for the same task identifier. The block verification record is used to record the verification value of each block, and the distributed lock record is used to save the distributed lock state.

[0053] The block confirmation bitmap stores the confirmation status of disk flushing and successful verification according to the block sequence number. The block size is preferably... The total number of blocks is obtained by dividing the file size by the block size and rounding up. The size of the last block is determined by the number of bytes remaining at the end of the file.

[0054] The block checksum record stores the cyclic redundancy check value for each block. The checksum can be calculated by the source end after reading the block data, or it can be calculated synchronously by the transport execution microservice when reading the source data stream and written to the coordination cache. The distributed lock record stores the lock holding container instance, lock acquisition time, lease expiration time, and task version.

[0055] The confirmation offset represents the maximum byte offset from the beginning of the file that has been continuously flushed to disk and verified. The confirmation offset is updated according to the end positions of consecutively confirmed blocks in the block confirmation bitmap. When the... Block to the first When all blocks have been confirmed, the confirmation offset is the th Block end byte position; when the first If a block is not confirmed, and subsequent blocks have independent confirmation states, the confirmation offset remains at the level of the first block. End of block.

[0056] In this embodiment, the source path, target path, file size, task version, current execution container instance, lease expiration time, block confirmation bitmap, block verification record, distributed lock record, and confirmation offset constitute the data loop required for task recovery. After the transmission execution microservice container instance restarts or migrates, the transmission start point, execution right ownership, and target file processing method can be determined by reading the data through the task identifier. The coordination cache can be implemented using a cache component that supports atomic writes and bitmap operations. The main cache record is used to read task fields, the block confirmation bitmap is used to locate unconfirmed blocks, and the distributed lock record is used to restrict the container instances that obtain the corresponding task write right at the same time.

[0057] In this embodiment, a transport abstraction layer can also be set up for emulators using traditional secure replication protocols. The transport abstraction layer exposes random location write, chunked acknowledgment, and breakpoint resume interfaces to both the task scheduling microservice and the transport execution microservice. For devices supporting secure file transfer protocols, the transport abstraction layer uses a location write command to continue writing from the acknowledgment offset; for devices using traditional secure replication protocols, the transport abstraction layer invokes in-situ synchronization capabilities to synchronize the source file from the acknowledgment offset to the target temporary file. The differences in actual commands generated by different transport tools are encapsulated in the transport abstraction layer, ensuring consistency between the task field and the chunked acknowledgment bitmap in the coordination cache.

[0058] S103, based on link quality, storage capacity, container instance load and deadline, filters out unschedulable instances and sorts them, and assigns backup tasks to the container instances that are ranked higher. The link quality includes available bandwidth, round-trip latency, and packet loss rate of the cross-base link; the storage capacity includes the remaining capacity of the target end and disk write capability; the container instance load includes the number of concurrent tasks of the container instance; the deadline is the deadline of the backup task; and the historical transmission performance includes the historical average transmission rate and successful records of the container instance to the same simulator or the same model.

[0059] Furthermore, the available bandwidth of the cross-base link, the round-trip latency, the packet loss rate, the remaining capacity of the target end, the disk write capability, the number of concurrent tasks of the container instance, the base to which the source simulator belongs, the deadline, and the historical transmission performance are read. Remove container instances that do not meet the storage reservation conditions, concurrency limit conditions, link availability conditions, or connection pool warm-up conditions; The link margin is determined based on the link quality, the urgency of the deadline is determined based on the deadline, and the affinity between the source simulator and the container instance is determined based on their respective bases. For the remaining container instances, a ranking score is calculated based on the remaining link capacity, the remaining storage capacity, the same-base affinity, the urgency of the deadline, the historical transmission performance, and the container instance load. Backup tasks are then assigned to container instances with higher ranking scores.

[0060] The task scheduling microservice periodically reads resource status. Link quality can also include link status; storage capacity can also be combined with disk health status; container instance load can also be combined with processor usage, memory usage, and connection pool saturation; task information can also include source file size and task priority; historical information includes the historical average transmission rate and successful records of the transmission execution microservice container instance to the same simulator or the same machine model.

[0061] When deploying across bases, each base can deploy a complete microservice cluster. The main base runs a global task scheduling microservice, a global metadata replica set master node, and a certificate management center. The bases run access microservices, transmission execution microservices, local metadata replicas, and local gateway microservices.

[0062] In multi-base scenarios, base gateway proxies can be deployed at the base edge. Internally, these proxies aggregate cross-base control traffic from microservices within their own base; externally, they communicate with other base gateway proxies via secure tunnels with two-way certificate authentication. The control plane link carries scheduling commands, heartbeats, state synchronization, metadata queries, and small-volume business data; the data plane link carries backup file transfers, with file movement executed via point-to-point encrypted channels established between microservices.

[0063] Certificates issued by the Certificate Management Center include a base identifier and a network segment declaration. The base gateway agent verifies the identity of the peer base based on the certificate content. Task-level session keys can be generated from the task identifier and the base session key using a key derivation function; each backup task uses an independent session key. During certificate updates, an atomic switch is performed after the new certificate has warmed up, and the old certificate is retained. The 24-hour grace period allows ongoing transmission tasks to continue or a resume channel to be established based on a new certificate.

[0064] The control plane and data plane links are separated, allowing microservice governance traffic and large file transfer traffic to be transmitted through their respective links. The cross-base link status collected by the base gateway proxy can be written to the coordination cache and participate in the resource-aware scheduling of task scheduling microservices.

[0065] Task scheduling microservices implement hard constraint filtering. Storage reservation conditions can be set to the remaining capacity on the target side. Source file size The concurrency limit can be set to the number of concurrent tasks executed by a single transport microservice container instance. Link availability conditions may include the link status not being marked as degraded and round-trip latency. And packet loss rate The connection pool preheating condition indicates that the control and data channels of the source simulator have been established by the device access microservice and are in a healthy state. Processor usage... Or memory usage Container instances can be removed as unschedulable instances. For simulators already marked as isolated, the task scheduling microservice stops assigning new backup tasks to the corresponding simulators; for tasks that have been issued but not yet started, the task scheduling microservice withdraws the tasks and re-selects schedulable instances.

[0066] The ranking score can be calculated according to the following relationship: Ranking score = 0.25 × Link margin score + 0.20 × Storage margin score + 0.20 × Co-location affinity score + × Deadline pressure score + 0.10 × Historical performance score - × Load penalty points; The link margin score is determined based on the available bandwidth, round-trip latency, and packet loss rate of the cross-base link; the storage margin score is determined based on the remaining capacity and disk write capability of the target end; the same-base affinity score is determined based on the base to which the source simulator belongs and the base where the container instance is located; the deadline urgency score is determined based on the remaining time of the backup task and the estimated transmission time; the historical performance score is determined based on the historical average transmission rate and successful records of the container instance to the same simulator or the same model; and the load penalty score is determined based on the number of concurrent tasks of the container instance, and can be adjusted in conjunction with processor usage and memory usage. Weighting is assigned to those with tight deadlines; This indicates the weight of the load penalty.

[0067] Link margin score can be calculated by combining the ratio of available bandwidth to rated bandwidth with round-trip latency and packet loss rate; storage margin score can be calculated by the percentage of remaining capacity after deducting the space required for backup tasks and disk write capacity; when the source simulator and target container instance are located in the same base, the affinity score for the same base is taken as the higher value, and when they are in different bases, it can be deducted based on round-trip latency, for example, for every increase of Deduction The score for tight deadlines can be determined by the ratio of remaining time to estimated transmission time, which can be determined by the source file size, available bandwidth, and connection establishment time; the score for historical performance is calculated from the historical average transmission rate and historical successful records.

[0068] Can be taken in regular tasks When the remaining time is less than the expected transmission time When doubled, it can be taken ; Can be taken in regular tasks When the source file Time can be taken When multiple container instances have the same sort score, the sort can be determined by comparing the current number of concurrent tasks, historical successful transmission records, and the most recent heartbeat latency.

[0069] In this embodiment, the base gateway agent can be each Link-state records are written to the coordination cache. These records include source base identifier, destination base identifier, available bandwidth, round-trip time, packet loss rate, link status, and update time. When the round-trip time... Or packet loss rate At this time, the base gateway agent writes the link status as degraded. After reading the link status record, the task scheduling microservice suspends the allocation of new tasks to the degraded link. Tasks that have been issued and are in an unstarted state are rescheduled to the same base instance or a backup cross-base link. Tasks in execution enter the takeover process, and the new transport execution microservice container instance resumes the transmission based on the acknowledgment offset.

[0070] In this embodiment, a three-level response can also be implemented to address resource drift during execution. Each task scheduling microservice... Poll the resource status. When the available bandwidth of a container instance decreases... Or the remaining capacity on the target device is lower than the task requirements. When the transmission rate is doubled, a rate-limiting command is issued to the microservice executing the transmission to reduce the number of concurrent chunks and the transmission rate; continuous monitoring is performed. If the resource status has not yet been restored, the transport execution microservice releases the distributed lock and writes the task status as interrupted. The task scheduling microservice recalculates the sorting score and selects a new container instance. When the backup deadline is less than the expected completion time, the task scheduling microservice reallocates the backup task according to the sorting results of the currently schedulable instances.

[0071] S104, the container instance writes block data to the target temporary file. When a single block is flushed to disk and the verification is successful, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. Furthermore, the chunked data is written to a temporary file with a temporary suffix on the target end via a secure file transfer channel; Forced flushing and cyclic redundancy check are performed on individual blocks that have been written. When the cyclic redundancy check result is consistent with the source block check value, the bit corresponding to the block number in the block confirmation bitmap is marked as confirmed, and the confirmation offset is updated to the maximum end position of the consecutive confirmed blocks. When all blocks are in the confirmed state, generate an accompanying verification file containing a block verification list, file size, overall hash, task identifier, and task version; Read the accompanying verification file to verify the target end temporary file; After verification, the target temporary file is renamed to a backup file. The task identifier and task version are used as unique keys to write the file index to the metadata microservice in an idempotent manner. After a successful write, the task status in the coordination cache is advanced to "committed".

[0072] The container instance that obtains execution rights can read the source path, target path, file size, task version, and acknowledgment offset from the main cache record, and determine the chunks to be transferred based on the acknowledgment offset. The filename of the target temporary file can consist of the target filename, task identifier, and task version, with a temporary suffix. The container instance requests a data channel from the source simulator through the device access microservice and establishes a target write channel using the target authentication information. During writing, the container instance reads data from the source path according to the chunk size and writes it to the corresponding offset position of the target temporary file according to the chunk sequence number.

[0073] When the source file size is The block size is At that time, according to calculate, correspond In total indivual Segmentation.

[0074] After a container instance completes writing to a single block, it performs a forced flush to the target's temporary file, ensuring the block data is stored persistently on the target. Once flushed, the container instance reads the written blocks or receives the block verification results from the target and calculates the cyclic redundancy check value. The source-side block verification value can be calculated synchronously by the container instance when reading the source-side blocks before transmission, or it can be pre-written into the block verification record by a file scanning task on the source emulator side.

[0075] The container instance compares the target block checksum with the source block checksum. If they match, the single block enters the confirmed state; if they don't match, the container instance rereads the source block and overwrites the corresponding offset in the target temporary file. After overwriting, it performs a disk flush and cyclic redundancy check. The number of retries can be set to... After the number of retries is exceeded, the distributed lock is released and the backup task is written to the interrupt queue.

[0076] The block confirmation bitmap is updated after a single block is flushed to disk and verified. The container instance sets the bit corresponding to the block sequence number to "confirmed" and updates the confirmation offset according to the maximum end position of consecutive confirmed blocks. After updating the confirmation offset, the container instance also refreshes the checkpoint time. The checkpoint time can be used as an auxiliary field for heartbeat stall detection. The container instance updates the checkpoint time after completing each block and... For distributed lock renewal, the lock lease can be set to... When the lock has expired and the checkpoint time has exceeded... If not updated, the task is moved to the interrupt queue. This state update relationship ensures that the acknowledgment progress in the coordination cache corresponds to the block state that has completed disk flushing and verification.

[0077] In this embodiment, a temporary accompanying record can also be synchronously written to the directory where the temporary file is located on the target end. The temporary accompanying record may include the current task identifier, task version, number of confirmed chunks, confirmation offset, and the sequence number of the most recently verified chunk. The temporary accompanying record is written to the target end directory as the chunk confirmation bitmap is updated, and is used to cross-verify the target end file status during coordination cache failure or takeover processes. The temporary accompanying record is used to record the recovery status during transmission. The accompanying verification file is generated after all chunks are completed and is used to carry the fields required for overall submission verification.

[0078] In this embodiment, for cross-site large file transfers, the container instance can set the number of concurrent blocks based on the link status issued by the task scheduling microservice. When the link bandwidth is sufficient and the packet loss rate is low, the number of concurrent blocks can be set to... When the link status degrades or the target end's write capability decreases, the number of concurrent blocks decreases to [a certain value]. or During concurrent writes, each block still independently performs disk flushing and cyclic redundancy check according to its block number to confirm that the offset has been pushed to the end of the consecutive confirmed blocks. For example, the first... Block, First Block and the first Block confirmed, number When the block is in an unacknowledged state, the acknowledgement offset remains at the [number]th [position]. At the end of the block, the first The confirmed status of a block can be stored in the block confirmation bitmap, and the recovery starting point is determined by the continuous confirmation interval during takeover.

[0079] The container instance scans the block confirmation bitmap to confirm that all blocks from the first to the last are in a confirmed state. The container instance calculates the overall hash of the target temporary file, preferably using a secure hash algorithm. The accompanying verification file can be stored in structured text or binary format, containing the task identifier, task version, source path, target path, file size, block size, total number of blocks, block verification list, overall hash, confirmation size, and generation time. The confirmation size is the file length corresponding to all consecutive confirmed blocks. The accompanying verification file and the target temporary file can reside in the same directory and share the same task identifier and task version name.

[0080] The container instance reads the accompanying verification file, compares the confirmed size in the accompanying verification file with the actual size of the target temporary file, and recalculates the overall hash of the target temporary file. It then compares the calculated overall hash with the overall hash in the accompanying verification file. If the file sizes are inconsistent, the container instance rereads the block confirmation bitmap and the target temporary file size, locates the differing blocks, and rolls back to the continuous confirmation offset. If the overall hash is inconsistent, the container instance locates the inconsistent blocks according to the block verification list and rewrites the corresponding blocks. After rewriting, the container instance performs forced disk flushing, block verification, and overall hash verification. When the target temporary file is missing and the coordination cache shows all blocks have been confirmed, the container instance changes the task status to interrupted and enters the recovery process.

[0081] After successful verification, the container instance atomically renames the target temporary file to a backup file. Before executing the atomic renaming, the container instance checks if a formal backup file with the same name already exists in the target path. The target filename always includes the task version, which monotonically increments to distinguish different execution versions of the same task.

[0082] If a formal backup file with the same name already exists, the container instance reads the file index record from the metadata microservice. If the task identifier, task version, file size, and overall hash in the file index record match the accompanying verification file, a coordination cache state synchronization is performed; if the fields are inconsistent, the container instance writes the current task to the conflict queue and stops committing, waiting for rescheduling to generate a new task version.

[0083] The metadata microservice establishes a unique index based on the task identifier and task version. When a container instance submits a file index, the index content includes the task identifier, task version, source path, target path, backup file name, file size, overall hash, chunk size, chunk checksum summary, submitting container instance, submission time, and task status.

[0084] The metadata microservice performs atomic writes: inserting a file index record if the unique index does not exist; returning idempotency success if the unique index already exists and the fields match; and returning a conflict if the unique index already exists and the fields do not match. After the metadata microservice returns a write success or idempotency success, the container instance coordinates the progress of the task status in the cache to committed.

[0085] When a container instance fails after an atomic renaming of a temporary file on the target end but before the coordination cache state is advanced, the takeover container instance queries the metadata microservice. If the file index record already exists and the fields are consistent, the takeover container instance's synchronization coordination cache task status is set to committed; if the file index record does not exist, the takeover container instance reads the accompanying verification file and backup file to perform verification, and after successful verification, writes the file index idempotently to the metadata microservice.

[0086] In this embodiment, the orphan detection task can also periodically scan for tasks that have not been submitted for a long time. The orphan detection task reads task records in the coordination cache that are in transit, verification, or submission, and checks whether there is an active holder of the distributed lock. For tasks without an active distributed lock and exceeding a preset waiting time, the orphan detection task reads the target-side temporary file and accompanying verification file in the target-side directory. If the accompanying verification file exists and the size of the target-side temporary file matches the confirmation size, the orphan detection task moves the task to the interrupt queue for the takeover container instance to continue processing; if the target-side temporary file and accompanying verification file have not been updated for a long time, the orphan detection task deletes the remaining target-side temporary file and accompanying verification file, and resets the backup task to a pending scheduling state. The orphan detection task can periodically... Execute once every minute; the threshold for long periods without submission can be set to... minute.

[0087] S105, when the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset and the accompanying verification file, truncates or resumes the transmission of the target end temporary file and outputs the file index after submission.

[0088] Furthermore, backup tasks with expired leases and stopped heartbeats are moved to the interrupt queue; The waiting container instances compete for the distributed lock through atomic writes; The takeover container instance of the distributed lock reads the block confirmation bitmap, the confirmation offset, the size of the target temporary file, and the confirmation size in the accompanying verification file, determines the minimum recoverable offset, truncates the target temporary file to the minimum recoverable offset, and continues to write block data from the minimum recoverable offset.

[0089] The coordinator can periodically scan the main cache records in the coordinating cache to read the currently executing container instance, lease expiration time, checkpoint time, and task status. If the current time exceeds the lease expiration time and the checkpoint time exceeds [a certain threshold], [the task will fail]. When not updated, the coordinator writes the task status as interrupted and adds it to the interrupt queue. If the current time exceeds the lease expiration time and the checkpoint time is still updating, the coordinator can postpone the judgment for one scan cycle to reduce false takeovers caused by network latency. The tasks in the interrupt queue retain the source path, destination path, file size, task version, chunk acknowledgment bitmap, acknowledgment offset, and chunk checksum record.

[0090] Multiple standby container instances can listen to the interrupt queue. Standby container instances compete for the distributed lock using atomic writes, which can be implemented by writing to a non-existent key. The takeover container instance that acquires the distributed lock increments the task version in the main cache record and writes the new currently executing container instance, lease expiration time, and checkpoint time; standby container instances that do not acquire the distributed lock exit the takeover and continue listening for other tasks.

[0091] After acquiring the distributed lock, the takeover container instance can query whether the metadata microservice already has a committed file index corresponding to the task identifier and task version. If a committed file index with consistent fields exists, the takeover container instance synchronizes and coordinates the cache state and outputs that file index; if no corresponding committed file index is found, it continues to execute subsequent steps.

[0092] The takeover container instance reads the block acknowledgment bitmap to locate the offset of the largest consecutive acknowledgment block end, reads the acknowledgment offset and the actual size of the target temporary file from the main cache record, and reads the acknowledgment size from the accompanying check file. If the accompanying check file has not yet been generated, the acknowledgment size can be determined from the temporary accompanying record or the offset of the largest consecutive acknowledgment block end in the block acknowledgment bitmap.

[0093] The takeover container instance uses the minimum of the following as the minimum recoverable offset: the acknowledgment offset, the actual size of the target temporary file, the acknowledgment size of the accompanying checksum file, and the maximum consecutive acknowledgment block end offset. If the actual size of the target temporary file is greater than the minimum recoverable offset, the takeover container instance truncates the target temporary file to the minimum recoverable offset. If the actual size of the target temporary file is less than the acknowledgment offset, the takeover container instance rolls back the acknowledgment offset to the block boundary corresponding to the actual size of the target temporary file and clears the acknowledgment markers above the rolled-back block boundary in the block acknowledgment bitmap. After truncation or rollback, the takeover container instance establishes a transmission channel from the minimum recoverable offset and continues to perform block writing, disk flushing, checksum verification, block acknowledgment, and file commit.

[0094] Furthermore, the block check list and confirmation size in the accompanying check file are read, and the confirmation size is compared with the maximum consecutive block end offset indicated by the block confirmation bitmap; When the size of the target temporary file is greater than the confirmation size or greater than the maximum continuous block end offset, the target temporary file is truncated to the smaller of the confirmation size and the maximum continuous block end offset; When the block confirmation bitmap is lost, delete the target temporary file and move the backup task back into the task scheduling queue.

[0095] The takeover container instance reads the accompanying verification file and parses the block verification list. The block verification list stores verification values ​​according to the block sequence number, and the confirmation size represents the length of the continuously verified file when the accompanying verification file was generated. When the confirmation size is less than the maximum consecutive block end offset, the takeover container instance uses the confirmation size as the candidate recovery offset; when the confirmation size is greater than the maximum consecutive block end offset, the takeover container instance uses the maximum consecutive block end offset as the candidate recovery offset. The candidate recovery offset and the actual size of the target temporary file are used together to determine the minimum recoverable offset.

[0096] If the target temporary file size is greater than the acknowledgment size or greater than the maximum contiguous block end offset, the tail of the target temporary file contains data exceeding the acknowledgment range shared by the reconciliation cache and the accompanying checksum file. The takeover container instance truncates the target temporary file at the smaller of the acknowledgment size and the maximum contiguous block end offset. After truncation, the takeover container instance rereads the target temporary file size and calculates the cyclic redundancy check (CRC) value of the last complete block. If the CRC value of the last complete block matches the block checksum list, the takeover container instance continues transmission from the next block; if the checks do not match, the takeover container instance backtracks one block and repeats the checksum until a contiguous block boundary with matching checks is determined.

[0097] When the block verification bitmap is lost, the takeover container instance cannot confirm which blocks in the target temporary file have been flushed to disk and passed verification. The takeover container instance deletes the target temporary file and temporary accompanying record, sets the task status to pending scheduling, and releases the distributed lock. The task scheduling microservice re-filters and sorts the schedulable container instances, selecting an available container instance to retransmit from the source path. If the accompanying verification file already exists and the overall hash verification passes, the takeover container instance can query the metadata microservice and execute an idempotent file index commit; if it cannot confirm the validity of the official backup file through the accompanying verification file and the metadata microservice, it performs temporary file cleanup and rescheduling.

[0098] In this embodiment, intermediate result records and runtime verification interfaces can also be exposed. Each time the transmission execution microservice acquires the distributed lock, completes block flushing, updates the block confirmation bitmap, generates the accompanying verification file, performs atomic renaming, and commits the file index, it outputs the task identifier, task version, container instance identifier, block sequence number, confirmation offset, target temporary file size, accompanying verification file confirmation size, distributed lock lease expiration time, and metadata commit return code to the runtime log. The runtime log can be written to the audit log set of the metadata microservice or queried through the read-only debug interface. The state cache read interface can return the main cache record, block confirmation bitmap digest, confirmation offset, and task status according to the task identifier, for implementation of detection and fault review. The runtime log and state cache read interface are independent of the backup task execution flow and are used to verify the data consistency between intermediate results and the final file index.

[0099] The following description, using specific examples, elaborates on this embodiment.

[0100] For example, the simulator SIM01 in base A needs to be backed up from the source path / data / flight / sim01.bin. Flight data files, target path is the disaster recovery storage directory of base A, backup deadline is after mission creation. The system contains three transport execution container instances, A1, A2, and B1, with A1 located at the same base as SIM01. The current number of concurrent tasks is [number missing]. Available bandwidth is The remaining capacity at the target end is A2 and SIM01 are located at the same base, and the current number of concurrent tasks is... Available bandwidth is The remaining capacity at the target end is B1 is located at base B, and the current number of concurrent tasks is [number missing]. The available bandwidth for cross-base links is The remaining capacity at the target end is Round-trip delay is .

[0101] The coordinator stores the service dependency topology between the device access microservice, task scheduling microservice, transport execution microservice, metadata microservice, and gateway microservice. The metadata microservice connection returns to a writable state. The device access microservice completes the preheating of the SIM01 control and data channels, and writes the instance address to the base layer ready token after service registration. The task scheduling and transport execution microservices read the base layer ready token and start, writing it to the core layer ready token after startup. The gateway microservice reads the core layer ready token and opens the backup task entry point. The device access microservice performs service checks on each SIM01 microservice. Perform control channel probing, and probe each data channel. Perform a file status probe and keep SIM01 in a healthy state.

[0102] The task scheduling microservice generates a task identifier T-SIM01-001 for the backup task of SIM01. The main cache records the source path, destination path, and file size. Task Version The currently executing container instance and its lease expiration time. calculate, correspond The block size is Formed in time Each block. Coordination cache establishment. A block confirmation bitmap is created and a block verification record is established to store the cyclic redundancy check value of each block, while a distributed lock record is also established. The initial confirmation offset is... .

[0103] The task scheduling microservice executes hard constraint filtering. The remaining capacity of the target end of A2 is... , Multiply get A2 meets the storage reservation requirements. The current number of concurrent tasks on A2 is close to the concurrency limit, resulting in a high load penalty score. B1 meets both the storage reservation and concurrency limit requirements, with a cross-base round-trip latency of [missing information]. According to each increase Deduction Example settings for points, corresponding to Deduction based on latency. A1 meets the storage reservation condition, concurrency limit condition, link availability condition, and connection pool warm-up condition. When using the sorted score calculation relationship, A1 receives a same-base affinity score, a higher storage margin score, and a lower load penalty score; B1 receives a higher link margin score and storage margin score, but also has a deduction corresponding to cross-base round-trip latency; A2 has a lower score due to available bandwidth and current load. The task scheduling microservice writes A1 into the backup task execution record and assigns the backup task to A1.

[0104] A1 acquires the distributed lock, writes the currently executing container instance to the main cache record, and sets... Lock lease. A1 reads the block sequence number from the data channel of SIM01. corresponding The data is divided into chunks and written to the target's temporary file sim01.bin.T-SIM01-001.v1.part. Offset position. After writing is complete, A1 performs a forced flush and calculates the target end's block cyclic redundancy check value. When the target end's block cyclic redundancy check value matches the source end's block check value, A1 will assign the block sequence number to the block confirmation bitmap. The corresponding bit flag is confirmed; update the confirmation offset to... And refresh the checkpoint time.

[0105] A1 continues transmitting the remaining blocks according to their sequence numbers. Block sequence numbers When data is written to the target temporary file but has not yet been flushed to disk and verified, and the node where A1 is located fails, the block sequence number in the block confirmation bitmap will be affected. to The block is in the confirmed state, and the number of consecutive confirmed blocks is [number missing]. One, confirmed offset is Block number Remain in an unconfirmed state.

[0106] The coordinator detected that the A1 lock lease had expired and the checkpoint time had exceeded the limit. Not updated; move task T-SIM01-001 to the interrupt queue. Standby container instance B1 acquires the distributed lock via atomic write and updates the task version to... It reads the block confirmation bitmap, confirmation offset, target temporary file size, and available accompanying verification information. When the actual size of the target temporary file is... The maximum consecutive block end offset indicated by the block confirmation bitmap is At that time, B1 truncates the target's temporary file to... And continue writing block data from the offset.

[0107] B1 continues to perform write, flush, cyclic redundancy check, block acknowledgment bitmap update, and acknowledgment offset update on the remaining blocks until... Each block has been confirmed as complete. B1 generates a companion verification file sim01.bin.T-SIM01-001.v2.meta, which includes the task identifier T-SIM01-001 and the task version. File size Block size Total number of blocks Block checklist, overall hash and confirmation size .

[0108] B1 reads the accompanying verification file and verifies the file size and overall hash of the target temporary file. After successful verification, B1 atomically renames the target temporary file to the backup file sim01.bin.T-SIM01-001.v2. B1 then uses the task identifier T-SIM01-001 and the task version... Submit the file index to the metadata microservice using a unique key. After the metadata microservice write is successful, B1 advances the task status in the coordination cache to "committed" and outputs the file index containing the task identifier, task version, backup file name, file size, overall hash, commit container instance, and commit time.

[0109] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0110] like Figure 2 As shown, the present invention also provides a microservice-based container deployment system for a flight simulator data disaster recovery system, comprising: The service splitting and gating module 201 is configured to split the monolithic disaster recovery application into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice, configure layered ready tokens based on service dependency topology, and release downstream service startup permits after the basic service health probe passes. The image and cache module 202 is configured to generate stateless container images for microservices that are executed for transport, and to write the source path, target path, file size, task version, block confirmation bitmap and confirmation offset of the backup task to the coordination cache. Resource scheduling module 203 is configured to filter out unschedulable instances and sort them based on link quality, storage capacity, container instance load and deadline, and assign backup tasks to the container instances that are ranked higher. The transmission confirmation and submission module 204 is configured to write block data to the target temporary file by the container instance. When a single block is flushed to disk and the verification is passed, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. The takeover module 205 is configured to, when the lease expires and the heartbeat stops, acquire the takeover container instance of the distributed lock, read the block confirmation bitmap, the confirmation offset and the accompanying verification file, truncate or resume the transmission of the target end temporary file and output the file index after submission.

[0111] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described separately according to their functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A microservice-based container deployment method for a flight simulator data disaster recovery system, characterized in that, include: The monolithic disaster recovery application is split into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice. Layered ready tokens are configured based on service dependency topology, and downstream service startup permits are released after the basic service health probe passes. To generate stateless container images for microservices to be executed during transport, the source path, target path, file size, task version, chunk confirmation bitmap, and confirmation offset of the backup task are written to the coordination cache. Based on link quality, storage capacity, container instance load, and deadline, unschedulable instances are filtered out and sorted, and backup tasks are assigned to the container instances that are ranked higher. The container instance writes block data to the target temporary file. When a single block is flushed to disk and passes verification, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. When the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset, and the accompanying verification file, truncates or resumes the transmission of the target temporary file, and outputs the file index after submission.

2. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 1, characterized in that, Based on the service dependency topology configuration, a layered ready token is generated, and downstream service startup permits are released after the basic service health probe passes. This further includes: The basic services include device access microservices and metadata microservices; the downstream services include core services and gateway microservices; the core services include task scheduling microservices and transmission execution microservices; and the layered ready tokens include basic layer ready tokens and core layer ready tokens. Save the service dependency topology; The initialization container of the basic service performs health probes on metadata connection, device connection pool preheating and instance address registration, and writes the basic layer ready token after the probe passes. The core service listens to the base layer ready token and reads the base service instance address before starting. After starting, it writes the core layer ready token. The gateway microservice starts after reading the core layer ready token.

3. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 2, characterized in that, After saving the service dependency topology, the following is also included: When a failure of a basic service is detected, a dependency interruption signal is sent to the core services that depend on the basic service. After receiving the dependency interruption signal, the core service stops accepting new backup tasks and closes the task delivery channel, while retaining the backup tasks that have already obtained execution rights in the coordination cache. After the basic service rewrites the base layer ready token, it notifies the core service to resume task processing based on the service dependency topology, and notifies the gateway microservice to resume interface forwarding.

4. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 1, characterized in that, Write the source path, destination path, file size, task version, chunk acknowledgment bitmap, and acknowledgment offset of the backup task to the reconciliation cache, further including: The task identifier records the source path, target path, file size, task version, currently executing container instance, and lease expiration time. A block confirmation bitmap and a distributed lock are established based on the task identifier, and the verification value of each block is recorded. The block confirmation bitmap marks the blocks that have been flushed to disk and passed the verification based on the block sequence number. The confirmation offset is updated with the end position of the consecutive confirmed blocks.

5. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 4, characterized in that, The container instance writes chunked data to a temporary file on the target end. When a single chunk is flushed to disk and passes verification, the chunk confirmation bitmap and the confirmation offset are updated, further including: The container instance writes chunked data to a target temporary file with a temporary suffix via a secure file transfer channel. Forced flushing and cyclic redundancy check are performed on individual blocks that have been written. When the cyclic redundancy check result is consistent with the source block check value, the bit corresponding to the block number in the block confirmation bitmap is marked as confirmed, and the confirmation offset is updated to the maximum end position of the consecutive confirmed blocks.

6. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 5, characterized in that, After file segmentation is completed, an accompanying verification file is generated. This file verifies the file size and overall hash of the target temporary file, then atomically renames it to a backup file. The file index is then submitted using the task identifier and task version as unique keys. Further steps include: When all blocks are in the confirmed state, generate an accompanying verification file containing a block verification list, file size, overall hash, task identifier, and task version; Read the accompanying verification file to verify the target end temporary file; After verification, the target temporary file is renamed to a backup file. The task identifier and task version are used as unique keys to write the file index to the metadata microservice in an idempotent manner. After a successful write, the task status in the coordination cache is advanced to "committed".

7. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 6, characterized in that, When the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset, and the accompanying verification file, truncates or resumes the target-end temporary file, and outputs the committed file index, further including: Move backup tasks with expired leases and stopped heartbeats to the interrupt queue; The waiting container instances compete for the distributed lock through atomic writes; The takeover container instance of the distributed lock reads the block confirmation bitmap, the confirmation offset, the size of the target temporary file, and the confirmation size in the accompanying verification file. It takes the minimum recoverable offset to truncate the target temporary file and continues to write block data from the minimum recoverable offset.

8. The microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 1, characterized in that, Unschedulable instances are filtered and sorted based on link quality, storage capacity, container instance load, and deadline, and further include: The link quality includes available bandwidth, round-trip latency, and packet loss rate of the cross-base link; the storage capacity includes the remaining capacity of the target end and disk write capability; the container instance load includes the number of concurrent tasks of the container instance; and the deadline is the deadline of the backup task. The task scheduling microservice reads the available bandwidth of the cross-base link, the round-trip latency, the packet loss rate, the remaining capacity of the target end, the disk write capacity, the number of concurrent tasks of the container instance, and the base to which the source simulator belongs; Remove container instances that do not meet the storage reservation conditions, concurrency limit conditions, link availability conditions, or connection pool warm-up conditions; The link margin is determined based on the link quality, and the urgency of the deadline is determined based on the deadline. For the remaining container instances, a ranking score is calculated based on the remaining link capacity, the remaining storage capacity, the affinity with the same base, the urgency of the deadline, historical transmission performance, and the container instance load. Backup tasks are then assigned to container instances with higher ranking scores.

9. A microservice-based container deployment method for a flight simulator data disaster recovery system according to claim 7, characterized in that, When the lease expires and the heartbeat stops, the takeover container instance that acquired the distributed lock reads the block confirmation bitmap, the confirmation offset, and the accompanying verification file, truncates or resumes the target-end temporary file, and outputs the committed file index, further including: Read the block check list and confirmation size in the accompanying check file, and compare the confirmation size with the maximum consecutive block end offset indicated by the block confirmation bitmap; When the size of the target temporary file is greater than the confirmed size or greater than the maximum continuous block end offset, the target temporary file is truncated to a smaller offset; When the block confirmation bitmap is lost, delete the target temporary file and move the backup task back into the task scheduling queue.

10. A microservice-based container deployment system for a flight simulator data disaster recovery system, characterized in that, include: The service splitting and gating module is configured to split the monolithic disaster recovery application into device access microservice, task scheduling microservice, transmission execution microservice, metadata microservice and gateway microservice. It configures layered ready tokens based on service dependency topology and releases downstream service startup permits after the basic service health probe passes. The image and cache module is configured to generate stateless container images for transport execution microservices, and write the source path, target path, file size, task version, chunk confirmation bitmap and confirmation offset of the backup task to the coordination cache. The resource scheduling module is configured to filter out unschedulable instances and sort them based on link quality, storage capacity, container instance load and deadline, and then assign backup tasks to the container instances that are ranked higher. The transmission confirmation and submission module is configured to have the container instance write block data to the target temporary file. When a single block is flushed to disk and the verification is successful, the block confirmation bitmap and the confirmation offset are updated. After the file is divided into blocks, an accompanying verification file is generated. After verifying the file size and overall hash of the target temporary file, it is atomically renamed to a backup file and the file index is submitted with the task identifier and task version as unique keys. The takeover module is configured to, when the lease expires and the heartbeat stops, acquire the takeover container instance of the distributed lock, read the block confirmation bitmap, the confirmation offset and the accompanying verification file, truncate or resume the transmission of the target end temporary file and output the file index after submission.

Images