Heterogeneous system data backup method and system based on virtual disk mounting

Abstract

This application provides a method and system for data backup of heterogeneous systems based on virtual disk mounting. By creating a unified virtual disk mounting entry point on heterogeneous operating systems, it solves the differences in storage access methods, file access protocols, and client implementations among Windows, Mac, and Linux systems. This allows upper-layer applications to access storage resources across data centers without modification, significantly reducing system complexity and operational costs. Employing cache-first write and cross-center write mechanisms effectively reduces front-end write latency and improves concurrency handling capabilities. A backup trigger mechanism is introduced, automatically releasing cache space and retaining metadata indexes after a successful backup, improving cache utilization efficiency and enabling dynamic cache space reclamation. Simultaneously, the metadata index ensures rapid location and on-demand recovery of backup data. In this way, efficient and reliable data writing and automatic backup are achieved while ensuring unified access across heterogeneous systems.

Description

Technical Field

[0001] This invention relates to the field of data backup and disaster recovery technology, and in particular to a method and system for data backup of heterogeneous systems based on virtual disk mounting. Background Technology

[0002] With the widespread application of cloud computing, edge computing, and multi-data center architectures across wide area networks, enterprise business data is flowing more and more frequently between local data centers, private clouds, and public clouds.

[0003] In existing technologies, different operating systems (such as Windows, Mac, and Linux) differ in storage access methods, file access protocols, and client implementations. These differences lead to inconsistent cross-platform data access interfaces and high operational complexity. Regarding cross-datacenter data writing, traditional solutions typically involve applications directly writing to remote storage. This approach suffers from high write latency in wide area network (WAN) scenarios, making it difficult to meet real-time business requirements. Furthermore, cross-WAN data transmission faces network security threats, making data transmission security difficult to guarantee.

[0004] In terms of data backup, traditional backup solutions generally have limitations such as scattered task triggering, inflexible backup mode switching, invisible process progress, imperfect failure retry mechanism, and insufficient audit tracking capabilities, making it difficult to meet the requirements for highly reliable, recoverable, and auditable backup data security access in scenarios with multiple data centers across wide area networks. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for data backup of heterogeneous systems based on virtual disk mounting, which can achieve efficient and reliable data writing and automatic backup while ensuring unified access of heterogeneous systems.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for data backup of heterogeneous systems based on virtual disk mounting includes: A virtual disk is created and mounted on a heterogeneous operating system, and the virtual disk provides a unified access point for upper-layer applications of the heterogeneous operating system. In response to the write request initiated by the upper-layer application through the virtual disk, the data to be written is stored in the cache, metadata corresponding to the write request is generated, and the data to be written is written to the target data center according to the metadata; The data to be written in the cache is backed up to heterogeneous backup storage. After the backup is successful, the cache space corresponding to the data to be written in the cache is released and the index information of the metadata is stored.

[0007] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A heterogeneous system data backup system based on virtual disk mounting includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the aforementioned heterogeneous system data backup method based on virtual disk mounting.

[0008] The beneficial effects of this invention are as follows: By creating a unified virtual disk mounting entry point on heterogeneous operating systems, it solves the differences in storage access methods, file access protocols, and client implementations among Windows, Mac, and Linux systems. This allows upper-layer applications to access storage resources across data centers without modification, significantly reducing system complexity and operational costs. The adoption of cache-first write and cross-center write mechanisms effectively reduces front-end write latency and improves concurrency handling capabilities. The introduction of a backup trigger mechanism automatically releases cache space and retains metadata indexes after a successful backup, improving cache utilization efficiency and enabling dynamic cache space reclamation. Simultaneously, the metadata index ensures rapid location and on-demand recovery of backup data. In this way, efficient and reliable data writing and automatic backup are achieved while ensuring unified access across heterogeneous systems. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating a heterogeneous system data backup method based on virtual disk mounting, according to an embodiment of the present invention. Figure 2 This is the main flowchart of virtual disk mounting and reading / writing in a heterogeneous system across multiple data centers in a wide area network according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the virtual disk creation and cross-center writing process according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the unified mounting and reading acceleration of heterogeneous systems in an embodiment of the present invention; Figure 5 This is a flowchart illustrating secure transmission of multi-datacenter content networks based on Virtual Endpoint Channel (VEN) according to an embodiment of the present invention. Figure 6 This is a flowchart illustrating the triggering of a backup task according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a heterogeneous system data backup system based on virtual disk mounting, according to an embodiment of the present invention. Detailed Implementation

[0010] Definitions:

[0011] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0012] In existing technologies, with the widespread application of cloud computing, edge computing, and multi-datacenter architectures across wide area networks, enterprise business data flows increasingly frequently between local data centers, private clouds, and public clouds. Different operating systems, such as Windows, Mac, and Linux, differ in storage access methods, file access protocols, and client implementations, resulting in inconsistent cross-platform data access interfaces. Operations personnel need to deploy and maintain different client programs for different operating systems, significantly increasing system complexity and operational costs. Meanwhile, traditional backup solutions generally suffer from limitations such as a single backup task triggering method, inflexible backup mode switching, invisible process progress, imperfect retry mechanisms after transmission failures, and insufficient audit trail capabilities, making it difficult to meet the high reliability, recoverability, and auditability requirements of multi-datacenter scenarios across wide area networks. Furthermore, existing technologies lack a collaborative mechanism between cache management and backup tasks. Data in the cache is usually not automatically released after backup, leading to low cache space utilization. There is also a lack of effective metadata index association between cached data and backup data, making it difficult to quickly locate and recover data during subsequent access.

[0013] To address at least some of the aforementioned issues, a unified virtual disk mounting entry point is created on heterogeneous operating systems. This is combined with cache-first, cross-center write, and intelligent backup triggering mechanisms to achieve collaborative management of efficient cross-center data writing and automatic backup. This approach masks the differences between heterogeneous systems, enabling unified access, reducing write latency and improving concurrency performance. Furthermore, intelligent triggering and cache release mechanisms optimize storage resource utilization and backup efficiency.

[0014] In this embodiment, it is assumed that an enterprise deploys three data centers, A, B, and C, in a cross-WAN environment, with business servers running Windows, Mac, and Linux systems respectively. The enterprise aims to achieve unified virtual disk mounting, transparent cross-center access, and automatic backup without changing the read / write methods of upper-layer applications, thereby reducing operational complexity and improving data reliability. To this end, this embodiment introduces the VEI mechanism: to achieve "topology hiding + secure routing + unified access" during cross-center access. The VEI, as a logical endpoint identifier for cross-data center access, does not directly expose the real internal network IP and real service port. Its structure can be defined as: {VirtualHostID}-{DataCenterID}-MCG-server-{EncryptedPort}; Where: VirtualHostID: Virtual alias of the target storage node; DataCenterID: Identifier of the target data center; MCG-server: Protocol signature field, used for fast legality verification; EncryptedPort: Port field that has been obfuscated / mapped, used to hide the real service port.

[0015] In this embodiment, the VEI is carried in the SOCKS5h request field for cross-center transmission. The receiving gateway performs the following on the VEI: protocol signature verification, host alias resolution, port deobfuscation / mapping, and internal network connection establishment. This achieves a cross-center access method that "only sees the logical identifier, not the physical topology".

[0016] The VEI mechanism described above involves the following five processes: (1) Mounting and reading / writing virtual disks across multiple data centers in a wide area network: Deploy a client agent on the user application server. The client supports heterogeneous systems such as Windows / Mac / Linux, creates and mounts virtual disks, and provides unified drive letters or mount points for upper-layer applications to access them.

[0017] When an application initiates a write or backup command, the client performs authentication and policy verification, writes the file content to the cache service, and generates metadata to write to the global metadata service.

[0018] The system determines the target data center based on metadata: the local center writes to the backend service using a direct connection, while the remote center encapsulates the request with VEI and forwards it through a gateway tunnel.

[0019] After writing is complete, update the file location, version, and backup status; when reading, prioritize accessing the cache, and if a cache miss occurs, retrieve the source based on the metadata and backfill the cache, and finally return the read result and perform a consistency check.

[0020] (2) Virtual disk creation and cross-center writing: The client creates and mounts a virtual disk on the application server, and the application executes file writing through the virtual disk.

[0021] The system first submits the written data to the cache service, and at the same time generates metadata such as file block mapping, version number, timestamp and target center identifier.

[0022] If the target center is local, a local direct write operation is performed; if the target center is remote, the write operation is performed via VEI encapsulation and forwarded through a WAN tunnel to the target center.

[0023] Upon successful writing, a confirmation message is returned, and the global metadata status is updated synchronously.

[0024] (3) Unified mounting and reading acceleration of heterogeneous systems: Windows, Mac and Linux applications initiate reading requests through a unified virtual disk entry point.

[0025] The system first checks if the cache is hit. If it is hit, the data is returned directly. If it is not hit, the system locates the data center based on global metadata and performs a read from the source.

[0026] After the data is returned from the source, it is written to the high-speed cache service for backfilling before being returned to the client, thereby improving the hit rate of subsequent similar requests and the efficiency of cross-center reading.

[0027] After the data is read, a data consistency check is performed to ensure that the returned content is consistent with the target version.

[0028] (4) Ensure secure transmission of content across multiple data centers via Virtual Endpoint Network (VEN): During cross-data center read-back to source, write-forward and backup transmission, the system carries and interprets VEI (Virtual Endpoint Identifier) ​​information through Virtual Endpoint Network (VEN), transforming the traditional direct exposure method of "real IP + port" into an access method of "logical endpoint identifier + controlled mapping".

[0029] Among them, VEI is used to define "who can access it, and under what permissions and time limits"; VEN is used to define "how VEI is securely transmitted, verified, parsed, and forwarded". VEI is the carrier of access semantics and target identification, and VEN is the cross-center transmission and execution channel. The two form an "identifier-channel" collaborative relationship.

[0030] The VEI is dynamically generated by the sending side based on the request context and includes at least a resource logical identifier, access type, policy version, timeliness parameter, and signature digest. The VEN includes at least a virtual host identifier, a target data center identifier, a protocol verification field, and a port obfuscation field. The sending side generates the VEI based on global metadata and real-time policies, and encapsulates / binds the VEI to the VEN before forwarding it through the gateway tunnel. The receiving side first performs channel-level validity verification on the VEN, and then performs integrity and timeliness verification on the VEI. After the verification passes, target resolution and intranet mapping are performed, and a forwarding connection is established.

[0031] Furthermore, VEI can adopt a one-time token or time-limited and use-limited mechanism, while VEN can combine encrypted tunnels and replay protection mechanisms to prevent identifier forgery, path detection, and unauthorized reuse.

[0032] Through the above mechanisms, the topology of backend storage nodes is hidden, access paths are controllable, identifier semantics are verifiable, and data streams are securely forwarded during cross-center transmission, thereby ensuring the secure transmission capabilities of multi-datacenter content networks.

[0033] (5) Policy-driven automatic backup: Supports manual configuration or centralized distribution of backup policies. The policy includes parameters such as backup cycle, execution window, target center, retention time and number of retries.

[0034] The system automatically triggers backup tasks according to policies and performs full, incremental, or differential backups based on the configuration.

[0035] The transmission progress and result status are recorded during task execution; if the task fails, it is re-executed according to the retry policy or enters the next scheduling cycle.

[0036] After a task is successfully completed, a backup integrity verification is performed, the backup status is updated, and the data is written to the audit log, forming a closed-loop management system of "policy issuance - task execution - result verification - status tracking".

[0037] By coordinating the entire chain of "unified architecture - cross-center writing - cache-accelerated reading - strategy-based backup", unified mounting, efficient local reading and writing, and automated backup are achieved in cross-WAN multi-center scenarios, significantly improving system performance, reliability, and maintainability.

[0038] The following details a heterogeneous system data backup method based on virtual disk mounting, as described in this invention. Please refer to [link / reference]. Figure 1 The method 100 includes steps 101 to 103: Step 101: Create and mount a virtual disk on the heterogeneous operating system, and provide a unified access point for the upper-layer applications of the heterogeneous operating system through the virtual disk.

[0039] Specifically, a client proxy is deployed on the user application server, adapting to heterogeneous operating systems such as Windows, Mac, and Linux. The client proxy creates virtual disk devices on the application server and mounts these virtual disks as system-recognizable drive letters or mount points. The virtual disk exposes a standard file system interface to upper-layer applications, allowing them to access data in a local file read / write manner. The client proxy uses FUSE virtual file system technology to transform standard file operations from upper-layer applications into unified access requests to the underlying multi-center storage. Upper-layer applications initiate read / write requests using the local file system, without needing to be aware of the backend multi-datacenter topology.

[0040] Step 102: In response to the write request initiated by the upper-layer application through the virtual disk, store the data to be written in the cache, generate metadata corresponding to the write request, and write the data to be written to the target data center according to the metadata.

[0041] The write request needs to be authenticated, tenant isolated, and policy validated. If the validation fails, a write rejection result is returned to the upper-layer application and an audit log is recorded.

[0042] Specifically, the upper-layer application submits a file write request to the virtual disk. The write request includes file content, file identifier, write offset, and length information. After receiving the write request, the client agent first performs identity authentication, tenant isolation verification, and access policy verification on the request. During identity authentication, the system verifies the identity information of the request initiator, including username, password, token, etc., to determine whether the request initiator is a legitimate user. During tenant isolation verification, the system verifies the tenant information to which the request initiator belongs to determine whether the request initiator has permission to access the target resource, avoiding data leakage issues caused by cross-tenant access. During policy verification, the system verifies whether the request meets the policy requirements according to the preset access policy, including access time, access frequency, access quota, and other restrictions. If the verification fails, a write rejection result is returned to the upper-layer application and an audit log is recorded. If the verification passes, the client agent prioritizes writing the data to the high-speed cache service to reduce front-end write latency and improve concurrency absorption capacity. While writing to the cache, the client agent generates file metadata and writes it to the metadata service. The metadata includes at least the file identifier, version number, block mapping, timestamp, target center identifier, and replica status. By establishing a mapping relationship between logical files and physical disk paths through metadata, a basis is provided for subsequent routing decisions and consistency management.

[0043] Specifically, writing the data to be written to the target data center based on the metadata involves: determining whether the target data center is a local data center based on the metadata; if it is a local data center, writing the data to be written to the backend storage service of the local data center using a direct connection method; otherwise, generating a virtual endpoint identifier based on the write request and its corresponding metadata, and transmitting the virtual endpoint identifier and the data to be written to the gateway of the target data center through a virtual endpoint network tunnel to write the data to be written to the backend storage service of the target data center.

[0044] Specifically, the system determines the target data center based on metadata policies, proximity access rules, or disaster recovery policies. The system then uses the target data center identifier field in the metadata to determine if the target data center is a local data center. If the target is a local data center, a direct connection is used to write to the local backend storage service. The client agent sends the write task directly to the local data center's backend storage service. After the local backend completes data persistence, it returns a confirmation result. The client agent then updates the write status and version information accordingly and records the audit log.

[0045] Step 103: Back up the data to be written in the cache to heterogeneous backup storage. After successful backup, release the cache space corresponding to the data to be written in the cache and store the index information of the metadata.

[0046] The heterogeneous backup storage includes at least one of tape library storage, object storage, cloud storage, and disk array storage.

[0047] Step 1031: Monitor the cache capacity usage, the access frequency and last access time of the data to be written, and the transmission rate and link load of the preset backup link; when the capacity usage reaches a preset threshold, or the data to be written is not accessed within a preset time, or the backup link meets the preset high bandwidth and low load conditions, back up the data in the cache.

[0048] Specifically, the system dynamically selects the backup timing and mode based on the distributed storage (i.e., cache occupancy status), data access characteristics, and off-site storage link conditions. During cache level detection, the system monitors cache capacity occupancy in real time. When cache occupancy reaches a preset threshold, the data to be migrated in the cache is identified as backup targets, and the backup process is triggered to release the corresponding cache space after backup completion. During data access frequency determination, the system statistically analyzes the access frequency and last access time of cached data. When target data has not been accessed within a preset time period, it is classified as low-frequency access data and backup migration is prioritized to reduce the occupancy of cache resources by low-frequency data. During off-site link condition determination, the system monitors the transmission rate or link load of backup storage media such as Huawei Cloud and Alibaba Cloud. When a high-bandwidth or low-load state is detected in the backup link, the backup task priority is increased or backup is triggered directly to improve backup transmission efficiency and reduce the impact on online services.

[0049] Step 1032: Match the backup mode according to the preset backup strategy and historical backup records. The backup mode includes full backup mode, incremental backup mode, or differential backup mode. If the data to be written is the first backup or the corresponding historical backup record reconstruction, then perform a full backup of the data to be written in the cache. If the data to be written is data that has changed since the last backup, then perform an incremental backup of the data to be written in the cache. If the data to be written is data that has changed since the most recent full backup, then perform a differential backup of the data to be written in the cache.

[0050] Specifically, the system determines the backup mode based on the historical backup status of the data to be backed up. The system queries metadata to retrieve historical backup records. If there are no historical records or the records are corrupted, a full backup mode is selected. Full mode is used for the initial backup or reconstruction of historical backup records, transferring all content of the data to be backed up. If there are historical records and it is a full backup, an incremental backup mode is selected. Incremental mode is used to transfer only data that has changed since the last backup. Differential mode is used to transfer data that has changed since the most recent full backup. After selecting a mode, a list of data to be transferred is generated, and the mapping relationship between version snapshots and file blocks is recorded.

[0051] Step 1033: Record the progress information during the backup transmission process in real time. The progress information includes the number of bytes transmitted, the completion rate, the current rate, the estimated remaining time, and the information on failed segments. If the progress information meets the preset abnormal conditions, then perform automatic retry and retransmission of failed segments.

[0052] Specifically, the system performs backup transmission according to the data list, supporting chunked upload, breakpoint resumption, and concurrent channel transmission. During transmission, progress information is recorded in real time, including the number of bytes transmitted, completion percentage, current rate, estimated remaining time, and information on failed segments. The system writes this progress information to the task status storage for user query and monitoring. When the progress information meets preset abnormal conditions, such as a transmission rate consistently below a preset threshold, multiple consecutive transmission failures, or the number of failed segments exceeding a preset threshold, the system performs automatic retries and retransmission of failed segments. The system configures the retry strategy, setting the number of retries, retry interval, and retry timeout. Automatic retries are performed for abnormal scenarios; if the retry is successful, transmission continues; if the retry fails, the failed segment information is recorded and the system enters the next scheduling cycle. For failed segments, the system locates the failed data blocks based on the failed segment information and re-initiates transmission requests until all data blocks are successfully transmitted.

[0053] Step 1034: If the backup transmission is successful, perform a consistency check on the transmission result data and update the backup status information.

[0054] Specifically, a consistency check is performed on the data to be backed up and the data transmitted, including file-level check, block-level check, and version consistency check. If the check passes, the backup status information is updated, which includes at least the task completion marker, completion time, backup version number, and recoverable point information.

[0055] Step 1035: Delete the data block corresponding to the data in the cache; store the index information of the metadata, the index information including the data block identifier, backup location and backup time.

[0056] Specifically, after the backup data is successfully written to the backup storage, the system calls the caching service to delete the corresponding data block from the cache. The system locates the corresponding data block in the cache based on the data block identifier, performs the deletion operation, and releases the cache space. The metadata retains index information for the data block identifier, backup location, and backup time. The data block identifier uniquely identifies the data block, the backup location records the data block's storage location in the backup storage, and the backup time records the data block's backup time. During subsequent accesses, if the cache is not found, the system locates the data block in the backup storage based on the index information in the metadata, restores the data from the backup storage, populates it back into the cache, and then returns it to the upper-layer application.

[0057] In this embodiment, a system architecture consisting of a client, an access control module, a high-speed cache service, a metadata management module, and storage nodes across multiple data centers in a wide area network is constructed. A unified virtual disk access entry is established to form a full-cycle data information record from request access, metadata parsing, read / write routing, backup path, and result return, thereby realizing unified security scheduling and management of cross-WAN multi-center data services.

[0058] As described above, this embodiment solves the differences in storage access methods, file access protocols, and client implementations among Windows, Mac, and Linux systems by creating and mounting virtual disks on heterogeneous operating systems. This allows upper-layer applications to transparently access storage resources across multiple data centers on a wide area network without modification, significantly reducing system complexity and operational costs. A cache-first write mechanism is adopted, storing data to be written in a high-speed cache and immediately returning a success response, reducing front-end write latency and improving concurrency handling capabilities. A three-dimensional intelligent backup triggering mechanism based on cache capacity occupancy, data access frequency, and backup link conditions is introduced, enabling automated and intelligent selection of backup timing, optimizing storage resource utilization while ensuring backup efficiency. After a successful backup, cache space is automatically released while retaining metadata indexes, improving cache utilization efficiency and enabling dynamic cache space reclamation. Simultaneously, the metadata index ensures rapid location and on-demand recovery of backup data, improving data service continuity and overall operational efficiency.

[0059] Please refer to Figure 2 , Figure 2 This document illustrates the main process of mounting and reading / writing virtual disks across a heterogeneous system with multiple data centers in a wide area network (WAN). It assumes an enterprise deploys multiple data centers in a WAN environment, with business servers running on heterogeneous operating systems such as Windows, Mac, and Linux. To achieve unified mounting, transparent cross-data center writing, and efficient reading without modifying the upper-layer application's read / write logic, a FUSE-based virtual disk is used as a semantic adaptation layer between the application and the underlying multi-data center storage. The upper-layer application still accesses the data center using POSIX file interfaces (open / read / write / close / fsync / rename, etc.), without needing to be aware of the underlying data center locations, replication strategies, or routing details. The process includes steps 201 to 207: Step 201, Client Proxy Deployment and Unified Mounting: Deploy a client proxy on the user application server. This client proxy is compatible with heterogeneous Windows / Mac / Linux systems. The client creates and mounts a virtual disk on its local machine, providing a unified drive letter or mount point access point for upper-layer applications. Upper-layer applications initiate read and write requests using the local file system, without needing to be aware of the backend multi-datacenter topology.

[0060] Step 202, Access Request and Authentication Policy Verification: The client receives a write request or backup request initiated by the application. It performs identity authentication, tenant isolation verification, and access policy verification on the request. If the verification passes, the data writing process begins; if the verification fails, a rejection result is returned and recorded in the audit log.

[0061] Step 203, Write to Cache and Register Global Metadata: The client prioritizes writing file content to the cache service to reduce write latency and absorb sudden traffic surges. Metadata is synchronously generated and written to the global metadata service. This metadata includes file identifier, version number, block mapping, timestamp, target center identifier, and replica status. A mapping relationship between "file logical view—physical storage location" is established based on the metadata.

[0062] Step 204, Target Data Center Determination and Write Routing: The system determines the target data center based on metadata policies, proximity access rules, or disaster recovery policies. If the target is a local data center, a direct connection is used to write to the local backend storage service. If the target is a remote data center, the write request is encapsulated with a VEI and forwarded across the WAN via a gateway tunnel to the remote data center's backend storage service to complete the write operation.

[0063] Specifically, the VEI encapsulation process includes: when the target is a remote center, the source gateway performs the following encapsulation: (1) Use data: Read the central database to obtain the target data center identifier (DataCenterID), the real intranet IP and real service port (Port) corresponding to the target resource, and the request task identifier.

[0064] (2) Data processing: Map the real IP to the virtual host identifier (VirtualHostID) according to the local address registration rules; obtain the real Port and access timestamp according to the custom encryption method (EncryptedPort); add the protocol signature field MCG-server, and can attach timeliness and integrity verification information.

[0065] (3) Packaging results: Assemble the VEI based on the above processing results: {VirtualHostID}-{DataCenterID}-MCG-server-{EncryptedPort} This VEI serves only as a cross-center logical endpoint identifier and does not contain a real internal network IP address or plaintext port. For example: Source data VEI encapsulation process: Data center A (ID=beijing, virtual host=data1, real IP=10.111.111.11); Data center C (ID=fuzhou, virtual host=node1, real IP=10.222.222.22). When A accesses port 9000 of C: the VEI can be encapsulated as: node1-fuzhou-MCG-server-ep_7F2C9D; where ep_7F2C9D is an example obfuscation value obtained by encrypting port 9000 and the access timestamp using a custom encryption method.

[0066] The VEI parsing and restoration process is as follows: After receiving the VEI, the remote gateway first verifies the protocol signature field "MCG-server" and its timeliness / integrity information; after the verification is successful, it parses the VEI to obtain VirtualHostID, DataCenterID, and EncryptedPort; it queries the target data center database using VirtualHostID as the key to obtain the real internal network IP; it decrypts EncryptedPort using a custom encryption method to obtain the real service port Port; it combines the two to obtain the real access endpoint IP:Port and forwards it to the corresponding backend storage service.

[0067] For example: After receiving a request from A via a VEN tunnel, C extracts the VEI (node1-fuzhou-MCG-server-ep_7F2C9D) from the SOCKS5 handshake domain field. It then resolves this to obtain VirtualHostID=node1, DataCenterID=fuzhou, and EncryptedPort=ep_7F2C9D. C uses VirtualHostID=node1 as the key to query the central database, obtaining the real IP address 10.222.222.22. It decrypts ep_7F2C9D using a custom encryption method to reveal the service port 9000, and combines these to obtain the target endpoint 10.222.222.22:9000. C establishes a connection with the target endpoint service and forwards the request data from A to this endpoint to complete the write / read operation.

[0068] Throughout the process, the real internal network IP and real port are not exposed in plaintext on the wide area network. Only the VEI logical identifier is visible to the outside, thus achieving topology hiding and minimal exposure access.

[0069] Step 205, Status Write-back and Version Update: After the write operation is complete, update the file location, version information, synchronization progress, and backup status. Send the write result back to the client and persist the latest state in the global metadata service. When a multi-replica strategy exists, record the completion status and consistency flags of each replica.

[0070] The system uses the metadata field Data Locate as a record of replica completion status: whenever a target data center completes the replica write and passes verification, its identifier is appended to Data Locate. A data center identifier already included in Data Locate indicates that the replica is complete; a data center identifier not included indicates that the replica is still being synchronized or is incomplete.

[0071] The consistency flag is generated based on the matching relationship between Data Locate and the policy target set: when Data Locate covers all data centers required by the policy, it is marked as "replica consistency completed"; otherwise, it is marked as "not fully consistent / in synchronization" and compensation synchronization continues to be triggered.

[0072] Taking cross-center backup of AI training data as an example, the policy requires that the replicas reside in Beijing and Fuzhou. After the file is written to Beijing, Data Locate=[Beijing], and the consistency flag is "Not fully consistent / in synchronization"; after the Fuzhou replica is completed and verified, Data Locate=[Beijing, Fuzhou], and the consistency flag is updated to "Replica consistency completed".

[0073] Step 206, Prioritize Cache Reading and Origin Server Compensation: After receiving a read request, the client prioritizes accessing the cache service. If the cache is hit, the data is returned directly; if the cache is not hit, the target center is located based on global metadata, and an origin server read is performed. After the origin server data is returned, the cache is populated, and the read result is returned to the application.

[0074] Step 207, Consistency Verification and Result Return: After read / write operations are completed, version consistency verification, integrity verification (optional hash verification), and policy compliance verification are performed. If the verification passes, the operation is confirmed as successful; if the verification fails, a retry, alarm, or rollback process is triggered. Finally, the execution result, conforming to standard file system semantics, is returned to the upper-layer application.

[0075] Through the above process, this embodiment realizes unified virtual disk mounting and transparent read / write across wide area networks, multiple data centers, and heterogeneous operating system environments; while ensuring access consistency, it also takes into account low-latency access capabilities, cross-center reachability, and backup traceability.

[0076] Please refer to Figure 3 , Figure 3 This illustrates a method for virtual disk creation and cross-center writing. It assumes the business application is deployed on the application server side, and the client agent already possesses virtual disk management and cross-center routing capabilities. Under the condition that the upper-layer application is unaware of the underlying storage topology, this embodiment implements a unified write entry point and local / remote adaptive writing. This includes the following steps 301 to 306: Step 301: The client creates and mounts a virtual disk on the application server: The client agent creates a virtual disk device on the application server and mounts it as a system-recognizable drive letter or mount point. The virtual disk exposes a standard file system interface to upper-layer applications, allowing applications to access data in a local file read / write manner. Optionally, the client loads access control policies, caching policies, and routing policies during the mounting phase.

[0077] Step 302: The application initiates a file write operation via the virtual disk: The upper-layer application submits a file write request to the virtual disk, the request including file content, file identifier, write offset, and length information. After receiving the write request, the client performs basic checks, including session validity checks, permission checks, and quota checks. If the checks pass, the write request is converted into an internal unified write task.

[0078] Step 303: Write to the cache service and generate metadata: The client prioritizes writing data to the cache service to reduce front-end write latency and improve concurrency handling. Simultaneously, file metadata is generated and written to the metadata service. This metadata includes at least: file ID, block mapping, version number, timestamp, target center identifier, and status bit. The metadata establishes a mapping relationship between logical files and physical disk paths, providing a basis for subsequent routing decisions and consistency management.

[0079] Step 304: Determine if the target center is a local center: The system determines whether the target writing center is a local center based on metadata policy, data ownership policy, or scheduling policy. If the determination result is "yes," the system proceeds to the local direct connection writing process. If the determination result is "no," the system proceeds to the remote encapsulation and forwarding writing process.

[0080] Step 305, Local Direct Connection Write Process (determined as "Yes"): The client sends the write task directly to the local data center backend storage service. After the local backend completes data persistence, it returns a confirmation result. The client updates the write status and version information accordingly and records the audit log.

[0081] Step 306, VEI encapsulation and tunnel forwarding of the remote write process (determined as "No"): The client performs VEI encapsulation on the write task to form a unified data unit adapted for cross-WAN transmission. The encapsulated data is forwarded to the target remote data center via the gateway tunnel. After the remote data center's backend storage service completes the disk write, it returns a confirmation result, and the client updates the global metadata status accordingly.

[0082] Finally, regardless of whether it's a local direct connection or a remote forwarding, a write result receipt is generated uniformly upon completion of the write operation. The client returns the result to the upper-layer application and synchronously updates the file version, storage location, and task status. Optionally, the system triggers subsequent consistency checks, retry mechanisms, or replica synchronization mechanisms to ensure the reliability of cross-center writes.

[0083] Through the above steps, this embodiment implements a "cache first, route later, and write to disk" write mechanism under the unified virtual disk access semantics, and supports automatic selection of local direct connection or remote tunnel forwarding based on the target center, thereby taking into account write performance, cross-center reachability and operation and maintenance consistency.

[0084] Please refer to Figure 4 , Figure 4 This paper illustrates a unified mounting and read acceleration method for heterogeneous systems. It assumes that Windows, Mac, and Linux applications all access storage resources across multiple data centers on a wide area network through a unified virtual disk entry point. To reduce cross-datacenter read latency and improve the efficiency of accessing hot data, this embodiment employs a "caching priority, origin cache miss, and refill after origin cache" read mechanism. This includes steps 401 to 405: Step 401: The unified entry point for heterogeneous systems initiates a read operation: Windows / Mac / Linux applications initiate a file read request through a unified virtual disk. The request includes a file identifier, read offset, and read length. After receiving the request, the client agent performs basic validity checks, which include at least session state checks, permission checks, and request parameter checks. Upon successful verification, the process proceeds to the cache determination stage.

[0085] Step 402, Cache Service Hit Determination: The client accesses the cache service based on the file identifier and block index, and determines whether the target data block has been hit. If the determination result is "yes" (cache hit), the data is directly read from the cache service and returned to the application. If the determination result is "no" (cache miss), the metadata location and origin retrieval process begins.

[0086] Step 403, Locate Local / Remote Origin Retrieval Based on Metadata: The client obtains the storage location, version, and copy information of the target file from the global metadata service. If the metadata indicates that the data is located in the local center, a local origin retrieval read is initiated; if it is located in a remote center, a remote origin retrieval read is initiated via a cross-WAN link. During the origin retrieval process, link health checks and timeout control can be optionally performed to ensure the stability of the read task.

[0087] Step 404: Backfill data from the origin server to the cache and return it: After a successful origin server read, the read data blocks are written back to the cache service, forming a hot data accumulation. The client returns the origin server result to the application process that initiated the read, maintaining standard file system read semantics. For continuous reads or frequently accessed files, subsequent requests can prioritize hitting the cache, thereby reducing the average read latency.

[0088] Step 405, Read Consistency and Integrity Verification: Perform version consistency verification before or after data return to ensure that the cached data matches the backend metadata version. Optional integrity verification (such as hash verification or block-level verification) can be performed to prevent data anomalies caused by cross-center transmission. If verification fails, trigger invalidation retrieval, cache eviction, or retry mechanisms, and record alarms and audit logs.

[0089] Through the above process, this embodiment realizes unified mounting and reading in heterogeneous operating system environments, and significantly reduces the number of cross-center origin accesses through a cache hit priority mechanism; in cache miss scenarios, it combines metadata for precise location and backfilling strategies, taking into account read performance, data consistency and cross-WAN access reliability.

[0090] Please refer to Figure 5 , Figure 5 This paper illustrates a secure content network transmission method for multiple data centers based on the Virtual Endpoint Channel (VEN). It assumes that the client has completed virtual disk mounting and can access the global metadata service and local gateway. To achieve secure access to target storage resources in a multi-data center environment across a wide area network, this embodiment completes cross-data center access through a closed-loop mechanism of "request triggering—dynamically generating VEI—VEN bearer transmission and interpretation—target resolution mapping—result return—audit logging." This includes the following steps 501 to 507: Step 501: The client initiates a cross-center access request: Based on business read / write requirements, the client initiates an access request, which includes at least the access object identifier, operation type, and context information. The local gateway receives the request and performs identity authentication, permission verification, and policy matching. If the verification fails, access is denied and a security log is recorded; if the verification passes, the process proceeds to the next step.

[0091] Step 502, Local / Cross-Center Path Determination: The local gateway uses the global metadata service to query the location of the target data. If the target data is located in the local data center, it directly accesses the local storage and returns the result. If the target data is located in a remote data center, it proceeds to VEI dynamic generation and completes the VEN transmission process.

[0092] Step 503, dynamically generate VEI access information: After receiving the access request, the local gateway first performs metadata parsing and policy determination; when it is determined that the target resource is located in a remote data center, the local gateway dynamically generates the VEI corresponding to this request based on the target data center identifier, virtual host identifier and port obfuscation value returned by the metadata, combined with the current session context.

[0093] The VEI is encapsulated with structured fields, including at least: VirtualHostID, DataCenterID, protocol signature field MCG-server, and EncryptedPort obtained by mapping / obfuscating the target service port; and may be supplemented with request identifier, issuance time, expiration time, random number, and integrity verification fields.

[0094] The VEI generation process establishes a mapping relationship based on the target service's real IP and port: the real IP is not directly written into the VEI, but is obtained by resolving the VirtualHostID through the central database in the target data center; the real port is not transmitted in plaintext, but is written into the VEI in the form of EncryptedPort or port token, so that the target side can restore it according to local rules.

[0095] Step 504, interpreting and transmitting VEI via the VEN channel: When the gateway determines that the target data is located in a remote data center based on metadata, the source gateway does not directly access the remote storage node. Instead, it first establishes a cross-center virtual endpoint network (VEN) channel with the internal SOCKS5h proxy port of the target center gateway. This channel is only open between gateways and completes peer authentication through mechanisms such as whitelisting or mTLS.

[0096] The VEN acts as the carrier and execution channel for the VEI: the source gateway transmits the domain name field containing the VEI injected into SOCKS v5; the target gateway performs VEI verification, resolution, and mapping locally, restoring it to the actual internal network IP:Port before initiating the backend connection. The entire process remains location transparent to the client while not exposing the real internal network topology.

[0097] During VEI transmission and parsing, the system performs end-to-end encryption protection, integrity verification, and timeliness control (such as short-term valid tokens / nonces / timestamps) to prevent VEI from being tampered with, replayed, or used across domains without authorization.

[0098] Step 505: The target center parses the VEI and maps the access path: The target center gateway receives the VEI transmitted via VEN and performs validity, integrity, and timeliness checks. If the checks pass, the target gateway maps the virtual identifier to the target intranet storage access path (including the intranet IP and port) based on the VEI. If the checks fail or the VEI times out, the request is rejected and an abnormal status is returned.

[0099] Step 506, Access Target Storage and Return Results: The target gateway accesses the backend storage service according to the mapping result and performs read and write operations. The operation results are securely returned to the local gateway via the original VEN path, and then returned to the client. The return link includes a status code, data checksum, and target-side confirmation information.

[0100] Step 507, Status Update and Audit Tracing: Synchronously update the access status locally and the target gateway. The status includes at least the request identifier, execution time, access result, and exception information. Write to the audit log, recording the initiating entity, dynamic VEI summary, VEN channel information, mapping result, and final execution conclusion. The audit and tracing mechanism satisfies security compliance, fault location, and post-event traceability requirements.

[0101] Through the above process, this embodiment realizes a secure cross-data center access loop using dynamically generated VEI as the virtual identifier of the access host and VEN as the transmission channel, ensuring the concealment of internal network addresses and improving the security, controllability and auditability of cross-data center access.

[0102] Please refer to Figure 6 , Figure 6 A method for triggering a backup task is illustrated. It is assumed that the user has already mounted the virtual disk and has access to the global metadata service and backend storage service. To meet the data protection needs of a multi-datacenter environment across a wide area network, this invention implements a closed-loop backup task through a policy-driven, task-classified, process-traceable, and result-auditable mechanism. This includes the following steps 601 to 606: The method for triggering backup tasks proposed in this invention is as follows: Figure 5 As shown. Assume the user has completed virtual disk mounting and can access the global metadata service and backend storage service. To meet data protection needs in a multi-datacenter environment across a wide area network, this invention implements a closed-loop backup task through a policy-driven, task-classified, process-traceable, and result-auditable mechanism. The specific process is as follows: Step 601, User creates backup policy: The user creates a backup policy based on business needs, or receives a backup policy from the management terminal. The backup policy includes at least: backup object scope, execution cycle, target data center, bandwidth limit, number of retries, and retention period. The system verifies the validity of the policy and persists it as the basis for triggering subsequent tasks.

[0103] Step 602, Trigger the backup task as scheduled: The system starts the backup task according to the scheduled plan, event triggering conditions, or manual triggering instructions. When the task starts, a task identifier is generated and bound to the policy version, execution time window, and priority information. If the triggering conditions are not met, the task enters the waiting queue until the execution conditions are met.

[0104] Step 603, Select Full / Incremental / Differential Backup Mode: The system selects the backup mode based on the policy and historical backup records, including full backup, incremental backup, or differential backup. Full mode is used for the first backup or reconstruction of historical backup records; incremental mode is used to transfer only data that has changed since the last backup; differential mode is used to transfer data that has changed since the most recent full backup.

[0105] The system dynamically selects backup timing and mode based on the distributed storage (i.e., cache) occupancy status, data access characteristics, and off-site storage link conditions. Specifically, this includes: (1) Cache water level detection: Real-time monitoring of cache capacity usage; When the cache usage reaches the preset water level threshold (80%), the data to be migrated in the cache is identified as backup objects and the backup process is triggered to release the corresponding cache space after the backup is completed.

[0106] (2) Data access popularity determination: Statistically count the access frequency and last access time of cached data; when the target data is not accessed within a preset time period, such as not being accessed for 30 consecutive days, it is determined to be low-frequency access data, and backup migration is performed first to reduce the occupation of cache resources by low-frequency data.

[0107] (3) Determining the conditions of the remote link: Monitor the transmission rate or link load of the backup storage medium (such as Huawei Cloud, Alibaba Cloud and other backup storage services); when the backup link is detected to be in a high bandwidth or low load state, such as when the available bandwidth increases at night, increase the priority of the backup task or directly trigger the backup to improve the backup transmission efficiency and reduce the impact on online services.

[0108] (4) Backup mode selection: The system determines the backup mode based on the historical backup status of the data to be backed up, including full backup, incremental backup or differential backup.

[0109] (5) Cache release process: After the backup data is successfully written to the backup storage, delete or clear the corresponding data block in the cache, and retain the metadata index or mapping relationship so that subsequent accesses can be restored or migrated from the backup storage as needed.

[0110] After selecting a mode, a list of data to be transferred is generated, and the mapping relationship between version snapshots and file blocks is recorded.

[0111] Step 604, Execute transmission and record progress: The system performs backup transmission according to the data list, supporting chunked upload, breakpoint resumption, and concurrent channel transmission. Progress information is recorded in real time during transmission, including the number of bytes transmitted, completion percentage, current rate, estimated remaining time, and information on failed segments. Automatic retries and failed segment retransmissions are performed for abnormal scenarios, and intermediate states are written to the task status storage.

[0112] Step 605, Task Success Determination: After the transmission is completed, the "Task Success" determination is performed, which is based on the transmission result, integrity verification result, and the target end confirmation receipt.

[0113] If the result is "no", then retry, downgrade or rollback will be performed according to the strategy, and the process will be returned to step 603 or step 604 to re-execute the corresponding process.

[0114] If the result is "yes", then proceed to the consistency verification and state update phase.

[0115] Step 606, Verify Consistency and Update Backup Status / Audit Log: Perform consistency verification on the source data and backup data. This verification may include file-level verification, block-level verification, and version consistency verification. After successful verification, update the backup status, which includes at least the task completion marker, completion time, backup version number, and recoverable point information. Synchronously write to the audit log, recording the task initiator, execution path, mode type, abnormal retry records, and final result to meet audit and traceability requirements.

[0116] Through the above process, this embodiment realizes a complete closed loop of backup tasks from policy creation, plan triggering, mode selection, transmission execution, result judgment to consistency confirmation, which can improve backup reliability, recoverability and operation and maintenance auditability in cross-WAN multi-data center scenarios.

[0117] Please refer to Figure 7 The present invention also provides a heterogeneous system data backup system 700 based on virtual disk mounting, including a memory 701, a processor 702, and a computer program stored on the memory 701 and executable on the processor 702. When the processor 702 executes the computer program, it implements the various steps of the above-described heterogeneous system data backup method based on virtual disk mounting.

[0118] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for data backup of heterogeneous systems based on virtual disk mounting, characterized in that, include: A virtual disk is created and mounted on a heterogeneous operating system, and the virtual disk provides a unified access point for upper-layer applications of the heterogeneous operating system. In response to the write request initiated by the upper-layer application through the virtual disk, the data to be written is stored in the cache, metadata corresponding to the write request is generated, and the data to be written is written to the target data center according to the metadata; The data to be written in the cache is backed up to heterogeneous backup storage. After the backup is successful, the cache space corresponding to the data to be written in the cache is released and the index information of the metadata is stored.

2. The method according to claim 1, characterized in that, Backing up the data in the cache includes: Monitor the cache capacity usage, the access frequency and last access time of the data to be written, as well as the transmission rate and link load of the preset backup link; When the capacity occupancy reaches a preset threshold, or the data to be written is not accessed within a preset time period, or the preset backup link meets the preset high bandwidth and low load conditions, the data in the cache is backed up.

3. The method according to claim 1, characterized in that, Backing up the data in the cache includes: The backup mode is matched according to the preset backup strategy and historical backup records. The backup mode includes full backup mode, incremental backup mode or differential backup mode. If the data to be written is the first backup or the reconstruction of the corresponding historical backup record, then a full backup of the data to be written in the cache is performed. If the data to be written is data that has changed since the last backup, then perform an incremental backup of the data to be written in the cache. If the data to be written is data that has changed since the most recent full backup, then a differential backup is performed on the data to be written in the cache.

4. The method according to claim 1, characterized in that, Writing the data to be written to the target data center based on the metadata includes: Determine whether the target data center is a local data center based on the metadata; If it is a local data center, the data to be written is written to the backend storage service of the local data center using a direct connection method. Otherwise, a virtual endpoint identifier is generated based on the write request and its corresponding metadata, and the virtual endpoint identifier and the data to be written are transmitted to the gateway of the target data center through a virtual endpoint network tunnel, so as to write the data to be written to the backend storage service of the target data center.

5. The method according to claim 1, characterized in that, Backing up the data in the cache also includes: The system records the progress information during the backup transmission process in real time. The progress information includes the number of bytes transmitted, the completion rate, the current rate, the estimated remaining time, and information on failed segments. If the progress information meets the preset abnormal conditions, then automatic retry and retransmission of failed segments will be performed.

6. The method according to claim 5, characterized in that, Backing up the data in the cache, followed by: If the backup transfer is successful, the consistency of the transfer result data is checked, and the backup status information is updated.

7. The method according to claim 6, characterized in that, Perform consistency checks on the transmitted data and update the backup status information, including: A consistency check is performed on the data to be backed up and the data to be transmitted. The consistency check includes file-level check, block-level check and version consistency check. If the verification passes, the backup status information is updated. The backup status information includes at least the backup task completion marker, completion time, backup version number, and recoverable point information.

8. The method according to claim 1, characterized in that, After a successful backup, the cache space for the corresponding data in the cache is released, and the index information of the metadata is stored, including: Delete the data block corresponding to the data in the cache; The index information stores the metadata, and the index information includes data block identifier, backup location, and backup time.

9. The method according to claim 1, characterized in that, In response to a write request initiated by the upper-layer application via the virtual disk, the method further includes: The write request is authenticated, tenant isolation is verified, and policy is validated. If the verification fails, a write rejection result is returned to the upper-layer application and an audit log is recorded.

10. A heterogeneous system data backup system based on virtual disk mounting, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the heterogeneous system data backup method based on virtual disk mounting as described in any one of claims 1 to 9.

Images