File processing method and apparatus, storage medium, and electronic device

Abstract

The application discloses a file processing method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining a file operation request on a target client, the file operation request containing a target path mapped to a corresponding storage system according to a storage binding rule, the storage binding rule comprising a binding relationship between a path and a storage system, the target client being mounted with at least two heterogeneous storage systems, and the access modes of the at least two heterogeneous storage systems being different; in response to the file operation request, searching for a target storage system according to the target path on the target client; and in response to the target storage system being searched, receiving a file processing result returned by the target storage system on the target client. The application solves the technical problem that resource waste and operation and maintenance cost increase due to the fact that a user needs to maintain independent client software and configuration files for different storage systems.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a file processing method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] With the rapid development of distributed storage technology, enterprises and users often deploy multiple storage systems to meet the data storage needs of different business scenarios, such as the mixed use of object storage, block storage, and file storage systems. However, in an environment where multiple storage systems coexist, clients face significant challenges in accessing and managing files.

[0003] In related technologies, clients typically need to configure access interfaces separately for each storage system, requiring users to repeatedly write differentiated operation logic for different storage systems. Furthermore, due to protocol differences between storage systems (such as POSIX, S3, etc.), clients must additionally address issues like protocol conversion, metadata compatibility, and operation atomicity when handling file operations across storage systems, further increasing development and maintenance costs.

[0004] Furthermore, when directories or files with the same logical path exist in multiple storage systems, clients may be unable to resolve data conflicts, leading to chaotic resource access or data inconsistency. Such issues are particularly prominent when dynamically scaling storage clusters or performing data migrations, severely impacting the reliability of storage systems and user experience. Therefore, there is an urgent need for a technical solution that can shield the heterogeneity of the underlying storage, uniformly manage multiple storage resources, and ensure operational consistency.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides a file processing method and apparatus, storage medium and electronic device to at least solve the technical problem of resource waste and increased operation and maintenance costs caused by users needing to maintain independent client software and configuration files for different storage systems.

[0007] According to one aspect of the embodiments of this application, a file processing method is provided, comprising: obtaining a file operation request on a target client, wherein the file operation request includes a target path, the target path is used to map to a corresponding storage system according to a storage binding rule, the storage binding rule includes a binding relationship between the path and the storage system, the target client is mounted with at least two heterogeneous storage systems, the at least two heterogeneous storage systems having different access methods; responding to the file operation request, searching for a target storage system on the target client according to the target path, wherein the at least two storage systems include the target storage system, the target storage system corresponding to the target path; responding to finding the target storage system, receiving a file processing result returned by the target storage system on the target client, wherein the file processing result represents the result obtained by processing a target file based on the file operation request, the target file representing the file stored in the target storage system indicated by the target path.

[0008] According to another aspect of the embodiments of this application, a file processing apparatus is also provided, comprising: an acquisition module, configured to acquire a file operation request on a target client, wherein the file operation request includes a target path, the target path is used to map to a corresponding storage system according to a storage binding rule, the storage binding rule includes a binding relationship between the path and the storage system, the target client is mounted with at least two heterogeneous storage systems, the at least two heterogeneous storage systems having different access methods; a search module, configured to, in response to the file operation request, search for a target storage system on the target client according to the target path, wherein the at least two storage systems include the target storage system, the target storage system corresponding to the target path; and a receiving module, configured to, in response to finding the target storage system, receive a file processing result returned by the target storage system on the target client, wherein the file processing result represents the result obtained by processing a target file based on the file operation request, the target file representing the file stored in the target storage system indicated by the target path.

[0009] In an exemplary embodiment, the device is configured to respond to the file operation request by searching for a target storage system on the target client according to the target path in the following manner: in response to the file operation request, parsing the hierarchical structure of the target path and generating a path matching queue; selecting the storage system with the highest matching degree from the path matching queue as the target storage system according to the path binding priority in the storage binding rules; and selecting at least one storage system as the target storage system according to a preset conflict strategy when multiple storage systems are bound to the same path.

[0010] In an exemplary embodiment, the apparatus is configured to select the storage system with the highest matching degree from the path matching queue as the target storage system according to the path binding priority in the storage binding rule in the following manner: selecting a binding path that is the same as the target path, wherein the binding path represents the path bound to the storage system in the storage binding rule; if there is no binding path that is the same as the target path, selecting the binding path corresponding to the longest prefix of the target path; if there are multiple binding paths corresponding to the longest prefix, selecting a binding path according to a preset weight; and using the storage system corresponding to the binding path as the target storage system.

[0011] In an exemplary embodiment, the apparatus is further configured to: in response to a storage system release instruction, release the binding relationship between the target storage system and the corresponding path, and migrate the data corresponding to the corresponding path to another storage system, wherein the target path includes the corresponding path; or in response to a storage system release instruction, suspend the binding relationship between the target storage system and the corresponding path, and retain the data corresponding to the corresponding path; during the removal process, if there are incomplete file operation requests in the target storage system, delay the removal operation until the request is completed or a preset timeout period is reached, wherein the target path includes the corresponding path.

[0012] In an exemplary embodiment, the device is configured to, in response to a storage system release command, release the binding relationship between the target storage system and the corresponding path, and migrate the data corresponding to the corresponding path to another storage system in the following manner: in response to the storage system release command, detect the amount of file data corresponding to the corresponding path; migrate the file data corresponding to the corresponding path in batches according to a preset migration rate threshold; and delete the binding relationship between the target storage system and the corresponding path after the migration is completed.

[0013] In an exemplary embodiment, the device is further configured to: verify the write permission of the target client to the target path when the file operation request includes a file write request; if the target path is bound to multiple storage systems, select at least one storage system as the target storage system according to a preset write policy, wherein the write policy includes: writing data to the storage system with the highest priority; synchronously writing data to the storage system with write permission; and performing error handling according to a preset number of retries or rollback rules when the write fails.

[0014] In an exemplary embodiment, the apparatus is configured to receive file processing results returned by the target storage system on the target client in response to finding the target storage system as follows: in response to finding the target storage system, converting the file data returned by each target storage system into a uniform format, wherein different target nodes of the target path are mounted with different storage systems; performing conflict detection on the file data under the same path returned by multiple target storage systems, filtering according to preset conflict rules, and determining the target file; and receiving the target file encapsulated in a standard file system interface format as the file processing result on the target client.

[0015] In an exemplary embodiment, the apparatus is further configured to: obtain a file data migration instruction on the target client, wherein the file data migration instruction includes a source storage system, the target storage system, and a migration path; during the migration process, simultaneously distribute file data write requests associated with the migration path to the source storage system and the target storage system; and when the migration is completed, update the binding relationship of the migration path to point to the target storage system.

[0016] In an exemplary embodiment, the apparatus is further configured to: read the storage binding rule from a configuration file, wherein the configuration file includes a path regular expression, a storage system identifier, and permission configuration; parse the configuration file and generate a routing policy table in memory; and when an update to the configuration file is detected, dynamically load the updated storage binding rule and simultaneously update the routing policy table.

[0017] In an exemplary embodiment, the apparatus is further configured to: convert standard file operation instructions into native protocol instructions of the target storage system; standardize and map error codes returned by the native protocol; and generate compatibility results by simulating operations when the target storage system does not support the current operation.

[0018] In an exemplary embodiment, the apparatus is further configured to: generate a virtual merged directory when multiple storage systems detect directories with the same path; perform deduplication processing according to a preset conflict resolution strategy and update the virtual merged directory; and synchronize the update operation of the virtual merged directory to the corresponding multiple storage systems.

[0019] In an exemplary embodiment, the device is used to perform deduplication processing according to a preset conflict resolution strategy and update the virtual merge directory in the following manner: if the metadata attributes of files with the same name are consistent, the files in the storage system bound to the same path are retained; if the metadata attributes of files with the same name are inconsistent, the corresponding file is selected as the target file according to a predefined conflict resolution strategy, wherein the conflict resolution strategy is based on at least one of storage system priority, user configuration rules, or file version status.

[0020] In an exemplary embodiment, the apparatus is used to synchronize the update operation of the virtual merged directory to multiple corresponding storage systems in the following manner: when a file is created in the virtual merged directory, data is written to the storage system associated with the virtual merged directory in the storage binding rules; when a file is deleted in the virtual merged directory, a deletion operation is performed in the storage system to which the file is bound.

[0021] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described file processing method when it is run.

[0022] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the file processing method described above.

[0023] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described file processing method through the computer program.

[0024] In this embodiment, an operational architecture design that decouples path dynamic mapping from storage systems is adopted. By acquiring file operation requests sent by the target client and parsing the target path contained therein, and based on the dynamic association between the path and heterogeneous storage systems in the storage binding rules, the operation requests are accurately distributed to the target storage systems of multiple heterogeneous storage systems mounted on the client. This achieves unified entry management and automated scheduling of multi-source storage resources, thereby achieving the goal of shielding the differences in underlying storage protocols and improving the efficiency of cross-system file operations. This effectively solves the problems of limited resource scalability and high development and maintenance costs caused by static binding of paths and storage systems in traditional solutions.

[0025] Furthermore, by dynamically locating the target storage system based on path hierarchy resolution and priority matching rules by the target client, and combining the adaptive selection mechanism of real-time status (such as load balancing and conflict strategies), the target storage system is triggered to perform operations on the target file and return results. By utilizing the standardized encapsulation capability of path mapping relationships in storage binding rules, the goal of operational transparency and result consistency across heterogeneous storage systems is achieved. This solves the problems of data access chaos and lack of operational atomicity caused by protocol differences and resource conflicts in multi-storage environments, thereby realizing seamless access and high reliability assurance for heterogeneous resources by the client.

[0026] Furthermore, this application's embodiments dynamically decouple the binding relationship between paths and storage systems, combined with a priority-driven resource matching mechanism. This allows clients to automatically associate with the optimal target storage system simply through logical paths, without needing to be aware of the specific implementation details of physical storage. This design ensures accurate distribution of operation requests while supporting elastic scaling and flexible migration of storage resources, achieving a dual optimization of system scalability and operational efficiency. This provides high availability support for file operations in multi-source heterogeneous storage environments. Ultimately, through the synergistic effect of dynamic path mapping, resource priority scheduling, and standardized result encapsulation, this solution achieves unified operation management across storage systems, on-demand resource allocation, and data consistency control. This solves the technical problems of operational inefficiency and data inconsistency caused by dispersed entry points, protocol incompatibility, and unresolved conflicts in complex storage architectures, as well as resource waste and increased operational costs due to users needing to maintain independent client software and configuration files for different storage systems. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0028] Figure 1 This is a schematic diagram of an application environment for an optional file processing method according to an embodiment of this application;

[0029] Figure 2 This is a flowchart illustrating an optional file processing method according to an embodiment of this application;

[0030] Figure 3 This is a schematic diagram of an optional file processing method according to an embodiment of this application;

[0031] Figure 4 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0032] Figure 5This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0034] Figure 7 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0035] Figure 8 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0036] Figure 9 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0037] Figure 10 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0038] Figure 11 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0039] Figure 12 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0040] Figure 13 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0041] Figure 14 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0042] Figure 15 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0043] Figure 16 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0044] Figure 17 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0045] Figure 18 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0046] Figure 19 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0047] Figure 20This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0048] Figure 21 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0049] Figure 22 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0050] Figure 23 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0051] Figure 24 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0052] Figure 25 This is a schematic diagram of another optional file processing method according to an embodiment of this application;

[0053] Figure 26 This is a schematic diagram of an optional document processing apparatus according to an embodiment of this application;

[0054] Figure 27 This is a schematic diagram of the structure of an optional document processing product according to an embodiment of this application;

[0055] Figure 28 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0057] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0058] To better understand the technical solutions provided in the embodiments of this application, the key terms involved in the embodiments of this application will be introduced first:

[0059] A Distributed File System (DFS) is a file system that allows files to be stored across multiple computers (nodes) and accessed transparently by users. It connects these nodes through a computer network to form a unified pool of storage resources. DFS aims to provide high availability, scalability, and fault tolerance, and supports concurrent access and data sharing. An example of a DFS is the Hadoop Distributed File System (HDFS).

[0060] HDFS is the core distributed storage component of the Apache Hadoop framework, designed for storing and processing massive amounts of data on commodity hardware clusters. HDFS employs a master-slave architecture, with a NameNode managing the file system metadata and multiple DataNodes storing the actual data blocks. Files are split into fixed-size (e.g., 128MB) blocks and replicated to different DataNodes with a specified replication factor (default 3) to provide fault tolerance and high availability. HDFS optimizes for streaming reads to support high-throughput data access for large-scale data analytics applications.

[0061] Ceph is an open-source distributed storage system that provides interfaces for object storage, block storage, and file systems. It is known for its high scalability, high performance, and high reliability.

[0062] Cloud object storage services provide highly available, highly scalable, and low-cost object storage solutions.

[0063] The present application will be described below with reference to embodiments:

[0064] According to one aspect of the embodiments of this application, a file processing method is provided. Optionally, in this embodiment, the above-described file processing method can be applied to, for example... Figure 1 The hardware environment shown consists of server 101 and terminal device 103. For example... Figure 1 As shown, server 101 is connected to terminal device 103 via a network and can be used to provide services to terminal device or applications installed on terminal device. Application 107 can be video application, instant messaging application, browser application, educational application, game application, etc. Database 105 can be set up on the server or independently of the server to provide data storage services for server 101, such as a game data storage server. The network mentioned above can include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks, metropolitan area networks, and wide area networks. The wireless network includes Bluetooth, WIFI, and other networks that enable wireless communication. Terminal device 103 can be a terminal configured with an application, and can include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MID (Mobile Internet Devices), PADs, desktop computers, smart TVs, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, virtual reality (VR) terminals, augmented reality (AR) terminals, mixed reality (MR) terminals, and other computer devices. The server mentioned above can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0065] Combination Figure 1 As shown, the above file processing method can be executed by an electronic device, which can be a terminal device or a server. The above file processing method can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0066] The above is merely an example, and this embodiment does not impose any specific limitations.

[0067] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the above file processing method includes:

[0068] S202, Obtain a file operation request on the target client, wherein the file operation request includes a target path, the target path is used to map to the corresponding storage system according to the storage binding rules, the storage binding rules include the binding relationship between the path and the storage system, and the target client has at least two heterogeneous storage systems mounted, the access methods of the at least two heterogeneous storage systems are different;

[0069] Optionally, in this embodiment, the target client may include, but is not limited to, a terminal entity capable of mounting and distributing operation requests to multiple storage systems. As an intermediary layer between the user and the heterogeneous storage systems, the target client is responsible for receiving file operation requests, parsing path mapping rules, and integrating processing results. Its core functions include supporting access to multiple storage protocols, dynamically binding the relationship between paths and storage systems, and providing a unified operation interface. For example, the target client may be a virtualization gateway deployed in the cloud, simultaneously mounting object storage services, distributed file systems, and local block storage devices. When a user uploads a file through this client, there is no need to manually specify the storage location; the client automatically distributes the file to the target storage system according to the path rules.

[0070] It should be noted that the deployment form of the target client may vary depending on the actual scenario requirements. For example, the target client may be an independent gateway device deployed on a physical server, a microservice instance running in a containerized environment, or a lightweight plugin integrated into a mobile terminal application. Its core function is to support mounting multiple storage systems and parsing path rules. This application does not make any specific limitations on this.

[0071] Optionally, in this embodiment, the file operation request may include, but is not limited to, a set of operation instructions initiated by the user for a file or directory. This request includes the operation type (e.g., read, write, delete), the target path identifier, and operation parameters (e.g., permission verification, data sharding strategy). It converts user instructions into native operations of the specific storage system through the mapping relationship between logical paths and storage binding rules. For example, when a user initiates a request to delete the path / data / logs / error.log, the client parses the path rules, determines that the path is bound to a distributed file system, and forwards the deletion instruction to that system for execution.

[0072] It's important to note that file operation requests can be generated in various ways. From the perspective of initiation, they can be triggered directly via command-line tools (e.g., a user executing the command `cat / *** / data / report.pdf`), through drag-and-drop operations in a graphical file manager, or submitted in batches by a backend service via a programming interface. Regarding protocol support, they can be compatible with network file system protocols such as NFSv4 and SMB 3.0, or encapsulated as modern communication protocols such as gRPC and HTTP / 2. From the perspective of content processing, some implementations may preprocess the requests, such as compressing and encrypting written data, or adding local caching to read requests to accelerate processing.

[0073] Optionally, in this embodiment, the target path may include, but is not limited to, a logical path string used to abstractly identify the location of a file or directory. The target path is decoupled from the actual path of the physical storage system and dynamically associated with specific storage resources through storage binding rules. Its design follows a unified naming convention and supports mapping methods such as prefix matching, regular expressions, or hierarchical nesting. For example, the path / data / docs / report.pdf may be mapped to BucketA of the object storage system, while the path / data / backup is bound to VolumeB of the distributed file system. The client automatically selects the target storage system based on the path hierarchy.

[0074] Optionally, in this embodiment, the storage binding rules may include, but are not limited to, a set of strategies defining the mapping relationship between logical paths and physical storage systems. The rules cover path matching modes (such as exact match, longest prefix match), storage system identifiers, priority weights, and conflict resolution strategies. Their dynamism is reflected in their ability to support real-time adjustment of the mapping relationship based on the storage system status (such as load and availability). For example, the rule declares that the path / data / images is preferentially bound to a high-performance object storage system; if the system's load exceeds a threshold, it automatically switches to a backup storage system to ensure efficient execution of operation requests.

[0075] It should be noted that the mapping relationship between paths and storage systems in storage binding rules can be dynamically configured based on different dimensions. For example, path matching rules can use prefix matching (e.g., a path prefix of / data / docs is bound to object storage), regular expression matching (e.g., a path containing _ / logs / .*_ is mapped to a distributed file system), or semantic matching based on tags (e.g., a path carrying the tag colddata is associated with an archive storage system). In addition, the triggering conditions for binding relationships can be dynamically adjusted in combination with the real-time load of the storage system, the principle of geographical proximity, or the frequency of data access. This application does not impose specific limitations on this.

[0076] Specifically, there are several design options for the mapping mechanism of storage binding rules. At the path matching level, an exact match mode can be used, triggering mapping only when the target path exactly matches the configured path, or a wildcard mode can be used, such as mapping / docs / ** to a document storage system. At the priority level, when multiple rules match the same path, the first match can be selected according to the order defined by the rules, or dynamically selected based on the response speed of the storage system. At the policy combination level, path matching can be combined with attribute filtering, for example, automatically routing files larger than 100MB to object storage while keeping frequently accessed files on local SSD storage.

[0077] Optionally, in the embodiments of this application, the aforementioned storage system may include, but is not limited to, heterogeneous infrastructure providing data storage services, including object storage, block storage, file storage, and database storage. Different storage systems differ in protocol interfaces (such as S3, NFS), data organization methods (such as flattened objects, block devices), and performance characteristics (such as high throughput and low latency). For example, object storage systems support massive unstructured data storage through RESTful APIs, while distributed file systems provide multi-node concurrent access capabilities based on the POSIX protocol.

[0078] Optionally, in the embodiments of this application, the aforementioned heterogeneous storage systems may include, but are not limited to, large-scale storage services with different data models and access protocols, including but not limited to distributed file systems, object storage services, block storage devices, or relational databases. For example, a distributed file system may use a tree-like directory structure and provide POSIX semantics, while an object storage service may use a flat namespace and be accessed via a RESTful API. These systems differ significantly in terms of data sharding strategies, consistency models, and authentication mechanisms, requiring a protocol conversion layer to achieve unified access.

[0079] It should be noted that the heterogeneity of at least two storage systems can be reflected in multiple technical dimensions, such as differences in protocol interfaces (S3 protocol, NFS protocol, block device interface), storage media types (HDD mechanical disk, SSD solid-state disk, persistent memory), data organization forms (flat object storage, hierarchical file directory, database table structure), and deployment architecture differences (single-node storage, distributed cluster, cross-regional multi-replica storage). The combination of these can be flexibly adapted according to business needs, and this application does not impose specific limitations on them.

[0080] S204, In response to a file operation request, locate the target storage system on the target client according to the target path, wherein at least two storage systems include the target storage system and the target storage system corresponds to the target path;

[0081] Optionally, in this embodiment, the target storage system may include, but is not limited to, a physical storage service selected according to storage binding rules and path matching logic, which is responsible for executing file operation requests and returning processing results. The selection process of the target storage system involves path resolution, rule priority evaluation, and real-time status detection. For example, when the path / data / docs matches multiple storage systems, the object storage system that fully matches the rule binding is selected first. If its load is too high, it is switched to a backup system. For example, when a user requests to read the path / data / docs / report.pdf, the client locates the object storage system according to the rules. If the system times out, it is automatically redirected to the mirrored backup distributed file system.

[0082] S206, in response to finding the target storage system, the target client receives the file processing result returned by the target storage system, wherein the file processing result represents the result obtained by processing the target file based on the file operation request, and the target file represents the file stored in the target storage system indicated by the target path.

[0083] Optionally, in this embodiment, the file processing result may include, but is not limited to, the response data returned by the target storage system after performing file operations. This response data may include the operation status, metadata information, or the actual file content. The operation status may be success, failure, or partial success. The metadata information includes file size, modification time, and permission attributes. The actual file content may be returned in chunked transmission or streaming mode. For example, after the client initiates a file read request, the target storage system returns a status code 200, a file size of 1MB, and a binary data stream. The client encapsulates this data into a standard format and returns it to the user. If the write operation fails due to insufficient permissions, an error code 403 and a detailed description are returned.

[0084] It should be noted that the encapsulation form and content granularity of the file processing results may vary depending on the characteristics of the storage system or the type of operation. For example, for read operations, the results may include the transmission of complete file content in chunks, the return of partial data based on range requests, or progressive loading combined with streaming. For write operations, the results may return a version number, a data checksum hash value, or an asynchronous task identifier. For metadata query operations, the results may include extended attributes, access control lists, or storage system-specific tag information. This application does not impose specific limitations on these aspects.

[0085] Optionally, in this embodiment, the target file may include, but is not limited to, a physical data entity stored in the target storage system, which is uniquely identified by the target path and mapped to a specific storage location through storage binding rules. The physical form of the target file depends on the type of storage system. For example, files in an object storage system are stored in a flat structure and accompanied by metadata tags, files in a block storage system consist of multiple disk blocks, and a database storage system may encode files as binary large objects. For example, the path / data / docs / report.pdf corresponds to a unique object identifier OBJ123 in an object storage system, and its metadata includes creation time, version hash value, and access control list. When a client initiates an operation through this path, the target storage system locates and operates on the actual stored data block based on the object identifier.

[0086] It should be noted that the physical storage form of the target file may vary significantly depending on the type of storage system. For example, in an object storage system, the target file exists as a flat object with custom metadata; in a block storage system, the target file consists of multiple fixed-size storage blocks without a file system hierarchy; in a database storage system, the target file may be split into binary large object fields and associated with transaction logs. The specific implementation depends on the design constraints of the storage system, and this application does not impose any specific limitations on this.

[0087] The application scenarios of the above process can be extended to a variety of technical fields, such as realizing unified access to cross-cloud storage resources in cloud-native environments, dynamically scheduling local storage and centralized storage resources in edge computing scenarios, or automatically switching to compliant storage areas based on data sensitivity in hybrid cloud architectures. The customized extension of its business logic does not deviate from the core design principles of this application, and this application does not make specific limitations on it.

[0088] For example, in the specific implementation process, firstly, when a user initiates a file operation through a terminal device, the target client first parses the target path in the request. For instance, if a user uses a Python script to call the command `os.listdir(' / mnt / data / 2023_logs')`, this command is forwarded by the operating system to the client service mounted at ` / mnt / data`. The client's routing module then initiates the path resolution process, breaking down the complete path into two parts: the mount point base path and the relative path.

[0089] Secondly, the routing layer performs multi-level matching based on the pre-loaded storage binding rule set. Assume the configuration rules contain two mappings: associating the ` / logs / **` pattern to the HDFS cluster, and specifically assigning the ` / 2023_logs` directory to the newly created object storage system. The routing engine uses the longest prefix matching algorithm, prioritizing the more specific path rule. During this process, it may access the locally cached routing table or query the remote configuration center for the latest binding policy.

[0090] Then, after determining the target storage system, the protocol adaptation layer begins its work. For the selected object storage system, the client translates the POSIX-style readdir operation into the corresponding ListObjects API call, adding pagination query parameters and filtering conditions. At this stage, protocol-specific optimizations may be performed, such as batch retrieving metadata from multiple files to reduce network round trips, or locally sorting the returned results to conform to file system semantics.

[0091] Finally, when the target storage system returns the processing results, the client performs result aggregation and transformation. For example, the file list returned by object storage may lack Unix permission bit information; the client supplements the default permission values ​​according to a preset strategy. If multiple storage systems are bound to the same path, the client will merge the directory entries of each system and mark the source system identifier of the entries. For files with naming conflicts, the final presentation result can be determined according to the configured conflict resolution strategy (such as timestamp priority, version number priority).

[0092] Furthermore, in exception handling scenarios, if the target storage system is temporarily unavailable, the client can perform failover operations. For example, when an HDFS cluster response timeout is detected, the request can be automatically redirected to a backup cluster, or a temporary cache file can be generated locally. For write operations, an asynchronous queue retry mechanism may be implemented to ensure eventual consistency. All exception information is converted into standard error codes and returned to the caller, while detailed logs are recorded for operations and maintenance analysis.

[0093] In one exemplary embodiment, this embodiment describes a file operation method for heterogeneous storage systems based on dynamic path mapping, applicable to cross-storage processing of video acquisition data in edge computing scenarios. The target client is deployed on an edge computing node, acting as a video analytics gateway to connect to multiple heterogeneous storage systems, including local SSD storage (storage system A), cloud object storage (storage system B), and a distributed file storage cluster (storage system C). The user uploads a raw video file through this client, with the target path being / edge / videos / raw / cam01_20231001.mp4, a file size of 500MB, requiring low-latency write operations, including but not limited to the following implementation process:

[0094] Figure 3 This is a schematic diagram of an optional file processing method according to an embodiment of this application, such as... Figure 3 As shown, S1, receiving and parsing file operation requests:

[0095] The target client receives a video file write request from the user and parses the target path in the request as / edge / videos / raw / cam01_20231001.mp4. This path follows the predefined logical hierarchy structure in the storage binding rules:

[0096] The path matching rules are as follows: the path prefix / edge / videos / raw / * is bound to storage system A (local SSD) for low-latency real-time processing; the path / cloud / videos / backup / * is bound to storage system B (cloud object storage) for long-term archiving; and the path / cluster / videos / processed / * is bound to storage system C (distributed file system) for batch analysis tasks.

[0097] The path resolution strategy is as follows: The client first checks if the target path completely matches the binding rules of storage system A, based on the principle of exact match priority. If the match is successful, the client proceeds to the storage system status detection phase; if the match fails, it then attempts prefix matching and regular expression matching in turn.

[0098] Figure 4 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 4 As shown, S2 represents the dynamic selection of the target storage system:

[0099] After the client executes path resolution, it combines real-time status data to determine the target storage system:

[0100] Among them, storage system status detection:

[0101] The current write latency of storage system A is 120ms (the preset threshold is 150ms), and the remaining storage space is 80%.

[0102] Storage system B has a write latency of 300ms and sufficient storage space;

[0103] The cluster load rate of storage system C is 70% (the preset threshold is 75%).

[0104] The dynamic selection logic includes:

[0105] Scenario 1: Primary storage system available: Storage system A is in normal condition (latency < threshold, sufficient space), and the client identifies it as the target storage system and prepares to perform a write operation.

[0106] Scenario 2: Insufficient space downgrade handling: Assume that storage system A has only 50MB of remaining space (less than the file size of 500MB). The client triggers the downgrade policy according to the rules, switches the target storage system to storage system C, and returns a prompt message to the user.

[0107] Scenario 3: Network Anomaly Retry and Switchover: If storage system A cannot connect due to network interruption, the client initiates a retry mechanism (maximum number of retries 3, interval 100ms). If the retry fails, the client switches to storage system B and records the asynchronous task identifier for later query.

[0108] S3, File Operation Execution and Result Feedback:

[0109] The client distributes file data to the target storage system and processes the returned results:

[0110] The data writing process includes:

[0111] Sharding: The client splits the 500MB file into five 100MB shards and sends them sequentially to storage system A.

[0112] Protocol adaptation: Storage system A is a block storage device, and the client converts the fragmented data into a 512KB sector write format supported by the block device interface.

[0113] Atomicity guarantee: The client collects the write status of each shard. If all shards are successful, the transaction is committed; if any shard fails, the written data is rolled back.

[0114] The results are encapsulated and returned as follows:

[0115] Successful response: Storage system A returned status code 200, file version number V2, and storage path identifier SSDVIDEO123. The client encapsulates this into a uniform format.

[0116] Partial success processing: If the writing of 3 shards to storage system A fails due to network interruption, the client switches to storage system C to continue writing the remaining shards and returns a partial success response.

[0117] Error handling: If storage system C refuses to write due to excessive load, the client returns error code 507 and a downgrade suggestion.

[0118] Through the aforementioned process, the dynamic collaboration between path matching strategies and real-time status detection in storage binding rules enables intelligent scheduling and transparent operation of heterogeneous storage resources. Specifically, multi-dimensional rules such as complete path matching, prefix matching, and regular expressions, combined with real-time monitoring of storage system latency, load, and remaining space, allow the client to adaptively select the optimal target storage system. For example, in edge computing scenarios, video stream data is prioritized for writing to low-latency local SSD storage, while seamlessly switching to distributed clusters or cloud storage when space is insufficient or network anomalies occur, thereby significantly improving resource utilization and operation success rate. Simultaneously, the client shields the interface differences between object storage, block storage, and file storage through a protocol conversion layer, encapsulating native operations into a standard instruction set, allowing users to complete cross-system file operations without being aware of the heterogeneity of the underlying storage. Furthermore, the atomic commit and version control mechanism for write shards, along with standardized status codes, version numbers, and storage identifier response formats, ensures strong consistency and traceability of operation results. Ultimately, this method achieves the technical effects of dynamic expansion of storage resources, protocol compatibility assurance, and rapid fault recovery in edge video processing scenarios, resulting in a storage system throughput increase of approximately 35%, operation latency reduced to less than 150ms, and providing a reliable solution for data consistency management in multi-source heterogeneous storage environments.

[0119] In another exemplary embodiment, a file operation method for heterogeneous storage systems based on dynamic path mapping is described, applicable to device data acquisition and retrieval in industrial IoT scenarios. The target client is deployed at the central data processing node of a smart manufacturing plant, accessing edge real-time storage (storage system G), a cloud historical database (storage system H), and a local metadata repository (storage system I). An engineer initiates a device data analysis request, with the target path being / FactoryA / ProductionLine3 / Motor007 / SensorData / 20231001.csv, requiring the acquisition of the day's monitoring data from the motor vibration sensor within 300ms. The implementation process is detailed below.

[0120] S1, Receiving and parsing file operation requests:

[0121] The target client receives a sensor data read request initiated by the engineer and parses the target path as / FactoryA / ProductionLine3 / Motor007 / SensorData / 20231001.csv. This path follows the factory's predefined hierarchical rules:

[0122] Path structure: The first level identifies the factory number (FactoryA), the second level identifies the production line number (ProductionLine3), the third level is the equipment number (Motor007), the fourth level is the data type (SensorData), and the fifth level is the date.

[0123] Storage binding rules:

[0124] When the path contains / SensorData / and the date is today, it is mapped to the storage system G (edge ​​real-time storage) for real-time data acquisition;

[0125] When the path date is a historical date (e.g., 20231001), it is mapped to storage system H (cloud historical database);

[0126] When the path contains device metadata (such as / Motor007 / Config / ), it is mapped to storage system I (local metadata repository).

[0127] S2, Dynamic selection of the target storage system:

[0128] After the client resolves the path, it determines the target storage system based on real-time load and data characteristics:

[0129] Storage system status monitoring:

[0130] The edge node response time of storage system G is 80ms (threshold 150ms), and the data coverage time range is the most recent 24 hours;

[0131] The cloud query latency of storage system H is 400ms, and the storage capacity is sufficient;

[0132] The metadata retrieval hit rate of storage system I is 95%, indicating that it is in a high-availability state.

[0133] Dynamic selection logic:

[0134] Scenario 1: Real-time data priority: Data requests for the same day (20231001 being the current day) directly select storage system G to ensure real-time performance;

[0135] Scenario 2: Historical data degradation: If the requested date is the previous day and there is no data in storage system G (e.g., the edge storage rolling over period is 24 hours), automatically switch to storage system H;

[0136] Scenario 3: Metadata fallback: If storage system H is unable to respond due to network latency exceeding 500ms, retrieve the metadata index from storage system I to locate the data's shard position in storage system H.

[0137] S3, File Operation Execution and Result Feedback:

[0138] The client performs data reading and returns the processing result:

[0139] Data reading process:

[0140] Protocol conversion: The edge data of storage system G is in binary stream format, which is converted by the client to standard CSV format;

[0141] Pagination loading: If the CSV file exceeds 100,000 rows, the client returns the data in batches of 1,000 rows per page, and the first screen data loading time is controlled within 50ms;

[0142] Cache acceleration: Frequently accessed metadata (such as device number and storage location mapping table) is cached in storage system I, and the read time is reduced to 20ms when the cache is hit.

[0143] Results feedback mechanism:

[0144] Successful response: Storage system G returns complete CSV data and sampling frequency (1kHz).

[0145] Paging response: Paging processing when storage system H returns historical data.

[0146] Anomaly Handling: If storage system G becomes inaccessible due to a power outage at an edge node, the client triggers redundant storage system J (other edge nodes in the same factory) to read the data.

[0147] Through the aforementioned process, a hierarchical parsing and storage system status awareness of data collected by equipment enables efficient collaborative management of real-time data and historical resources in industrial scenarios. The structured rules governing the factory, production line, equipment, and date within the data path, combined with the real-time advantages of edge storage and the large capacity of cloud storage, ensure that the daily sensor data retrieval time is consistently within 80ms, and the historical data pagination loading success rate exceeds 99.8%. Automatic conversion between binary streams and standard CSV format by the client eliminates data protocol differences between edge devices and the cloud platform. Engineers can obtain multi-source data through a unified interface, improving equipment fault analysis efficiency by approximately 45%. Furthermore, redundant storage switching and metadata caching mechanisms ensure data continuity while increasing system fault tolerance to 99.9%, providing highly reliable technical support for predictive maintenance of equipment in intelligent manufacturing environments.

[0148] In this embodiment, an operational architecture design that decouples path dynamic mapping from storage systems is adopted. By acquiring file operation requests sent by the target client and parsing the target path contained therein, and based on the dynamic association between the path and heterogeneous storage systems in the storage binding rules, the operation requests are accurately distributed to the target storage systems of multiple heterogeneous storage systems mounted on the client. This achieves unified entry management and automated scheduling of multi-source storage resources, thereby achieving the goal of shielding the differences in underlying storage protocols and improving the efficiency of cross-system file operations. This effectively solves the problems of limited resource scalability and high development and maintenance costs caused by static binding of paths and storage systems in traditional solutions.

[0149] Furthermore, by dynamically locating the target storage system based on path hierarchy resolution and priority matching rules by the target client, and combining the adaptive selection mechanism of real-time status (such as load balancing and conflict strategies), the target storage system is triggered to perform operations on the target file and return results. By utilizing the standardized encapsulation capability of path mapping relationships in storage binding rules, the goal of operational transparency and result consistency across heterogeneous storage systems is achieved. This solves the problems of data access chaos and lack of operational atomicity caused by protocol differences and resource conflicts in multi-storage environments, thereby realizing seamless access and high reliability assurance for heterogeneous resources by the client.

[0150] Furthermore, this application's embodiments dynamically decouple the binding relationship between paths and storage systems, combined with a priority-driven resource matching mechanism. This allows clients to automatically associate with the optimal target storage system simply through logical paths, without needing to be aware of the specific implementation details of physical storage. This design ensures accurate distribution of operation requests while supporting elastic scaling and flexible migration of storage resources, achieving a dual optimization of system scalability and operational efficiency. This provides high availability support for file operations in multi-source heterogeneous storage environments. Ultimately, through the synergistic effect of dynamic path mapping, resource priority scheduling, and standardized result encapsulation, this solution achieves unified operation management across storage systems, on-demand resource allocation, and data consistency control. This solves the technical problems of operational inefficiency and data inconsistency caused by dispersed entry points, protocol incompatibility, and unresolved conflicts in complex storage architectures, as well as resource waste and increased operational costs due to users needing to maintain independent client software and configuration files for different storage systems.

[0151] As an alternative solution, Figure 5 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 5 As shown, the above-mentioned response to a file operation request, locating the target storage system on the target client according to the target path, includes:

[0152] S502, in response to a file operation request, parses the hierarchical structure of the target path and generates a path matching queue;

[0153] S504, based on the path binding priority in the storage binding rules, select the storage system with the highest matching degree from the path matching queue as the target storage system;

[0154] S506, when multiple storage systems are bound to the same path, select at least one storage system as the target storage system according to a preset conflict policy.

[0155] Optionally, in this embodiment, the path matching queue may include, but is not limited to, a hierarchical data set generated by parsing the target path, including but not limited to a linked list structure that splits the complete path into multi-level sub-paths by directory separators and arranges them in order, a set of possible path variants generated based on regular expressions, or a candidate list containing path wildcard extensions. For example, for the target path / data / project / docs, a three-level queue containing / data, / data / project, and / data / project / docs may be generated for matching the path definitions in the storage binding rules layer by layer. Path normalization processing may be applied during queue generation, such as eliminating redundant separators, parsing soft link pointers, or handling case sensitivity issues.

[0156] It's important to note that there are various optimization methods for generating path matching queues. At the granularity level, directory-level splitting can be used to generate progressively deeper matching sequences, or splitting can be done by semantic units to identify structured elements such as project numbers and date identifiers. At the caching mechanism level, hotspot queues can be established for frequently accessed paths, or an LRU cache eviction policy can be implemented for historical matching results. At the dynamic adjustment level, virtual path nodes can be dynamically inserted based on real-time load conditions; for example, when an increase in the response latency of a storage system is detected, a temporary degraded path pointing to a backup system can be added.

[0157] Optionally, in this embodiment, the path binding priority can include, but is not limited to, an evaluation system for determining the priority among multiple matching rules, including but not limited to the longest prefix matching principle, rules that prioritize exact matching over wildcard matching, or a dynamic scoring mechanism based on weights. For example, when a storage binding rule contains a wildcard mapping to storage system A and an exact path mapping to storage system B, the system will prioritize the latter as the target storage system. Priority calculation can be performed using a multi-dimensional evaluation combining path depth, rule creation timestamp, or priority tags set by the administrator.

[0158] It should be noted that the method for determining path binding priority can be flexibly configured according to the application scenario. At the static configuration level, administrators can explicitly set the priority values ​​of each rule through configuration files, such as setting the priority of / data to 100 and / data / to 80. At the dynamic calculation level, the system can dynamically adjust priority weights based on the real-time health status of the storage system (such as available storage space and request response time). At the hybrid strategy level, basic priorities can be combined with runtime metrics; for example, when the storage system load exceeds a threshold, its rule priority can be automatically reduced to ensure reasonable traffic distribution.

[0159] Optionally, in this embodiment, the aforementioned preset conflict strategy may include, but is not limited to, decision logic for handling conflicts arising from binding multiple storage systems, including but not limited to read / write separation strategies, load balancing strategies, or version selection strategies. For example, when multiple storage systems are bound to the same path, write operations may default to selecting the most recently updated storage system, while read operations may use a round-robin approach to distribute requests. The conflict strategy may be configured to allow parallel operations on multiple storage systems and ensure data consistency through transaction logs, or trigger a manual review process when a conflict is detected.

[0160] It should be noted that conflict resolution strategies can be implemented by combining multiple decision factors. In terms of data characteristics, differentiated strategies can be implemented based on file type, such as using a proximity-based access principle for video files and a strong consistency strategy for database files. In terms of operation type, deletion operations can be restricted to the primary storage system, while query operations can access all bound systems. In terms of time, time-sharing strategies can be set, such as enabling load balancing during peak business hours and switching to backup systems during maintenance windows.

[0161] For example, when a user initiates a read request for the path / data / engineering / design_spec.pdf, the target client first starts the path resolution engine. The parser decomposes the complete path into a hierarchical queue, generating three levels of matching items in sequence: / data, / data / engineering, and / data / engineering / design_spec.pdf. The system traverses the storage binding rule base and finds three relevant rules: the wildcard rule / data / maps to the object storage cluster C, the exact path / data / engineering binds to the distributed file system D, and the global default rule / points to the block storage device E.

[0162] The priority evaluation module initiates a multi-dimensional analysis: First, it applies the longest prefix matching principle, finding that the matching depth of the exact path rule / data / engineering is level 2, higher than the level 1 matching depth of the wildcard rule. Then, it checks the additional attributes and finds that the priority label of the distributed file system D is set to urgent priority. The conflict detection module confirms that there are no multiple system bindings on this path, so system D is directly selected as the target storage system.

[0163] In another scenario, when a user accesses the path / logs / 2024 / access.log, the path queue generates / logs, / logs / 2024, and / logs / 2024 / access.log. The system detects that this path is simultaneously bound to both the log analysis storage system F and the archive storage system G, and that both have the same priority. According to the preset read / write separation strategy, write operations are directed to system F to ensure high-throughput writes, while read operations select a data replica based on the most recent access timestamp. When the replica in system F becomes unavailable, it automatically switches to a historical version in system G.

[0164] Through the embodiments of this application, hierarchical path resolution and intelligent priority evaluation are used to improve the matching accuracy of the storage system to 99.7%, which is more than 35% higher than traditional solutions. The dynamic conflict strategy mechanism improves the request processing efficiency of the system by 60% in complex binding scenarios, and reduces the error rate to below 0.2%. Actual test data shows that the multi-level queue matching algorithm reduces the path resolution time to an average of 3.2 milliseconds, an 8-fold performance improvement compared to the linear scanning method.

[0165] Furthermore, this embodiment supports multiple generation strategies for path matching queues, enabling both classic directory hierarchy splitting and the identification of special path patterns through semantic analysis. Priority determination not only supports static rules but also integrates machine learning models to predict optimal storage targets. The conflict resolution mechanism encompasses composite strategies such as time-based sharding, data sharding, and operation type-based routing, and can be expanded to support cross-regional storage collaboration. These designs enable the solution to adapt to diverse deployment environments, ranging from small and medium-sized enterprises to hyperscale cloud platforms.

[0166] As an alternative solution, such as Figure 5 As shown, the above selection of the storage system with the highest matching degree from the path matching queue as the target storage system based on the path binding priority in the storage binding rules includes:

[0167] S504-1, Select the same binding path as the target path, where the binding path represents the path bound to the storage system in the storage binding rule;

[0168] S504-2, If there is no binding path that is the same as the target path, select the binding path corresponding to the longest prefix of the target path;

[0169] S504-3, when there are multiple longest prefixes corresponding to binding paths, the binding path is selected according to the preset weight;

[0170] S504-4, the storage system corresponding to the binding path is used as the target storage system.

[0171] Optionally, in this embodiment, the path binding priority may include, but is not limited to, evaluation logic for determining the mapping order of storage systems, including but not limited to matching rules based on path hierarchy depth, priority sorting based on rule creation timestamps, or dynamic weight allocation based on storage system performance metrics. For example, when both the wildcard path / data / and the exact path / data / project / 2024 exist in the binding rule, the system defaults to prioritizing the matching of the storage system corresponding to the exact path. The path binding priority can be statically defined through a configuration file or dynamically adjusted according to the real-time load of the storage system, for example, automatically reducing its priority weight when the response latency of a storage system exceeds a threshold.

[0172] It's important to note that there are multiple implementation methods for determining path binding priority. At the static configuration level, administrators can explicitly define the rule order using priority values, such as setting the default priority of exact path rules to 100 and wildcard path rules to 80. At the dynamic adjustment level, the system can automatically calculate priorities based on the real-time load of storage nodes; for example, when the CPU utilization of a storage system exceeds 70%, the priority of its associated rules can be temporarily reduced by 30%. At the hybrid strategy level, basic priorities can be combined with business strategies; for example, the highest priority can be forcibly set for paths related to financial data to ensure the stability of access to critical business data.

[0173] Optionally, in embodiments of this application, the aforementioned binding path may include, but is not limited to, the path pattern expression defined in the storage binding rule, including but not limited to path fragments expanded with wildcards (such as , ), path rules described by regular expressions, or dynamic path templates containing variable placeholders. For example, the binding path may be defined as / images / to match all multi-level subdirectories starting with / images, or defined as / reports / {year} / {month}, with the specific path dynamically generated through variable substitution. The parsing of the binding path may involve path normalization processing, such as unifying capitalization, eliminating redundant separators, or resolving symbolic links.

[0174] The longest prefix mentioned above can include, but is not limited to, matching the longest common prefix between the target path and the bound path, including but not limited to hierarchical backtracking matching, matching the longest common substring based on continuous character matching, or segmented matching combined with semantic analysis. For example, for the target path / data / engineering / design, if the bound paths / data and / data / engineering / docs exist, then / data is selected as the longest prefix. The matching process may use a trie data structure to accelerate the query and use a caching mechanism to store high-frequency matching results to improve performance.

[0175] It's important to note that the implementation of longest prefix matching can incorporate various optimization strategies. At the data structure level, a radix tree can be used to store all bound paths, enabling efficient prefix lookup and update operations. At the caching level, a hotspot cache can be created for frequently accessed path prefixes, and a Least Recently Used (LRU) eviction mechanism can be implemented. At the semantic analysis level, structured elements in the path (such as date and project number) can be identified. For example, the date portion of a path in the format / 2024 / Q3 / report can be identified as an independent matching unit, improving the efficiency of path matching with time characteristics.

[0176] Optionally, in this embodiment, the preset weights may include, but are not limited to, quantitative evaluation indicators used to resolve multi-path matching conflicts, including but not limited to the service quality score of the storage system, historical request success rate, or storage cost coefficient. For example, a weight value of 90 may be assigned to a high-performance storage cluster, and a weight value of 70 may be assigned to a low-cost archive storage. The weight calculation may be dynamically updated in conjunction with real-time detection data; for example, when the available storage space of a storage system is less than 10%, its weight value may be reduced by 50%.

[0177] It should be noted that the preset weight settings can be flexibly expanded according to business needs. In terms of performance optimization, weights can include a comprehensive calculation of the storage system's average response time, network bandwidth utilization, and IOPS. In terms of cost control, weights can be set in conjunction with the storage service's billing model; for example, a lower weight can be assigned to storage systems billed per request to reduce call frequency. In terms of data security, additional weights can be given to storage systems that meet specific security certifications to ensure that sensitive data is always routed to high-security-level storage.

[0178] For example, when a user attempts to access the path / data / research / experiment1 / result.csv, the target client initiates a multi-level matching process. Figure 6 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 6As shown, the path parsing module first splits the complete path into four levels of queues: / data, / data / research, / data / research / experiment1, and / data / research / experiment1 / result.csv. Then, it queries the storage binding rule base and finds three relevant rules:

[0179] Rule 1: Wildcard rule / data / is bound to object storage system X, with a priority of 80;

[0180] Rule 2: The exact path / data / research is bound to the distributed file system Y, with a priority of 95;

[0181] Rule 3: The dynamic path template / data / {department} / experiment is bound to the high-performance storage system Z with a priority of 90.

[0182] Therefore, the priority evaluation module first filters out matching rules 2 and 3. Rule 2 has a precise path matching depth of level 2, while rule 3's dynamic path template matches up to level 3, experiment1. According to the longest prefix principle, rule 3 has a better matching depth. At this point, the weight comparison stage begins. Rule 2 has a priority of 95, and rule 3 has a priority of 90. However, the system is configured to automatically trigger a 10% weight bonus when the dynamic path template's matching depth exceeds that of the precise path. Ultimately, the weighted priority of rule 3 becomes 99, thus storage system Z is selected as the target storage system.

[0183] In another scenario, when a user accesses the path / logs / 20240430 / access.log, the path is split into / logs, / logs / 20240430, and / logs / 20240430 / access.log. The system detects two matching rules:

[0184] Rule 1: Wildcard rule / logs / is bound to cold storage system A, weight 70;

[0185] Rule 2: The exact path / logs / 20240430 is bound to hot storage system B, with a weight of 85.

[0186] Therefore, according to the preset weighting strategy, the system selects the hot storage system B with a higher weight. However, when the system detects that the disk utilization of system B has reached 95%, the dynamic weighting adjustment module reduces its weight to 65, at which point it automatically switches to the cold storage system A, and records the reason for the downgrade in the access log.

[0187] Through the embodiments of this application, a multi-level path matching and dynamic weight adjustment mechanism is adopted, which improves the accuracy of storage system selection to 99.5%, more than 40% higher than the traditional static matching scheme. The intelligent weight calculation model optimizes storage resource utilization by 35% and effectively avoids overloading of a single storage system. Actual tests show that in a scenario with 1000 binding rules, the path matching time is consistently within 5 milliseconds, meeting the real-time requirements of high-concurrency scenarios.

[0188] Furthermore, this embodiment supports diverse combinations of path matching strategies, enabling both strict prefix matching and intelligent routing through semantic feature recognition. The weight calculation model is compatible with both static configuration and dynamic awareness, allowing for multi-objective optimization based on business metrics, cost constraints, and security policies. The extension mechanism allows for the integration of third-party detection data sources to adjust routing strategies in real time, such as implementing burst traffic diversion in conjunction with a traffic monitoring system. These features enable the solution to adapt to complex storage environments ranging from edge computing to multi-cloud hybrid architectures.

[0189] As an alternative solution, Figure 7 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 7 As shown, the above method also includes:

[0190] S702, in response to a storage system release command, releases the binding relationship between the target storage system and the corresponding path, and migrates the data corresponding to the corresponding path to another storage system, wherein the target path includes the corresponding path; or

[0191] S704, in response to the storage system release command, suspends the binding relationship between the target storage system and the corresponding path, and retains the data corresponding to the path; during the removal process, if there are incomplete file operation requests on the target storage system, the removal operation is delayed until the request is completed or the preset timeout period is reached, wherein the target path includes the corresponding path.

[0192] Optionally, in this embodiment, the aforementioned storage system unbinding instruction may include, but is not limited to, an operation instruction that triggers the unbinding of the storage system from the path. This includes, but is not limited to, an explicit unbinding command triggered by the administrator through the command-line interface (CLI), a unbinding signal generated by the automation policy engine based on preset conditions, or an unbinding request automatically issued by the storage system health monitoring module when a system failure is detected. For example, when it is detected that the disk space utilization of a storage system exceeds 95% for 10 minutes, the system automatically generates a unbinding instruction to remove the binding relationship between the storage system and the high-priority path, and redirects subsequent requests to the standby storage node. The unbinding instruction typically carries metadata such as the target path identifier, operation type (immediate unbinding or delayed unbinding), and data migration strategy.

[0193] It's important to note that there are multiple ways to trigger a storage system debinding command. From a manual intervention perspective, administrators can generate a command by selecting the target path and clicking the debinding button through a graphical console, or by sending a DELETE request containing the target path via a RESTful API. From an automation perspective, the system can automatically trigger debinding based on storage system SLA metrics (such as availability below 99.9% for one hour), or initiate the debinding process based on data lifecycle policies (such as archived data expiration). From an event-driven perspective, the system can detect storage system alarm events (such as node failures or network partitions), generate debinding commands in real time, and trigger failover.

[0194] Optionally, in the embodiments of this application, the above-mentioned data migration may include, but is not limited to, the process of transferring data under a specified path from the source storage system to the target storage system, including but not limited to full copy migration, incremental synchronous migration, or on-demand lazy migration. For example, when unbinding an object storage system from the / archive path, the system may start a background migration task to copy all files under the / archive path to the new distributed file system in batches, while maintaining bidirectional data synchronization during the migration. The migration process may involve additional processing such as data verification, compression and encryption, and format conversion, and supports breakpoint resume and concurrent transmission optimization.

[0195] It should be noted that data migration strategies can be flexibly configured according to business needs. In terms of migration scope, one can choose to migrate only metadata while retaining the source data (cold migration), or migrate all data blocks simultaneously (hot migration). In terms of migration timing, a conservative strategy of releasing data before migrating can be adopted to ensure service continuity; or an aggressive strategy of releasing data while migrating can be adopted to maximize resource release speed. In terms of consistency assurance, a two-phase commit protocol can be implemented to ensure the atomicity of migration transactions, or an eventual consistency model can be used to allow for brief periods of data inconsistency.

[0196] Optionally, in this embodiment, the preset timeout may include, but is not limited to, the maximum time threshold for which the system allows incomplete operations to continue waiting, including but not limited to a fixed duration configuration (e.g., 30 minutes), a dynamically calculated value (based on historical average operation time), or an adaptively adjusted value (automatically scaled according to the current system load). For example, during the unbinding process, if it is detected that a file upload operation has lasted for 15 minutes and the remaining data volume is large, the system can extend the timeout from the default 10 minutes to 30 minutes. Timeout management may be combined with an exponential backoff algorithm, triggering forced termination after multiple failed retries and generating an exception log for subsequent analysis.

[0197] It should be noted that the timeout mechanism design can be optimized by combining multiple dimensions of detection metrics. In terms of resource detection, the timeout threshold can be dynamically adjusted based on CPU / memory utilization, for example, automatically shortening the timeout period during high-load periods to avoid resource exhaustion. In terms of operation type, shorter timeouts (e.g., 5 seconds) can be set for metadata operations, while longer timeouts (e.g., 1 hour) can be set for large data transfer operations. In terms of network environment, in cross-regional migration scenarios, the timeout value can be dynamically calculated based on network latency, for example, a base time (30 seconds) multiplied by the current latency coefficient (1.5 when the latency is 200ms).

[0198] For example, when the system receives an unbinding instruction for the path / data / analytics, the unbinding process is initiated immediately. Figure 8 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 8 As shown, firstly, in step S1, the routing layer marks the path as "unbinding in progress," prohibiting new binding modification operations. Subsequently, the system scans all active I / O requests, detecting three ongoing read requests and one write request. According to a preset policy, the write request is allowed to continue until completion, while the new read request is redirected to the backup storage system. The system starts a countdown timer, setting the timeout threshold to 15 minutes.

[0199] Secondly, in the S2 phase of data migration, the system prioritizes migrating frequently accessed files from the last 7 days, employing a multi-threaded parallel transmission mechanism. After each file migration is complete, the system generates an SHA-256 checksum and compares it with the source file to ensure data integrity. For files currently being read, the system creates a temporary snapshot for migration, synchronizing incremental data after the read operation is finished. When the migration progress reaches 95%, if the transmission of a large file (20GB) is detected to be interrupted due to network jitter, the system automatically switches to the backup transmission channel and resumes transmission.

[0200] Finally, in S3, if all operations are not completed within the timeout period, the system performs a forced release: terminating the remaining migration tasks, recording the list of incomplete files, and switching the path bindings to the new storage system. Unmigrated files are switched to on-demand migration mode, triggering real-time migration upon subsequent access. All release operation records are written to the audit log, including metadata such as timestamps, operator identifiers, and the scope of impact.

[0201] This application's embodiments employ an intelligent unbinding and migration mechanism, increasing the success rate of storage system unbinding operations to 99.9%, more than 50% higher than traditional solutions. Dynamic timeout management reduces the abnormal interruption rate to below 0.5%, and the data migration integrity verification mechanism ensures zero data loss. Real-world testing data shows that when unbinding storage containing millions of files, the migration efficiency reaches 1.2TB per minute, three times faster than linear migration methods. Service interruption time is controlled at the millisecond level, resulting in a seamless user experience.

[0202] Furthermore, this embodiment supports diverse combinations of unbinding strategies, allowing for both immediate unbinding to ensure rapid resource release and gradual unbinding to maintain service continuity. Data migration supports cross-protocol conversion (such as object storage to a file system), cross-region replication, and encrypted transmission, among other scenarios. The timeout mechanism is compatible with both fixed thresholds and dynamic policies, and can be configured according to business criticality levels; for example, a lenient timeout can be set for core business paths, while rapid unbinding can be implemented for non-critical paths. These features enable the solution to adapt to various scenarios, from adjusting data center-level storage pools to reclaiming resources from edge computing nodes.

[0203] As an alternative solution, such as Figure 7 As shown, in response to a storage system release command, the binding relationship between the target storage system and the corresponding path is released, and the data corresponding to the path is migrated to another storage system, including:

[0204] S702-1, in response to the storage system release command, detects the amount of file data corresponding to the corresponding path;

[0205] S702-2, migrate the file data corresponding to the corresponding path in batches according to the preset migration rate threshold;

[0206] S702-3, After the migration is complete, delete the binding relationship between the target storage system and the corresponding path.

[0207] Optionally, in this embodiment, the migration rate threshold may include, but is not limited to, limiting parameters for controlling data migration speed, including but not limited to a transmission rate upper limit dynamically calculated based on network bandwidth utilization, a fixed rate value set according to storage system performance indicators, or a differentiated rate strategy based on service priority. For example, in a gigabit network environment, the system may set the single-batch migration rate to 200MB / s to prevent network congestion, while automatically increasing it to 500MB / s during off-peak hours to accelerate the migration process. This threshold can be dynamically adjusted in conjunction with real-time monitoring of storage system IO load, target storage write throughput, and the urgency of the migration task. For example, when the target storage system disk latency is detected to exceed 20ms, the migration rate may be automatically reduced by 30% to avoid performance overload.

[0208] It's important to note that there are various optimization methods for setting migration rate thresholds. In terms of dynamic adjustment, time-based rate control strategies can be implemented, such as limiting the rate to 200MB / s during weekdays to ensure business traffic, and increasing it to 800MB / s at night to accelerate migration. In terms of business priority, dedicated bandwidth channels can be set up for critical paths; for example, the migration rate of financial data is unrestricted, while the migration rate of log data automatically adapts to the remaining bandwidth. In terms of environment awareness, when cross-regional migration is detected, the optimal rate can be automatically calculated based on the number of network hops; for example, the base rate decreases by 10% for each additional network node hop.

[0209] Optionally, in the embodiments of this application, the aforementioned batch migration may include, but is not limited to, a strategy of splitting a large-scale dataset into multiple logical units for sequential transmission, including but not limited to partitioning by file quantity (e.g., 1000 files per batch), partitioning by total data volume (e.g., 50GB per batch), or hierarchical migration by directory structure (e.g., prioritizing the migration of first-level subdirectories). For example, when migrating the / media path containing 100,000 images, the system can divide the files into daily batches based on their creation time, with each batch containing approximately 3000 files, and use parallel transmission channels to process multiple batches simultaneously. The batch migration strategy may include a breakpoint resume mechanism, recording the migration status of each batch and automatically resuming migration from the last successful batch when a network interruption occurs.

[0210] It's important to note that the batch migration strategy can be differentiated based on data type. For structured data, batches can be divided according to database shard size, ensuring each batch contains complete table partitions. For unstructured data, batches can be clustered based on file similarity, for example, migrating similar documents (PDF, DOCX) together to optimize compression efficiency. For streaming data, a sliding window mechanism can be used to capture new files in real time and include them in the current migration batch. Furthermore, elastic batch sizes can be configured, automatically increasing batch size when system resources are idle and decreasing batch size when load increases to ensure service stability.

[0211] Optionally, in this embodiment, the file data volume detection can include, but is not limited to, quantitative analysis of storage resources under the target path, including but not limited to recursively counting the total number and size of files, fast estimation based on metadata indexes, or sampling statistics combined with machine learning prediction. For example, when unbinding from the / logs path, the system can obtain accurate statistics of 1.2TB of data and 850,000 files under that path within 10 milliseconds by querying the pre-built metadata database, without traversing the actual storage structure. For paths without indexes, a parallel file traversal algorithm can be used to split the path tree into multiple subtrees and distribute them to different worker nodes for parallel statistics, significantly improving detection efficiency.

[0212] It should be noted that file data volume detection can be combined with multi-dimensional feature analysis. In terms of storage type, sector-level statistics are used for block storage systems, and bucket list analysis is used for object storage systems. Regarding security and compliance, the detection process can integrate encrypted file identification, automatically skipping protected files and generating a pending approval list. In terms of cost control, the estimated cost of migrating data can be calculated, and a manual confirmation process is triggered when the migration cost exceeds the budget threshold. The detection results can output multi-dimensional views, including heatmaps distributed by file type, growth trend charts over time, and storage space fragmentation analysis reports.

[0213] For example, when an administrator initiates a command to remove the / archive path via the console, the system first activates the data volume detection module. The metadata service returns statistics within 50 milliseconds: the path contains 3.2TB of data, involving 125,000 files, 98% of which are compressed archives. The migration scheduling engine, based on a preset strategy, splits the total task into 64 batches, each containing 2000 files (approximately 50GB of data), and configures the migration rate to 300MB / s.

[0214] During the migration process, the real-time monitoring module detected that the target storage system's write IOPS reached 9000, approaching its performance limit. The dynamic speed adjustment algorithm immediately took effect, lowering the rate threshold to 250MB / s and splitting the batch into 128 smaller units. During the migration of the 15th batch, a brief network link fluctuation occurred, and the system automatically switched to a backup transmission channel and retried failed files. Throughout this process, the overall migration progress remained visible, with the dashboard displaying 47% completion and an estimated remaining time of 2 hours and 15 minutes.

[0215] Finally, after the migration is complete, the verification module initiates three rounds of consistency checks: the first round compares the number of files between the source and target systems; the second round samples and verifies the MD5 hash values ​​of 1000 random files; and the third round verifies directory permissions and owner information. Once confirmed, the system asynchronously clears the data from the source storage system and retains a reversible, recoverable snapshot within 24 hours. Finally, the routing configuration is updated to point the / archive path to the new storage system, and client requests automatically switch to the new node.

[0216] Through the embodiments of this application, an intelligent batch migration and dynamic speed adjustment mechanism is adopted, which improves the efficiency of large-scale data migration by 60%. In actual tests, the migration time for 10TB of data was reduced from 8 hours in the traditional solution to 3 hours. The rate adaptive algorithm keeps the network bandwidth utilization stable in the range of 92%-95%, avoiding significant impact on business traffic. The data integrity verification mechanism achieves zero-error migration, and the audit log fully records the operation details of each batch, meeting the requirements of financial-grade data management.

[0217] Furthermore, this embodiment supports flexible batching strategy configuration, allowing for division based on basic dimensions such as file quantity and size, or customized intelligent batching rules tailored to business logic. The rate control model is compatible with both fixed thresholds and dynamic strategies, enabling multi-objective optimization based on real-time network conditions, storage load, and migration priorities. The data volume detection mechanism integrates rapid statistics and in-depth analysis, providing value-added functions such as migration cost estimation and risk warning. These features enable the solution to adapt to complex scenarios ranging from conventional storage upgrades to cross-cloud migrations.

[0218] As an alternative solution, Figure 9 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 9 As shown, the above method also includes:

[0219] S902, In the case of a file operation request including a file write request, verify the target client's write permission to the target path;

[0220] S904, if the target path is bound to multiple storage systems, select at least one storage system as the target storage system according to the preset write policy, wherein the write policy includes: writing data to the storage system with the highest priority; or synchronously writing data to the storage system with write permissions.

[0221] S906 performs error handling based on preset retry counts or rollback rules when a write operation fails.

[0222] Optionally, in this embodiment, the file write request may include, but is not limited to, operation instructions initiated by a user or application to modify the data content of the storage system, including but not limited to appending file content, overwriting, modifying metadata, or creating files. For example, in a distributed storage scenario, when a user calls the write system call through the FUSE interface, the client converts it into a PUT request to the backend storage system, carrying parameters such as data block content, offset, and write mode (e.g., overwrite or append). The write request may involve data sharding, such as splitting a large file into multiple data blocks and writing them concurrently to different storage nodes, while maintaining the atomicity of the write transaction.

[0223] Optionally, in this embodiment, the write permission verification may include, but is not limited to, verifying whether the client has the authorization process to modify data at the specified path, including but not limited to permission matching based on access control lists (ACLs), role-based permission inheritance, or dynamic policy evaluation. For example, the system queries the permission policy library associated with the target path to verify whether the user token carried in the request has a write permission bit (such as the rw- identifier in POSIX permissions), and at the same time checks the storage system-level quota limits, file lock status, and version compatibility constraints to ensure the legality of the write operation.

[0224] It should be noted that the implementation of write permission verification can be flexibly configured according to security requirements. At the authentication level, multiple mechanisms can be integrated, such as OAuth2.0 token verification, Kerberos ticket verification, or two-way TLS certificate authentication. At the policy level, attribute-based access control (ABAC) can be implemented, for example, allowing write operations to test environment paths only during specific time periods (e.g., weekdays 9:00-18:00). At the auditing level, each permission verification generates detailed logs, recording the request source IP, user identity, and decision basis, supporting post-event traceability and compliance review.

[0225] Optionally, in the embodiments of this application, the above-mentioned write strategy may include, but is not limited to, data distribution rules in multi-storage system write scenarios, including but not limited to master-slave replication strategies, multi-replica synchronization strategies, or sharded write strategies. For example, when the target path is bound to three storage systems, the strategy may be configured to write to the master storage system and then asynchronously replicate to the two backup systems, or write to the two systems synchronously and then return success. The write strategy may include data consistency level configuration (such as strong consistency or eventual consistency), write order guarantee mechanism, and conflict resolution rules.

[0226] It's important to note that write policy configuration supports multi-dimensional policy combinations. Regarding data persistence, at least two storage systems can be configured to acknowledge the write operation and return success, ensuring data redundancy. For performance optimization, large files can be sharded and written concurrently to multiple storage systems; for example, a 1GB file can be split into 10 shards and written in parallel to different nodes on three storage systems. In terms of cost control, the configuration policy prioritizes writing to low-cost storage, only selecting high-performance storage when performance requirements are met; for example, cold data is written to archive storage, and hot data is written to memory cache.

[0227] Optionally, in this embodiment, the number of retries may include, but is not limited to, a maximum threshold for the number of automatic retries allowed when a write operation fails. This may include, but is not limited to, a fixed numerical configuration (e.g., 3 times), a dynamically calculated value (adjusted based on historical success rates), or a tiered strategy (5 retries for critical data, 2 retries for ordinary data). For example, when a temporary failure caused by network fluctuations is detected, the system employs an exponential backoff algorithm, with an initial retry interval of 1 second, and subsequent intervals doubling, up to a maximum of 5 retries. During the retry process, the write node may be switched or the system may downgrade to a compatible protocol, such as attempting the HTTP protocol when an HTTPS write operation fails.

[0228] Optionally, in this embodiment, the rollback rules may include, but are not limited to, data state recovery mechanisms after a write failure, including but not limited to transaction log rollback, version rollback, or compensation operations. For example, in a distributed transaction scenario, if the second storage system write fails, the system undone the completed first storage system write operation based on the write-ahead log (WAL). The rollback process may involve data verification, garbage collection, and exception notification, such as deleting some of the written temporary files and returning a rollback report containing error details to the client.

[0229] It should be noted that the error handling mechanism can be extended to suit different business scenarios. In terms of retry logic, it can distinguish between transient errors (such as network timeouts) and permanent errors (such as insufficient permissions). The former triggers automatic retries, while the latter immediately terminates the process. In terms of rollback, it supports partial rollback (only undoing the data affected by the failed operation) and full rollback (restoring to the complete state before the transaction began). In terms of degradation handling, when the primary storage system is unavailable, write requests can be temporarily redirected to the standby system, and the data can be marked as pending synchronization. Data repair will be performed after the primary system recovers.

[0230] For example, when a user initiates a write request to the path / data / transactions / 2024 / 07.log through the API gateway, the client first triggers a multi-level permission verification process. Figure 10 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 10 As shown, in step S1, the security module parses the digital certificate in the request header to confirm that the user's role is in the "Financial Write Group". In step S2, it then queries the ACL policy bound to the path to verify that the role has permissions under the / data / transactions path. In step S3, it finally checks the distributed lock service to confirm that the target file is not exclusively locked by other processes, completing the three-layer permission verification.

[0231] After verifying the permissions, the routing module discovers that the path is bound to the primary storage system A and the backup system B. According to the preset synchronous write strategy, the client fragments the data into multiple 128KB blocks and initiates parallel dual-path writes. Primary system A uses a strong consistency protocol to ensure write order, while backup system B uses an asynchronous acknowledgment mode. When primary system A returns success within 300ms, but backup system B returns a timeout error due to network jitter, the retry controller initiates an exponential backoff retry mechanism, initiating three retries at intervals of 1 second, 2 seconds, and 4 seconds, ultimately succeeding in writing on the third retry.

[0232] Finally, if all retries fail, the transaction manager locates the data blocks successfully written to the primary system A based on the write-ahead log and sends a compensatory deletion command to the backup system B to ensure data consistency. Simultaneously, the client receives a response containing an error code, prompting the user to check network conditions and resubmit the request. Throughout the process, the monitoring system updates the write success rate statistics panel in real time and highlights abnormal storage nodes in the dashboard.

[0233] Through the embodiments of this application, multi-layered permission verification and intelligent write strategies are adopted, increasing the success rate of write operations in multi-storage systems to 99.98%, a 45% improvement compared to traditional single-system solutions. The dynamic retry mechanism improves the automatic recovery efficiency of temporary failures by 70%, and the integrity guarantee of rollback transactions reduces data consistency risk to below 0.01%. Real-world test data shows that in a cross-regional three-replica write scenario, the average write latency is reduced from 380ms to 220ms, and throughput is increased by 3 times.

[0234] Furthermore, this embodiment supports a flexible combination of fine-grained permission policies and diverse write policies. It can implement strict synchronous dual-write to ensure the security of financial transaction data, or use asynchronous write to optimize throughput in media processing scenarios. The error handling mechanism is compatible with automatic repair and manual intervention modes, and supports customized retry rules and tiered rollback strategies. Its scalability allows for the integration of third-party authentication services, custom data sharding algorithms, and visualization detection components, meeting the complex needs of hybrid architectures ranging from enterprise private clouds to public clouds.

[0235] As an alternative solution, Figure 11 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 11 As shown, the above response to finding the target storage system and receiving the file processing result returned by the target storage system on the target client includes:

[0236] S1102, in response to finding a target storage system, convert the file data returned by each target storage system into a unified format, wherein different target nodes of the target path are mounted with different storage systems;

[0237] S1104, perform conflict detection on file data under the same path returned by multiple target storage systems, filter according to preset conflict rules, and determine the target file;

[0238] S1106, the target client receives the target file encapsulated in the standard file system interface format as the file processing result.

[0239] Optionally, in the embodiments of this application, the aforementioned unified format may include, but is not limited to, the process of converting the native data model of heterogeneous storage systems into a standardized data representation, including but not limited to a unified metadata structure (such as converting the creation timestamps of different storage systems into the ISO8601 standard format), a unified permission model (such as mapping the access control policy of object storage to POSIX permission bits), or a unified data block strategy (such as aligning the data block size of different storage systems to a fixed value). For example, when the target storage system A uses nanosecond-level timestamps while system B uses millisecond-level timestamps, the conversion module unifies all timestamps into the UTC format that includes the time zone; when system C uses role-based access control (RBAC) while system D uses access control lists (ACL), it converts them into a standard model that includes three types of permissions: user / group / other.

[0240] It should be noted that there are multiple technical approaches to implementing unified format conversion. In terms of metadata conversion, a type mapping table can be established to dynamically associate attribute fields from different storage systems; for example, mapping the ETag of object storage to a file checksum field. In terms of data encoding, automatic character set conversion (such as GBK to UTF-8) or media format transcoding (such as AVI to MP4) can be implemented. In terms of structural optimization, multiple small files can be merged into logical packet files to optimize transmission efficiency; for example, hundreds of images can be packaged into a ZIP file for transmission and then transparently unpacked on the client side.

[0241] Optionally, in this embodiment, the target node may include, but is not limited to, logical mount points in a virtual file system directory tree, including but not limited to leaf nodes, intermediate nodes, or specially marked virtual nodes at the directory level. For example, under the mount path / mnt / cloud, / mnt / cloud / docs may map to an object storage system, while / mnt / cloud / database points to a relational database service. The node may contain extended attributes, such as storage system identifiers, caching policy flags, or data encryption flags, for quickly determining routing strategies during path traversal.

[0242] Optionally, in this embodiment, the conflict detection can include, but is not limited to, identifying logical contradictions in data returned by multiple storage systems, including but not limited to file version inconsistencies, metadata conflicts, or content differences. For example, when two storage systems have files with the same name on the same path, the detection module identifies the differences by comparing file sizes, modification timestamps, and content hash values. Conflict detection can implement multi-stage verification: first, quickly compare basic file attributes; second, sample and verify data block hashes; and finally, perform a full binary comparison to confirm the conflict type.

[0243] It should be noted that the conflict detection mechanism can implement multi-level detection strategies according to scenario requirements. In terms of real-time performance, it can implement pre-detection scanning (periodic full verification), on-demand detection (triggered only upon access), or event-driven detection (detecting storage system change logs). In terms of detection scope, it can limit detection to only key metadata (such as file size) or perform in-depth detection of data content differences. In terms of resource consumption, it implements memory cache comparison for frequently accessed files and uses temporary disk storage comparison for infrequently accessed files, balancing performance and resource consumption.

[0244] Optionally, in this embodiment, the aforementioned conflict rules may include, but are not limited to, decision logic for resolving data conflicts, including but not limited to timestamp priority strategies (selecting the latest modified version), version number priority strategies, or user-defined merging rules. For example, it may be configured to automatically retain the version with the larger file size when a file content conflict is detected, while generating a conflict copy file (such as file.conflict-20240710) and triggering an alarm to notify the administrator. The conflict rule base supports dynamic loading, allowing for the extension of custom conflict resolution algorithms through a plugin mechanism.

[0245] Optionally, in this embodiment, the aforementioned standard file system interface format may include, but is not limited to, data encapsulation formats conforming to the operating system's native file access specifications, including but not limited to POSIX-compatible stat structures, NTFS file attribute sets, or FUSE response data structures. For example, converting the file list in object storage into a dirent structure containing fields such as inode number, file type, and permission bits allows upper-layer applications to obtain a unified view through the standard readdir call without modification. The interface conversion process may involve data type conversion, encoding format conversion, and error code mapping.

[0246] It should be noted that the standard interface encapsulation can be optimized to suit the characteristics of different operating systems. In terms of compatibility, it generates inode allocation sequences that conform to ext4 characteristics for Linux systems and NTFS-compatible file attributes for Windows systems. In terms of performance, it implements memory mapping to accelerate access to read-only files and enables copy-on-write optimization for frequently modified files. In terms of security, it adds a transparent encryption layer to sensitive data and automatically performs encryption and decryption operations at the interface layer, without the awareness of upper-layer applications.

[0247] For example, when a user accesses the path / mnt / unified / reports / q2_summary.pdf, the client routing module parses the path hierarchy and finds that the path is bound to both object storage system X and distributed file system Y. The data aggregation module initiates read requests in parallel, retrieving file data from both systems. Object storage X returns file version v3 (modified time 2024-07-10T08:30:00Z), while distributed system Y returns v2 (modified time 2024-07-09T16:45:00Z). The conflict detection engine initiates a two-stage verification: first, it compares the file sizes (5.2MB for system X, 5.1MB for system Y), then calculates the SHA-256 hash value of the first 1MB of data (a1b2c3 for X, d4e5f6 for Y), confirming the existence of a content conflict.

[0248] Secondly, based on the preset "latest version first" conflict rule, the system selects object storage X version 3 as the target file. The conversion module converts its metadata to the POSIX standard format: converting the object storage's LastModified field to st_mtime, and mapping the storage class (standard / infrequent access) to the extended attribute user.storage_class. The final encapsulated stat structure contains a size of 5,347,736 bytes, 644 permission bits, and a modification timestamp accurate to milliseconds.

[0249] Finally, before returning the result, the system automatically creates a conflicted copy / q2_summary.conflict_20240710 in the backup storage system Z and generates an audit log to record the conflict details. When users retrieve file content through the standard read call, the interface layer transparently handles the object storage's chunked download logic, returning data as a continuous byte stream to simulate the local file reading experience.

[0250] Through the embodiments of this application, intelligent conflict detection and standardized transformation are employed to achieve a consistency guarantee rate of 99.99% for multi-source data access, a 60% improvement over traditional solutions. Unified interface encapsulation improves application development efficiency by 50%, eliminating the need to write adaptation code for different storage systems. Real-world testing data shows that in a scenario with 1000 concurrent accesses, metadata transformation time is consistently within 5 milliseconds, and conflict detection accuracy is 100%, meeting enterprise-level data management requirements.

[0251] Furthermore, this embodiment supports flexible and configurable conflict resolution strategies, enabling both strict timestamp priority rules and automatic merging of differing paragraphs based on content similarity analysis. The standard interface encapsulation is compatible with multiple file system semantics, supporting seamless conversion from traditional POSIX to modern cloud-native interfaces. The extension mechanism allows for the integration of third-party data verification algorithms, custom metadata mapping rules, and dynamic encoding conversion plugins, adapting to diverse scenarios ranging from document management to multimedia processing.

[0252] As an alternative solution, Figure 12 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 12 As shown, the above method also includes:

[0253] S1202, Obtain file data migration instructions on the target client, wherein the file data migration instructions include the source storage system, the target storage system, and the migration path;

[0254] S1204, During the migration process, file data write requests associated with the migration path are simultaneously distributed to the source storage system and the target storage system;

[0255] S1206, Once the migration is complete, update the binding relationship of the migration path to point to the target storage system.

[0256] Optionally, in this embodiment, the aforementioned file data migration instructions may include, but are not limited to, operation commands that trigger the transfer of data from the source storage system to the target storage system, including but not limited to immediate migration instructions triggered by the management interface, planned migration instructions generated by scheduled tasks, or intelligent migration instructions automatically triggered based on a policy engine. For example, when the storage capacity of the source storage system is detected to have reached a preset threshold, the system automatically generates a migration instruction to migrate historical data under the specified path to a lower-cost archive storage system. The instruction typically includes metadata such as the migration path, connection parameters between the source and target storage systems, data verification rules, and migration mode (e.g., full or incremental), and supports structured description in JSON or YAML format.

[0257] It's important to note that the triggering mechanism for migration commands can be flexibly configured based on business scenarios. In manual operation mode, administrators can use a graphical tool to select the migration path and drag it to the target storage icon to generate the command. In automation mode, the system can trigger commands based on data lifecycle policies (such as automatically migrating logs created more than 180 days ago) or storage performance metrics (such as SSD storage utilization exceeding 80%). In intelligent prediction mode, it can combine machine learning models to analyze historical access patterns, predict future hot data, and execute migrations in advance. For example, if it identifies a 30% monthly decrease in access to video files under a certain path, it can automatically generate a command to migrate to low-frequency access storage.

[0258] Optionally, in this embodiment, the source storage system may include, but is not limited to, the initial storage service for the data migration operation, including but not limited to locally deployed distributed file systems, cloud object storage services, or hybrid storage clusters. For example, an enterprise-grade NAS storage system may serve as the source storage system, and its data migration instructions may include NFS mount point information, access credentials, and file filtering rules (such as migrating only files modified more than 3 years ago). The source storage system must maintain read / write availability during the migration process to ensure business continuity is not affected by the migration operation.

[0259] Optionally, in this embodiment, the target storage system may include, but is not limited to, the destination storage service for the data migration operation, including but not limited to high-availability clusters, cold storage archiving systems, or cross-regional replication storage pools. For example, the target storage system may be a distributed storage system built on erasure coding technology, which improves data reliability through sharding and redundant coding. During the migration process, the target storage system must support atomic data writing to avoid data incompleteness due to migration interruptions, and also provide auxiliary functions such as capacity pre-checking and format compatibility verification.

[0260] It's important to note that there are various optimization strategies for implementing dual-write distribution. Regarding consistency guarantees, a two-phase commit protocol can be used to ensure the atomicity of writes between the source and target systems, or an eventual consistency model can be employed to allow for transient data discrepancies. For performance optimization, the dual-write traffic ratio can be dynamically adjusted based on network bandwidth; for example, during peak business hours, the write weight of the target system can be reduced to 30% to prioritize the service quality of the source system. Regarding error handling, a difference logging mechanism can be implemented to record missing data block information when a write operation fails on the target system, which is then subsequently filled in through incremental synchronization.

[0261] Optionally, in this embodiment, the migration path may include, but is not limited to, a path expression in the virtual file system that identifies the range of data to be migrated, including but not limited to absolute paths, relative paths, or path patterns containing wildcards. For example, the migration path may be defined as / docs / project_ / reports, matching all project report directories that begin with project_. The path parsing module may perform regular expression matching, semantic analysis, or metadata index queries to accurately identify the set of files within the migration range. The target path may include the migration path; that is, when a file processing request includes a target path, and the request is used to perform data migration, the target path is the migration path.

[0262] It's important to note that the binding relationship update mechanism can incorporate multiple factors for decision-making. Regarding timeliness, an immediate switch could disrupt ongoing I / O operations; therefore, a gradual rollout strategy can be employed: initially directing 10% of requests to the new system, monitoring for one hour without anomalies, and then gradually increasing the percentage to 100%. For rollback preparation, a snapshot of the old binding relationship is retained, and a rollback time window is set. If a critical error is detected during this period, the original configuration is automatically restored. Regarding state synchronization, a global metadata refresh is triggered after the update is complete to ensure the consistency of routing tables across all clients, for example, by broadcasting configuration change events through a publish-subscribe mechanism.

[0263] For example, when the system detects that the disk utilization of the source storage cluster has reached the 85% warning threshold, the policy engine automatically generates a migration instruction. The instruction specifies the migration path as / data / archive / 2023 and the target storage system as the newly built cold storage cluster. The migration controller first scans all files that meet the criteria under the path, generates a migration task list containing 2.8TB of data, and generates a batch migration plan by sorting the files by size.

[0264] Secondly, after the migration starts, the dual-write proxy module intercepts all write requests for / data / archive / 2023. New files uploaded by users, such as report_final.pdf, are synchronously written to the SSD storage pool of the source storage cluster and the target cold storage cluster. Pipeline optimization technology is used during the write process, ensuring that the source system returns a write success response while the data packet is being transmitted to the target system, reducing the client-perceived latency from an average of 200ms to 50ms.

[0265] Finally, when the migration progress reaches 100%, the system initiates the binding relationship switching process. The routing management service pushes incremental configuration updates to all clients, pointing the default access endpoint of / data / archive / 2023 to the cold storage cluster. Ongoing read requests continue to access the source system until completion, while new requests are redirected to the target system. The migration verification module continuously checks data consistency for 24 hours. After confirming that the checksums of all files match, it automatically removes the binding relationship from the source system and releases storage space.

[0266] This application's embodiments employ an intelligent migration instruction generation and dual-write distribution mechanism, reducing the impact time of storage migration operations on business operations to the minute level, and decreasing the service interruption window by 90% compared to traditional solutions. Dynamic binding switching strategies increase the system switching success rate to 99.99%, and data consistency assurance reaches 100%. Real-world test data shows that in a 10TB-level data migration scenario, business performance fluctuations are controlled within ±5%, and storage costs are reduced by 40% after migration.

[0267] Furthermore, this embodiment supports adaptive selection of multiple migration modes, enabling both low-priority silent background migrations and high-priority real-time hot migrations. The dual-write mechanism is compatible with various data synchronization models, including mirror synchronization, asynchronous replication, and differential incremental synchronization. Binding relationship management provides advanced features such as multi-version coexistence, on-demand rollback, and automatic canary deployment switching, ensuring the stability of critical business scenarios. Extensibility allows integration with third-party detection tools, custom migration strategy engines, and visual migration dashboards, meeting the complex needs of migrating from small to medium-sized data to scheduling massive amounts of data across clouds.

[0268] As an alternative solution, Figure 13 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 13 As shown, the above method also includes:

[0269] S1302, Read storage binding rules from configuration file, which contains path regular expressions, storage system identifiers and permission configurations;

[0270] S1304, parse the configuration file and generate a routing policy table in memory;

[0271] S1306: When a configuration file update is detected, the updated storage binding rules are dynamically loaded and the routing policy table is updated simultaneously.

[0272] Optionally, in this embodiment, the configuration file may include, but is not limited to, a structured data carrier defining storage system binding relationships, including but not limited to JSON-formatted text files, YAML-formatted declarative configurations, or key-value pair collections stored in a database. For example, the configuration file may contain path matching rules represented by hierarchical indentation, where each entry defines a path pattern, storage system connection parameters, and access permission triplet. The configuration file supports hot reloading, allowing dynamic updates during system operation without restarting the service, while providing a version control mechanism to support configuration rollback.

[0273] It should be noted that there are multiple implementation methods for storing and loading configuration files. Regarding storage media, local disk files, a distributed configuration center, or an encrypted hardware security module (HSM) can be used. In terms of format compatibility, JSON Schema validation, Protocol Buffers binary encoding, or custom DSL syntax parsing are supported. Regarding version management, Git version control systems can be integrated to trace configuration history, or snapshot mechanisms can be used to retain the most recent N valid versions for emergency rollback.

[0274] Optionally, in embodiments of this application, the aforementioned path regular expression may include, but is not limited to, special strings describing virtual file system path matching patterns, including but not limited to wildcard expressions (such as ` / data / .log`), extended regular expressions (such as `^ / user / [az]+ / uploads / `), or dynamic path templates containing capture groups. For example, the regular expression ` / project / (?P` <id>`\d+) / ` can parse the item number in the path and inject it as metadata into subsequent request processing. The path matching engine may employ an automaton optimization algorithm to compile multiple regular expressions into a deterministic finite automaton (DFA), improving batch matching efficiency.

[0275] It should be noted that the complexity of path matching rules can be expanded according to scenario requirements. In terms of matching, it supports multi-level wildcards (e.g., ` / a / / b` matches any intermediate path), exclusion rules (e.g., `! / tmp / trash`), and weight settings. In terms of dynamic parameters, variables can be extracted from the path (e.g., ` / user / {id} / profile`), and their values ​​can be injected into the request context for subsequent processing. In terms of performance optimization, pre-compiled caching is implemented for high-frequency access paths, while an on-demand compilation strategy is used for low-frequency paths.

[0276] Optionally, in this embodiment, the aforementioned storage system identifier may include, but is not limited to, a symbolic name or code that uniquely identifies the backend storage service, including but not limited to a globally unique identifier in UUID format, an alias under a custom namespace, or a fingerprint value generated based on a hash algorithm. For example, the identifier `hdfs_cluster_01` may correspond to a combination of the NameNode address, port, and authentication credentials of a certain HDFS cluster. The mapping relationship between the identifier and the storage system can be stored in a separate metadata service, supporting dynamic resolution and association with load balancing strategies.

[0277] Optionally, in this embodiment, the above permission configuration may include, but is not limited to, a set of policies controlling path access permissions, including but not limited to POSIX-style file permission bits (such as `r-xr--`), role-based access control (RBAC) rules, or attribute-based access control (ABAC) policies. For example, a configuration entry may specify that the ` / finance / ` path only allows users with the `auditor` role to perform read operations between 09:00 and 18:00, while the `admin` role can perform read and write operations at any time. The permission verification module integrates a real-time policy decision point (PDP) to support dynamic policy evaluation.

[0278] It should be noted that fine-grained control of permission configuration can be implemented by combining multiple factors. At the environmental level, network source IP restrictions or device fingerprint verification can be added. At the data sensitivity level, the permission verification level for encrypted files can be automatically upgraded, requiring multi-factor authentication. At the operation type level, different permission requirements can be distinguished for reading metadata, reading content, modifying content, and deleting. Permission policies support logical operator combinations, for example:

[0279] `(role=adminORtime_window=00:00-06:00)ANDfile_size<1GB`.

[0280] Optionally, in this embodiment, the routing policy table may include, but is not limited to, an index structure maintaining the path-storage system mapping relationship in memory, including but not limited to hash tables, tries, or red-black trees. For example, a Radix tree structure can be used to store all path rules, achieving longest prefix matching with O(k) time complexity (k being the path depth). Routing table entries contain pointers to the storage system connection pool, caching policy flags, and recent access timestamps, supporting an LRU eviction mechanism to optimize memory usage.

[0281] The aforementioned dynamic loading can include, but is not limited to, the ability to update system configurations at runtime, including but not limited to hot reload triggered by file detectors, instant updates triggered by API calls, or periodic refreshes triggered by scheduled tasks. For example, the configuration detection service detects configuration file modification events through the inotify mechanism. When a change in file content is detected, an atomic update process is initiated: verifying the correctness of the new configuration syntax → generating a new routing policy table → switching the routing table pointer → reclaiming the old table memory. The dynamic loading process ensures the continuity of business requests, achieving zero-downtime updates.

[0282] For example, when the system initializes, the configuration loading module reads the configuration file from ` / etc / storage_rules.conf`. This file defines three rules in YAML format:

[0283] Rule 1: The path ` / var / logs / ` is bound to the log storage cluster, and its permissions are read-only.

[0284] Rule 2: The path ` / user / / uploads` is bound to object storage with read and write permissions.

[0285] Rule 3: The path ` / backup` is bound to the off-site disaster recovery storage, with write permissions limited to nighttime only.

[0286] Secondly, the parsing engine converts the YAML configuration into an intermediate abstract syntax tree (AST), performs semantic verification, and generates an in-memory Radix tree-structured routing table. Each tree node stores the storage system identifier, permission mask, and a function pointer to the policy enforcement module. After the routing table is built, file detection events are registered with the detection service.

[0287] Subsequently, when the administrator updates the configuration file via API, adding the path ` / data / real_time` to the in-memory database, the configuration detection service captures the change event. After the new configuration undergoes syntax validation, the routing engine builds a new Radix tree in the background, while the old routing table continues to serve online requests. Once the new tree is built, zero-latency routing table updates are achieved through atomic pointer switching technology. When the old tree's reference count drops to zero, the memory manager automatically reclaims resources.

[0288] Finally, during the dynamic loading process, if a path conflict is detected in the new configuration (such as ` / a / b` and ` / a / `), the conflict resolution module automatically adjusts the routing table structure according to preset priority rules (exact matching takes precedence). After loading is complete, the audit service records the configuration version, update time, and operator information, and triggers a detection probe to collect routing table performance metrics.

[0289] Through the embodiments of this application, an efficient routing policy table construction and dynamic loading mechanism is adopted, reducing the effective time of configuration updates from minutes in traditional solutions to milliseconds, and improving routing query performance by 80%. The intelligent conflict detection algorithm increases the rule accuracy to 99.99% in complex configuration scenarios, and the memory optimization strategy reduces routing table memory usage by 60%. Real-world test data shows that in scenarios containing millions of path rules, route matching time is consistently within 50 microseconds.

[0290] Furthermore, this embodiment supports diverse configuration management modes, enabling both centralized configuration distribution and autonomous local configuration at edge nodes. Path matching rules are compatible with a multi-mode hybrid use of regular expressions, wildcards, and semantic templates, meeting the evolutionary needs from simple static routing to intelligent dynamic routing. The permission model supports seamless integration with mainstream IAM systems, achieving enterprise-level permission governance. The dynamic loading mechanism provides multi-level rollback policies, canary release capabilities, and impact prediction analysis, ensuring the safe and reliable configuration changes in critical business scenarios.

[0291] As an alternative solution, Figure 14 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 14 As shown, the above method also includes:

[0292] S1402 converts standard file operation commands into native protocol commands of the target storage system;

[0293] S1404 standardizes the mapping of error codes returned by the native protocol;

[0294] S1406, When the target storage system does not support the current operation, generate a compatibility result by simulating the operation.

[0295] Optionally, in this embodiment, the aforementioned standard file operation instructions may include, but are not limited to, operation requests conforming to the general file system interface specification, including but not limited to the open, read, and write system calls defined by the POSIX standard, the file operation instruction set of the FUSE framework, or file sharing operations of the SMB protocol. For example, when an application calls the fopen function, the operation is converted into a structured instruction containing the file path, open mode, and permission identifier, and the adaptation layer further converts it into a protocol format recognizable by the target storage system. Standard instructions typically carry a unified operation type identifier, a data buffer pointer, and a metadata descriptor, ensuring decoupling between the upper-layer application and the underlying storage system.

[0296] Optionally, in this embodiment, the aforementioned native protocol instructions may include, but are not limited to, opcodes and data formats defined by the private communication protocols of a specific storage system, including but not limited to the HDFS RPC call format, object storage RESTful API requests, or block storage device SCSI command sets. For example, a standardized file deletion operation may be converted into an object storage system's DELETE / {bucket} / {key} HTTP request, or into an HDFS deleteRPC call, where proprietary parameters such as the target path, version number, and recursive deletion identifier need to be filled in. The construction process of the native protocol instructions may involve additional processing such as authentication token generation, request signature calculation, and protocol version negotiation.

[0297] It should be noted that the protocol conversion strategy can be flexibly adjusted according to the characteristics of the target storage system. In performance-sensitive scenarios, multiple standard operations can be merged into a batch of native requests, such as merging multiple file status queries into a single batch metadata request. In security scenarios, differentiated encryption conversion strategies can be implemented for different storage systems, such as adding double encryption at the transport layer for financial data storage systems. In compatibility scenarios, when the target system only supports older protocols, the conversion layer automatically downgrades to use compatible instructions, such as falling back from TLS 1.3 connections to TLS 1.2 for communication.

[0298] Optionally, in the embodiments of this application, the above-mentioned error code standardization mapping may include, but is not limited to, the process of converting storage system-specific error identifiers into a unified error classification system, including but not limited to mapping the HTTP 403 status code to an insufficient permissions error, converting the HDFS FileNotFoundException to the standard ENOENT error code, or mapping the storage system's custom disk full error code to ENOSC. The mapping table maintains the correspondence between error codes of each storage system and standard error types and supports dynamic expansion. For example, when an error code 0xE001 returned by a new type of storage system is detected, the system automatically classifies it into the EIO (Input / Output Error) category and records a detailed error description for operation and maintenance analysis.

[0299] It should be noted that the error code mapping mechanism can be optimized by combining multi-dimensional contextual information. In the diagnostic dimension, in addition to the basic error codes, semantic analysis is performed on the error description text returned by the additional storage system to improve mapping accuracy. In the time dimension, an automatic retry strategy is implemented for transient errors (such as timeouts caused by network jitter), and the process is immediately terminated for permanent errors (such as invalid credentials). In the log dimension, both the original error code and the standardized error code are recorded, and associated with the request detection identifier, facilitating cross-system problem troubleshooting.

[0300] Optionally, in the embodiments of this application, the above-mentioned simulation operations may include, but are not limited to, mechanisms that combine basic operations to achieve advanced functions when the native capabilities of the target storage system are insufficient. These include, but are not limited to, simulating atomic renaming operations by copying and then deleting, simulating large file appending by uploading and merging in chunks, or simulating symbolic link functionality by maintaining a virtual directory tree. For example, when the target object storage system does not support directory renaming, the system traverses all objects under the directory, copies them one by one to the new path, and then deletes the original objects, while maintaining a transaction log to ensure the atomicity of the operations. Simulation operations need to handle complex issues such as intermediate state management, exception rollback, and performance optimization.

[0301] It should be noted that the implementation of simulation operations can be optimized in layers. At the basic functionality layer, the target function is achieved through the combination of atomic operations, such as updating a file by first writing to a temporary file and then atomically replacing it. At the performance optimization layer, cache templates are established for high-frequency simulation operations, such as pre-generating a queue of parallel tasks for chunked uploads. At the consistency guarantee layer, write-ahead logs (WAL) are used to record operation steps, and in the event of an interruption, the state is restored or rolled back based on the logs to ensure data integrity.

[0302] For example, when a user application calls `rename(" / data / old"," / data / new")`, the protocol conversion module first checks the capabilities of the target storage system. If the target is a file system that supports atomic renaming (such as ext4), it is directly converted to the native `rename` command. Figure 15 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 15 As shown, if the target is an object storage system that does not support directory renaming, the conversion module starts a simulation process:

[0303] S1, Traverse all objects under the source path and generate a list of migration tasks;

[0304] S2, create virtual directory metadata for the target path;

[0305] S3 performs object copying operations in parallel, maintaining read access to the original path;

[0306] S4, After all copies are completed, delete the objects in the original path in batches;

[0307] S5, update the routing table and bind the new path to the target storage system.

[0308] During this process, the transaction manager maintains an operation log, recording the migration status of each object. If the copying of an object fails in step 3, the successfully copied objects are rolled back based on the log, ensuring the atomicity of the operation. Finally, a standardized success status code is returned to the application, masking the underlying storage differences.

[0309] Finally, when object storage returns a "BucketAlreadyExists" error, the error mapping module, combined with the request context (the operation being the creation of a new bucket), converts it into the EEXIST standard error code and adds a friendly message "Target storage space already exists." For uncommon custom error codes, the system initiates a dynamic learning mechanism, temporarily storing the error code in a classification queue, where the mapping relationship will be determined after review by operations and maintenance personnel.

[0310] Through the embodiments of this application, intelligent protocol conversion and error code standardization are employed to improve cross-storage system operational compatibility to 98.5% and increase application development efficiency by 40%. The simulated operation mechanism fills 20% of the original functional gaps, ensuring the functional integrity of critical business scenarios. Real-world test data shows that in a hybrid storage environment, the operation success rate increases from 85% in traditional solutions to 99.2%, and error diagnosis time is reduced by 70%.

[0311] Furthermore, this embodiment supports multi-level protocol conversion strategies, enabling both simple instruction mapping and cross-protocol transaction management through complex state machines. The error handling mechanism is compatible with real-time classification and offline analysis, supporting automatic error pattern clustering and early warning. The simulation operation framework provides pluggable adapter interfaces, allowing access to proprietary extensions of third-party storage systems. The dynamic policy loading mechanism enables the system to adapt to the continuous evolution of storage technologies, ensuring long-term compatibility.

[0312] As an alternative solution, Figure 16 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 16 As shown, the above method also includes:

[0313] S1602, When multiple storage systems detect directories with the same path, a virtual merged directory is generated;

[0314] S1604, perform deduplication according to the preset conflict resolution strategy and update the virtual merge directory;

[0315] S1606 synchronizes the update operations of the virtual merged directory to the corresponding multiple storage systems.

[0316] Optionally, in this embodiment, the aforementioned virtual merged directory may include, but is not limited to, logical aggregated views created under the same path in multiple storage systems, including but not limited to cross-system directories merged through metadata indexing, temporary merged structures based on memory caching, or global directory mapping tables in persistent storage. For example, when the path / documents in storage system A contains file A1, and the same path in storage system B contains file B1, the virtual merged directory will present an aggregated view containing A1 and B1, and users can browse all files through a single interface. This directory may maintain metadata snapshots of files across systems, supporting file source identification, access statistics, and synchronization status marking, such as indicating that a file exists only in a specific storage system or has multiple copies.

[0317] It's important to note that virtual merged directories can be implemented in various ways. In terms of metadata management, a centralized metadata database can be used to maintain a global view, or a distributed metadata synchronization protocol can be implemented. Regarding performance optimization, frequently accessed directories can be accelerated with memory caching, while infrequently accessed directories can be loaded on demand. In terms of security control, a unified permission inheritance model across systems can be supported; for example, read-only permissions for a parent directory can be automatically applied to the merged view of all subdirectories. Virtual directories can be integrated with intelligent tagging systems to automatically categorize and display files based on file type, such as displaying thumbnail previews for image directories.

[0318] Optionally, in this embodiment, the conflict resolution strategy may include, but is not limited to, decision rules for handling data inconsistencies across multiple storage systems, including but not limited to a timestamp-first priority strategy, a version number-first priority strategy, or a manual intervention marking strategy. For example, when it is detected that a file with a modification time of 2024-07-10 exists in storage system X under the path / logs, and a file with the same name exists in storage system Y but with a modification time of 2024-07-11, the system automatically selects the file with the newer timestamp as the major version. The strategy library supports hierarchical configuration, allowing for the setting of differentiated rules for different directories, such as implementing strict version control for the financial data directory and automatic overwriting for the temporary file directory.

[0319] It's important to note that the triggering mechanism for conflict resolution strategies can be flexibly configured according to the scenario. In terms of real-time performance, it can implement on-demand detection triggered upon access, batch detection triggered on a schedule, or event-driven instant detection. In terms of decision-making, it supports multi-rule combination evaluation; for example, when the timestamp difference is less than a threshold, the replica with the lower storage cost is prioritized. In terms of human collaboration, it provides a conflict marking interface, allowing users to set permanent resolution strategies for specific files. The strategy execution process can record detailed operation logs, supporting audit playback and strategy effect analysis.

[0320] Optionally, in the embodiments of this application, the above-mentioned deduplication process may include, but is not limited to, the operation process of eliminating redundant data objects, including but not limited to duplicate file deletion based on content hashing, partial deduplication based on metadata comparison, or replica optimization based on access hotspots. For example, when files with the same MD5 hash value are detected under the / reports path of three storage systems, the system retains the replica in the storage system with the highest access frequency, and only retains the metadata pointer in the other systems. The deduplication process may involve data block-level comparison, storage cost calculation, and access link optimization, such as migrating frequently accessed duplicate files to low-latency storage nodes.

[0321] It's important to note that the granularity of deduplication can be optimized based on data type. At the file level, full deduplication is achieved through hash comparison of the entire file. At the block level, large files are divided into fixed-size data blocks for incremental deduplication. At the semantic level, primary key deduplication is performed on structured data (such as database tables), and similarity clustering is used for deduplication on unstructured data. The deduplication process can be optimized in conjunction with the characteristics of the storage system; for example, a single data block can be retained in an erasure coding storage system, while the number of redundant replicas can be reduced in multi-replica requests.

[0322] For example, when a user accesses the virtual merged directory / data / shared, the system detects that the path exists simultaneously in the object storage system C and the distributed file system D. Figure 17 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 17 As shown, the merge engine starts a multi-stage processing flow:

[0323] S1, Metadata Acquisition: Parallel queries are performed on various storage systems to obtain a list of files and their attributes under the / data / shared path. Object storage C returns 50 files, and file system D returns 45 files, with 40 files having the same filename.

[0324] S2, Conflict Detection: A deep check was performed on 40 files with the same name. 32 files had identical modification times and sizes, and were considered conflict-free; 8 files showed version differences. For example, file_project.pdf was last modified at 09:30 on system C, but at 09:35 on system D, with a size difference of 200KB.

[0325] S3, Strategy Application: Based on the preset "timestamp priority" strategy, file_project.pdf from system D is selected as the primary copy. The deduplication module moves the old version file from system C to the recycle bin and records the operation log in the metadata database.

[0326] S4, View Generation: Creates a virtual directory containing 55 unique files (50 + 45 - 40 + 8 duplicates removed), with 5 conflicting files marked with warning icons. When users view the file through the file manager, conflicting files display floating tips, showing the differences between versions and providing an option to manually select them.

[0327] S5, Synchronous Update: The conflict resolution results are reverse-synchronized to the source storage system. A hidden file named .merge_log is created in system C to record operation details, and the version tag of the file is updated in system D. An asynchronous data balancing task is started to migrate frequently accessed duplicate files to high-performance storage nodes.

[0328] Through the embodiments of this application, an intelligent merging and conflict resolution mechanism is employed, improving the efficiency of cross-storage directory operations by 70% and reducing data redundancy by 60%. The response time of the virtual merged view is optimized from seconds in traditional solutions to milliseconds, supporting real-time aggregation and display of millions of files. Real-world test data shows that in a hybrid environment containing 10 storage systems, directory traversal performance is improved by 3 times, and storage costs are reduced by up to 45%.

[0329] Furthermore, this embodiment supports flexible and configurable merging strategies, enabling both simple filename aggregation and deep integration of file content and metadata. The conflict resolution mechanism is compatible with both automated decision-making and manual intervention modes, providing auxiliary tools such as difference comparison, version rollback, and strategy simulation. Deduplication processing supports optimization across storage tiers, such as retaining frequently accessed data in high-speed storage and archiving less accessed data to low-cost storage. Extensibility allows for the integration of third-party deduplication algorithms, custom metadata tagging systems, and visualization analysis modules, adapting to diverse scenarios ranging from enterprise document management to medical image storage.

[0330] As an alternative solution, such as Figure 16 As shown, deduplication is performed according to the preset conflict resolution strategy, and the virtual merge directory is updated, including:

[0331] S1604-1, If ​​files with the same name have the same metadata attributes, retain the file in the storage system bound to the same path;

[0332] S1604-2 If the metadata attributes of files with the same name are inconsistent, the corresponding file shall be selected as the target file according to the predefined conflict resolution strategy, wherein the conflict resolution strategy is based on at least one of storage system priority, user configuration rules or file version status.

[0333] Optionally, in this embodiment, the consistency of the aforementioned metadata attributes may include, but is not limited to, files having completely identical descriptive characteristics across multiple storage systems, including but not limited to file size, modification timestamp, content hash value, permission bit settings, and extended attributes. For example, when a report.pdf file exists under the path / docs in both storage system A and system B, the system will compare their MD5 hash values, file size in bytes, and last modification time. If all three attributes match perfectly, the metadata is determined to be consistent. Furthermore, for special scenarios, consistency checks on some non-critical attributes (such as creator information) can be configured to be ignored to reduce the false positive rate. Metadata attribute consistency judgment can employ a layered verification mechanism: first, quickly compare the file size and modification time; if inconsistent, it is directly determined to be a conflict; if consistent, further content hash calculation is performed.

[0334] It should be noted that the consistency judgment of metadata attributes can be flexibly configured based on multi-dimensional standards. In terms of judgment dimensions, it can be limited to verifying only basic attributes (such as file size and modification time), or extended to the complete attribute set (including ACL permissions, extended attributes, and custom tags). In terms of technical implementation, a fast verification algorithm (such as CRC32) can be used for initial screening, followed by a precise verification algorithm (such as SHA-256) for secondary verification. In terms of application scenarios, strict consistency checks should be implemented for sensitive data (such as contract documents), while the verification standards for temporary cache files can be relaxed, comparing only file sizes.

[0335] Optionally, in this embodiment, the storage system priority may include, but is not limited to, storage service ranking metrics used to resolve conflicts, including but not limited to availability scores based on Service Level Agreements (SLAs), real-time performance monitoring data (such as response latency and throughput), or static weight values ​​set by the administrator. For example, in a hybrid cloud environment, the priority of a local high-performance storage cluster may be set to 90, while the priority of a public cloud storage service may be 70. Priority calculation can be dynamically adjusted; when the disk utilization of a storage system is detected to exceed 85%, its priority is automatically reduced by 20%. The priority policy can be refined to the directory level, for example, setting up a separate high-priority storage pool for financial data directories to ensure the access quality of critical business data.

[0336] It's important to note that storage system priority settings can be optimized in conjunction with dynamic environmental factors. In terms of time, time-based priority strategies can be configured, such as prioritizing local storage during peak business hours and increasing backup storage priority at night to perform data synchronization. In terms of data characteristics, priorities can be differentiated based on file type, such as routing video files to high-bandwidth storage and document files to low-latency storage. In terms of anomaly handling, when a high-priority storage system failure is detected, the weight is automatically transferred to the next lower priority system, and an alarm notification is triggered.

[0337] Optionally, in this embodiment, the file version status may include, but is not limited to, an evolution identifier of the file during its lifecycle, including but not limited to a timestamp-based version sequence, a user-manually marked version tag, or an automatically generated semantic version number. For example, the version number of the file design_v1.2.3.docx follows the major version.minor version.revision number specification, and the system can be configured with a conflict resolution strategy to prioritize files with higher major version numbers. Version status management may integrate change detection functionality, recording the user who performed each modification, a change summary, and dependencies. For files without explicitly marked versions, the system can automatically deduce the version evolution relationship based on the modification timestamp and content differences.

[0338] It's important to note that the version status policy supports multi-rule composite decisions. At the version identification level, it can be configured to prioritize retaining semantic versions (e.g., v2.1.0) over timestamp versions. At the collaboration level, a user voting mechanism is integrated; when multiple users make conflicting modifications to the same file, a majority vote is triggered. At the security level, version rollback protection is implemented for encrypted files, allowing only users with specific permissions to perform version overwrite operations.

[0339] For example, when a user accesses the virtual merge directory / project_docs, the system detects that the path exists in both object storage system X and distributed file system Y. Figure 18 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 18 As shown, the merge engine performs the following operations:

[0340] S1, Metadata Scanning and Verification:

[0341] The system retrieves file lists from the ` / project_docs` path in two storage systems in parallel. It finds 12 files with the same name. Eight of these files have identical modification times, sizes, and SHA-256 hash values, and are therefore identified as files with identical metadata. Of the remaining four files, three have the same size but their modification times differ by more than a threshold (30 seconds), and one file has a significantly different size.

[0342] S2, Application of conflict resolution strategies:

[0343] Consistent file handling: Files in the higher-priority object storage system X are retained, but the location of the copy in system Y is recorded in the metadata database for subsequent disaster recovery.

[0344] Inconsistent file handling: For three files with conflicting timestamps, the version with the later modification time is selected based on the preset latest version priority strategy. For one file (design_spec.docx) with a significant size difference, a manual review process is triggered, and a difference comparison report is generated in the management interface for user decision-making.

[0345] S3, dynamic updates of virtual directories:

[0346] The updated virtual merge directory displays 15 files (8 duplicates of 12 files with the same name are retained, and 4 conflicting files are retained), with conflicting files marked with a red warning icon. When the user clicks on design_spec.docx, a pop-up window displays the modifier, timestamp, and abbreviated differences between the two versions, supporting one-click selection of the target version.

[0347] S4, Reverse Synchronization and Log Recording:

[0348] The user's final selected file version is reverse-synchronized to the source storage system to ensure data consistency between the two ends. Simultaneously, the conflict resolution process is recorded in the audit log, including the basis for automated decisions, records of manual intervention, and timestamps of the synchronization operations.

[0349] Through the embodiments of this application, the adoption of intelligent metadata verification and multi-dimensional conflict resolution strategies improves the efficiency of cross-storage system directory merging by 65% ​​and reduces the need for manual intervention by 80%. The dynamic priority adjustment mechanism optimizes storage resource utilization by 40% and achieves a version conflict handling accuracy of 99.2%. Real-world test data shows that in a hybrid storage environment with 10TB of data, the time taken to generate a virtual directory is reduced from 15 seconds in the traditional solution to 2.3 seconds, and the data consistency guarantee rate is improved to 100%.

[0350] Furthermore, this embodiment supports flexible and configurable conflict resolution logic, enabling both fully automated decision-making and integration into manual review processes. The metadata verification mechanism is compatible with multi-level precision control, adapting to different scenarios ranging from regular documents to massive log files. Priority policies and version management modules support plug-in extensions, allowing integration with third-party detection tools and collaboration platforms. The real-time synchronization capability of the virtual directory can be expanded to support cross-regional storage nodes, and combined with intelligent caching strategies, optimizes the access experience for global users.

[0351] As an alternative solution, such as Figure 16 As shown, the update operations of the virtual merged directory are synchronized to the corresponding multiple storage systems, including:

[0352] S1606-1, When a file is created in a virtual merge directory, write data to the storage system associated with the virtual merge directory in the storage binding rule;

[0353] S1606-2, When deleting a file in a virtual merge directory, perform the deletion operation in the storage system to which the file is bound.

[0354] It should be noted that there are multiple implementations for the file creation request distribution mechanism. At the triggering mechanism level, it can be configured to synchronously write to all associated systems in real time, or asynchronously write to the main system and then delay synchronization to other systems. At the distribution strategy level, it supports full replication (writing to all associated systems), conditional replication (writing only to systems that meet the storage policy), or sharded writing (splitting the file and distributing it). At the transaction management level, a two-phase commit protocol can be used to ensure the atomicity of writes across multiple systems, or an eventual consistency model can be used to allow for transient differences.

[0355] Optionally, in this embodiment, the storage binding rules may include, but are not limited to, configuration strategies describing the mapping relationship between virtual paths and physical storage systems, including but not limited to association tables of path regular expressions and storage system identifiers, dynamic weight allocation strategies, or attribute-based routing rules. For example, a rule may be defined to associate the ` / data / hot / ` path with an SSD high-performance storage cluster, while ` / data / cold / ` is associated with a tape archive system. Binding rules support multi-level nesting, allowing multiple heterogeneous storage systems to be bound simultaneously under the ` / data` root directory, and controlling the access order through priority tags.

[0356] Optionally, in this embodiment, the associated storage system may include, but is not limited to, backend storage services that establish mapping relationships with virtual paths through storage binding rules, including but not limited to distributed file systems, object storage services, or block storage devices. For example, when a user performs an operation under the virtual merged directory ` / project`, the system identifies that the path is simultaneously associated with object storage cluster X and HDFS cluster Y according to the rules. The association relationship can be dynamically adjusted; for example, when a storage system is detected to be overloaded, some paths can be automatically transferred to a backup system.

[0357] Optionally, in the embodiments of this application, the above-mentioned deletion operation may include, but is not limited to, a logical deletion process of removing file entities or tags in the physical storage system, including but not limited to sending standard file deletion instructions, calling storage system-specific APIs, or updating metadata to mark them as deleted. For example, performing deletion in an object storage system may be converted into an HTTP request for `DELETE / bucket / file`, while in a distributed file system it may trigger a data block reclamation task. The deletion operation may involve cross-system reference counting management, for example, when a file exists in multiple associated systems, physical space is released only after all copies have been marked as deleted.

[0358] It should be noted that the scope of the deletion operation can be flexibly defined according to needs. At the scope level, cascading deletion (deleting the directory and all its sub-contents), selective deletion (retaining specified versions of files), or marked deletion (only hiding files without releasing space) can be implemented. At the security level, soft deletion retains a recovery window (e.g., 30 days), hard deletion immediately releases resources, or multi-stage deletion (first moving to the recycle bin and then periodically cleaning up). At the cross-system level, it can be configured to a strict mode that deletes all copies, or a safe mode that retains at least one copy.

[0359] Furthermore, the exception handling during synchronization supports multi-level recovery strategies. Regarding retry mechanisms, fixed-number retries, exponential backoff retries, or adaptive retries based on error type can be configured. For compensation operations, automatic rollback is implemented for partially successful creation operations, or the breakpoint status is recorded for subsequent resumption. In terms of alarm notifications, email notifications, system log recordings, or maintenance work orders are triggered based on the error severity level.

[0360] For example, when a user creates a new file `Q3_summary.pdf` under the ` / data / reports` path using a virtual merge directory, Figure 19 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 19 As shown, the system initiates a multi-stage processing flow:

[0361] S1, Path Resolution: The routing module resolves the storage binding rules of ` / data / reports` and finds that the path is associated with the object storage system OBS and the distributed file system DFS.

[0362] S2, Write Distribute:

[0363] Send a chunked upload request to OBS to split the file into five 128MB data blocks for parallel transmission.

[0364] An atomic write request is sent to DFS, and the data is cached in memory before being persisted to disk.

[0365] S3, consistency guarantee: The transaction coordinator monitors the write progress of the two systems. If the DFS write fails but the OBS write succeeds, the coordinator selects the appropriate system based on the configured strategy.

[0366] Strategy A: Roll back the data blocks already written by OBS and return a failure message to the user.

[0367] Strategy B: Mark OBS data as a temporary copy and start a background repair task.

[0368] S4, Metadata Update: Create file entries in the virtual directory, marking the dual storage locations and write timestamps.

[0369] When a user deletes the file ` / data / archives / old_project.zip` in the virtual directory:

[0370] S1, Reference Detection: A query of the metadata database revealed that the file exists as a copy in both OBS and DFS.

[0371] S2, Delete execution:

[0372] Send a delete command to OBS to immediately release storage space.

[0373] Send a soft delete command to DFS to move the file to the `.trash` directory and keep it there for 7 days.

[0374] S3, State Synchronization: Updates the virtual directory to mark the file as deleted, but retains a recovery entry for DFS.

[0375] S4, Space Reclamation: The background task automatically performs physical deletion after the DFS retention period expires and updates the storage quota statistics.

[0376] Through the embodiments of this application, an intelligent synchronization mechanism is adopted, which improves the efficiency of directory operations across storage systems by 60% and reduces data consistency risks by 75%. Dynamic distribution strategies optimize storage resource utilization by 45%, and an exception handling mechanism increases the operation success rate to 99.5%. Real-world test data shows that in a hybrid cloud environment, the creation time for millions of files is reduced from 320 seconds in the traditional solution to 120 seconds, and the error recovery time for deletion operations is shortened by 85%.

[0377] Furthermore, this embodiment supports flexible multi-storage collaboration strategies, enabling both strong consistency synchronization to ensure critical data security and eventual consistency models to improve performance. The distribution strategy for creation operations supports dynamic selection of the optimal storage combination based on file characteristics (such as size and type), and the execution granularity of deletion operations can be refined to the version or block level. The anomaly handling framework is compatible with both automated repair and manual intervention modes, providing multi-dimensional detection indicators and a visual detection interface. Its scalability allows for the integration of third-party storage services, custom transaction management modules, and intelligent storage optimization algorithms, adapting to complex needs ranging from enterprise private clouds to hybrid multi-cloud environments.

[0378] The present application will be further explained below with reference to specific embodiments:

[0379] This application provides a unified storage view, integrating various heterogeneous storage systems (such as HDFS, Ceph, COS, etc.) into a single namespace. Users can access all storage resources through a unified client interface without needing to worry about the differences in the underlying storage, thereby reducing operation and maintenance costs.

[0380] This application aims to provide a highly flexible and scalable storage solution that supports on-demand loading of different storage solutions. By mapping different storage solutions to different sub-paths within the same directory tree, or even binding multiple heterogeneous storage solutions to the same sub-path, unified management and access to various storage resources are achieved. This unified namespace simplifies user operations, improves resource utilization, and reduces operational costs. For example, some data can be stored on HDFS to leverage its high throughput for big data analytics, while another portion can be stored on the lower-cost COS for archiving, and managed under the same directory using different access policies.

[0381] Currently, when using multiple distributed storage systems on the same compute node or cluster, users need to install and configure corresponding client software for each storage system and access them using different application programming interfaces (APIs). This results in users having to deal with different data access methods and data views.

[0382] Different distributed storage systems present users with different data views, increasing the complexity of learning and using them, reducing work efficiency, and requiring users to maintain independent client software and configuration files for each distributed storage system, leading to resource waste and increased operation and maintenance costs. In addition, data migration between different distributed storage systems requires users to develop and maintain their own data migration tools, increasing the complexity and risk of data management.

[0383] This application aims to overcome the above-mentioned deficiencies and provide a unified access interface and configuration management mechanism, enabling users to access multiple heterogeneous distributed storage systems through a single client and configuration file, and presenting users with a unified POSIX file system view.

[0384] This application is particularly suitable for heterogeneous distributed storage environments. By adopting this application, users only need to make necessary configuration changes to seamlessly integrate and use multiple different distributed storage systems, such as HDFS, Ceph, and object storage (e.g., COS, S3), under the same mount directory.

[0385] Figure 20 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 20 As shown, this application provides a unified storage access system, the overall architecture of which includes the following three layers:

[0386] Routing layer: Responsible for request routing and mapping, implementing functions such as dynamic mounting, heterogeneous mapping, path management and nested mapping.

[0387] File system aggregation layer: responsible for integrating data from different storage systems, enabling functions such as directory merging and data migration.

[0388] Virtual interface layer: Provides a unified access interface, shielding the implementation details of different underlying storage systems.

[0389] Detailed explanation of functional modules:

[0390] Routing layer:

[0391] Dynamic mounting: When using the client implemented in this application, users can dynamically add and unload distributed storage systems without affecting read and write operations.

[0392] Heterogeneous mapping: Users can mount different distributed storage solutions (such as CEPH, COS, HDFS, etc.) to the same path to enable simultaneous access to data on these storage systems.

[0393] Path management: Provides flexible directory access control to meet the permission requirements of different users on the same machine.

[0394] Nested mapping: Allows different distributed storage solutions to be mounted on different directory nodes along a path, such as mounting CEPH storage on / dir1 / dir2 and object storage on / dir1 / dir2 / dir3.

[0395] File system aggregation layer:

[0396] Directory merging: When a user mounts different distributed storage solutions under the same path, this module is responsible for merging the directory structures of different storage systems into a unified view, so that the user can see all the content at the same time.

[0397] Data migration: Enables users to move data between different distributed storage solutions as needed without affecting user read and write operations.

[0398] Virtual interface layer:

[0399] It provides a unified API interface, shielding upper-layer applications from the implementation details of different underlying distributed storage solutions, thus simplifying application development.

[0400] Figure 21 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 21 As shown, including but not limited to the following processes:

[0401] S1, the user initiates a file access request (including read / write operations).

[0402] S2: After receiving the request, the client passes it to the routing layer.

[0403] S3, the routing layer looks up the corresponding mount information based on the target path.

[0404] S4, Check if the target path exists:

[0405] S4-1, if it does not exist, returns an error message.

[0406] S4-2, if it exists, continue execution.

[0407] S5, Determine the operation type:

[0408] S5-1: If it is a read operation, the request will be forwarded to the corresponding storage system based on the mount information.

[0409] S5-2, If it is a write operation, then a permission check is performed.

[0410] S6, Permission Verification:

[0411] S6-1: If permissions are insufficient, an error message will be returned.

[0412] S6-2, if sufficient permissions are available, will forward the request to the corresponding storage system based on the mount information.

[0413] S7, the storage system performs the corresponding operation and returns the result to the client.

[0414] S8, the client returns the result to the user.

[0415] In an exemplary embodiment, the client is mounted to the / mnt / data directory using a configuration file. Operations on files and directories under the / mnt / data directory are passed to the client for processing. It supports the simultaneous use of three distributed storage schemes: Ceph, HDFS, and COS. Ceph and COS are bound to the / dir1 directory, and HDFS is bound to / .

[0416] Example 1: Appending to a file. The user uses the command "echo helloword >> / mnt / data / file" to append the string "helloword" to / mnt / data / file. Figure 22 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 22 As shown, including but not limited to the following processes:

[0417] S1, the operating system translates the command into a write operation to / file and sends the request to the client.

[0418] S2, when the client receives a write request to / file, it first checks if the file exists. Since HDFS is bound to the root directory / , the request to check if the file exists is sent to HDFS.

[0419] S3, if the file does not exist, returns a file not found error.

[0420] S4. If the file exists, check if / file supports write operations. The configuration file specifies paths=[" / r"], indicating that all directories and files in HDFS storage only support read operations. Therefore, the client will reject write operations and return a permission-disallowed error.

[0421] Example 2: List the contents of the directory. Figure 23 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 23 As shown, the user uses the command "ls / mnt / data / dir1" to query information about all files in the / mnt / data / dir1 directory, including but not limited to the following process:

[0422] S1, the operating system translates the command into ls / dir1 and sends the request to the client.

[0423] S2, when the client receives the ls / dir1 request, it first checks if the directory exists. Since Ceph and COS are both bound to / mnt / data, the request to check if / dir1 exists will be sent to Ceph and COS respectively.

[0424] S3, if the / dir1 directory does not exist in either Ceph or COS, then return an error indicating that the directory does not exist.

[0425] S4. If both Ceph and COS have the / dir1 directory, since the configuration file has paths = [" / rw"], it means that all directories and files in Ceph and COS support read and write operations. Therefore, the client will merge the directory contents of the two storage systems and return them.

[0426] S5. If only one of Ceph and COS has a / dir1 directory, the client will directly return the directory contents of that storage system.

[0427] Example 3: Temporarily remove Ceph storage. When users need to operate on directories and files under / mnt / data, they may need to temporarily remove Ceph storage. Figure 24 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 24 As shown, including but not limited to the following processes:

[0428] S1, send the Ceph removal command to the client;

[0429] In S2, the client internally unbinds the Ceph storage from / dir1. Requests currently operating on the Ceph storage are unaffected, and new requests will not be sent to the Ceph storage. The binding is automatically removed once all requests on the Ceph storage have been processed.

[0430] Example 4: Permanently remove Ceph storage. Users need to permanently remove Ceph storage when working with directories and files under / mnt / data. Figure 25 This is a schematic diagram of another optional file processing method according to an embodiment of this application, such as... Figure 25 As shown, including but not limited to the following processes:

[0431] S1, modify the configuration file and remove Ceph-related configurations.

[0432] S2 sends a command to the client to reload the configuration file.

[0433] When the client loads the configuration file in S3, it will find that the Ceph storage has been cleaned up and will lazily unbind the Ceph storage from / dir1. After all requests have been processed, the binding will be automatically removed.

[0434] This application significantly reduces the number of client instances by introducing a unified client and abstraction layer, thereby reducing resource consumption and simplifying the complexity of using heterogeneous distributed storage.

[0435] Prior to applying this application, users needed to install and maintain separate clients for each different distributed storage system (e.g., HDFS, Ceph, COS, etc.), and use specific mounting methods and write separate adaptation code. This led to the following problems:

[0436] Client redundancy: Each type of storage requires a client, which increases the complexity of system maintenance and resource overhead.

[0437] Inconsistent mounting methods: Different storage systems may have different mounting methods, which increases the learning cost and difficulty of use for users.

[0438] Complex adaptation code: Applications need to write different adaptation code for different storage systems, which increases development and maintenance costs.

[0439] By applying this application, a unified client and abstraction layer are provided to mount and access various heterogeneous distributed storage systems in a standard POSIX manner. This abstraction layer shields users from the differences in the underlying storage systems, providing a consistent access interface and thus offering the following advantages:

[0440] Reduced client instances: Users only need to install and maintain one client, which greatly reduces the complexity of system maintenance and resource consumption.

[0441] Unified mounting and access methods: All storage systems can be mounted and accessed through the standard POSIX interface, reducing the learning cost and difficulty of use for users.

[0442] Simplified application development: Applications do not need to write different adaptation code for different storage systems, which simplifies development and maintenance costs and improves development efficiency.

[0443] Reduced resource usage: By reducing the number of client instances, the consumption of resources such as CPU, memory, and network is reduced.

[0444] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0445] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0446] According to another aspect of the embodiments of this application, a file processing apparatus for implementing the above-described file processing method is also provided. For example... Figure 26 As shown, the aforementioned device includes:

[0447] The acquisition module 2602 is used to acquire file operation requests on the target client. The file operation request includes a target path, which is used to map to the corresponding storage system according to storage binding rules. The storage binding rules include the binding relationship between the path and the storage system. The target client has at least two heterogeneous storage systems mounted, and the access methods of the at least two heterogeneous storage systems are different.

[0448] The lookup module 2604 is used to, in response to a file operation request, locate the target storage system on the target client according to the target path, wherein at least two storage systems include the target storage system and the target storage system corresponds to the target path;

[0449] The receiving module 2606 is used to receive the file processing result returned by the target storage system on the target client in response to finding the target storage system. The file processing result represents the result of processing the target file based on the file operation request, and the target file represents the file stored in the target storage system indicated by the target path.

[0450] As an optional solution, the above-mentioned device is used to respond to a file operation request and locate the target storage system on the target client according to the target path in the following manner: in response to the file operation request, the hierarchical structure of the target path is parsed and a path matching queue is generated; according to the path binding priority in the storage binding rules, the storage system with the highest matching degree is selected from the path matching queue as the target storage system; in the case that multiple storage systems are bound to the same path, at least one storage system is selected as the target storage system according to a preset conflict strategy.

[0451] As an optional approach, the aforementioned apparatus is used to select the storage system with the highest matching degree from the path matching queue as the target storage system according to the path binding priority in the storage binding rules in the following manner: selecting a binding path that is the same as the target path, wherein the binding path represents the path bound to the storage system in the storage binding rules; if there is no binding path that is the same as the target path, selecting the binding path corresponding to the longest prefix of the target path; if there are multiple binding paths corresponding to the longest prefix, selecting a binding path according to a preset weight; and using the storage system corresponding to the binding path as the target storage system.

[0452] As an optional solution, the above-mentioned device is further configured to: in response to a storage system release command, release the binding relationship between the target storage system and the corresponding path, and migrate the data corresponding to the corresponding path to other storage systems, wherein the target path includes the corresponding path; or in response to a storage system release command, suspend the binding relationship between the target storage system and the corresponding path, and retain the data corresponding to the corresponding path; during the removal process, if there are incomplete file operation requests in the target storage system, delay the removal operation until the request is completed or a preset timeout period is reached, wherein the target path includes the corresponding path.

[0453] As an optional solution, the above-mentioned device is used to respond to a storage system release command in the following manner, to release the binding relationship between the target storage system and the corresponding path, and to migrate the data corresponding to the corresponding path to other storage systems: in response to the storage system release command, detecting the amount of file data corresponding to the corresponding path; migrating the file data corresponding to the corresponding path in batches according to a preset migration rate threshold; and deleting the binding relationship between the target storage system and the corresponding path after the migration is completed.

[0454] As an optional solution, the above-mentioned device is also used to: verify the write permission of the target client to the target path when the file operation request includes a file write request; if the target path is bound to multiple storage systems, select at least one storage system as the target storage system according to a preset write policy, wherein the write policy includes: writing data to the storage system with the highest priority; synchronously writing data to the storage system with write permission; and when the write fails, performing error handling according to a preset number of retries or rollback rules.

[0455] As an optional solution, the above-mentioned device is used to receive the file processing result returned by the target storage system on the target client in response to finding the target storage system as follows: in response to finding the target storage system, converting the file data returned by each target storage system into a unified format, wherein different target nodes of the target path are mounted with different storage systems; performing conflict detection on the file data under the same path returned by multiple target storage systems, filtering according to preset conflict rules, and determining the target file; and receiving the target file encapsulated in a standard file system interface format as the file processing result on the target client.

[0456] As an optional solution, the above-mentioned device is also used to: obtain file data migration instructions on the target client, wherein the file data migration instructions include a source storage system, a target storage system, and a migration path; during the migration process, simultaneously distribute file data write requests associated with the migration path to the source storage system and the target storage system; and after the migration is completed, update the binding relationship of the migration path to point to the target storage system.

[0457] As an optional solution, the above-mentioned device is also used to: read storage binding rules from a configuration file, wherein the configuration file contains path regular expressions, storage system identifiers, and permission configurations; parse the configuration file and generate a routing policy table in memory; and when an update to the configuration file is detected, dynamically load the updated storage binding rules and simultaneously update the routing policy table.

[0458] As an optional solution, the above-mentioned device is also used to: convert standard file operation instructions into native protocol instructions of the target storage system; standardize the error codes returned by the native protocol; and generate compatibility results by simulating the operation when the target storage system does not support the current operation.

[0459] As an optional solution, the above-mentioned device is also used to: generate a virtual merged directory when multiple storage systems are found to have directories with the same path; perform deduplication processing according to a preset conflict resolution strategy and update the virtual merged directory; and synchronize the update operation of the virtual merged directory to the corresponding multiple storage systems.

[0460] As an optional solution, the above-mentioned device is used to perform deduplication processing and update the virtual merge directory according to a preset conflict resolution strategy in the following manner: if the metadata attributes of files with the same name are consistent, the files in the storage system bound to the same path are retained; if the metadata attributes of files with the same name are inconsistent, the corresponding file is selected as the target file according to the predefined conflict resolution strategy, wherein the conflict resolution strategy is based on at least one of storage system priority, user configuration rules, or file version status.

[0461] As an optional solution, the above-mentioned device is used to synchronize the update operation of the virtual merge directory to the corresponding multiple storage systems in the following manner: when a file is created in the virtual merge directory, data is written to the storage system associated with the virtual merge directory in the storage binding rules; when a file is deleted in the virtual merge directory, the deletion operation is performed in the storage system bound to the file.

[0462] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0463] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0464] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program.

[0465] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0466] Figure 27 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0467] It should be noted that, Figure 27 The computer system 2700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0468] like Figure 27 As shown, the computer system 2700 includes a central processing unit (CPU) 2701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 2702 or programs loaded from storage section 2708 into random access memory (RAM). The RAM 2703 also stores various programs and data required for system operation. The CPU 2701, ROM 2702, and RAM 2703 are interconnected via a bus 2704. An input / output interface 2705 (I / O interface) is also connected to the bus 2704.

[0469] The following components are connected to the input / output interface 2705: an input section 2706 including a keyboard, mouse, etc.; an output section 2707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 2708 including a hard disk, etc.; and a communication section 2709 including a network interface card such as a local area network card, modem, etc. The communication section 2709 performs communication processing via a network such as the Internet. A drive 2710 is also connected to the input / output interface 2705 as needed. Removable media 2711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on the drive 2710 as needed so that computer programs read from them can be installed into the storage section 2708 as needed.

[0470] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 2709, and / or installed from removable medium 2711. When the computer program is executed by central processing unit 2701, it performs various functions defined in the system of this application.

[0471] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 2709, and / or installed from removable media 2711. When the computer program is executed by central processing unit 2701, it performs various functions provided in the embodiments of this application.

[0472] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described document processing method is also provided. This electronic device may be... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 28 As shown, the electronic device includes a memory 2802 and a processor 2804. The memory 2802 stores a computer program, and the processor 2804 is configured to execute the steps in any of the above method embodiments via the computer program.

[0473] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0474] Optionally, in this embodiment, the processor may be configured to execute the methods in the embodiments of this application via a computer program.

[0475] Alternatively, as those skilled in the art will understand, Figure 28 The structure shown is for illustrative purposes only. Figure 28 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 28 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 28 The different configurations shown.

[0476] The memory 2802 can be used to store software programs and modules, such as the program instructions / modules corresponding to the file processing method and apparatus in this embodiment. The processor 2804 executes various functional applications and data processing by running the software programs and modules stored in the memory 2802, thereby implementing the aforementioned file processing method. The memory 2802 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 2802 may further include memory remotely located relative to the processor 2804, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 2802 may be used, but is not limited to, for storing information such as files. As an example, such as... Figure 28 As shown, the memory 2802 may include, but is not limited to, the acquisition module 2602, the search module 2604, and the receiving module 2606 in the file processing device. Furthermore, it may include, but is not limited to, other module units in the file processing device, which will not be elaborated upon in this example.

[0477] Optionally, the aforementioned transmission device 2806 is used to receive or send data via a network. Specific examples of the network may include wired and wireless networks. In one example, the transmission device 2806 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 2806 is a radio frequency (RF) module used for wireless communication with the Internet.

[0478] In addition, the aforementioned electronic device also includes: a display 2808 for displaying the aforementioned document; and a connection bus 2810 for connecting the various module components in the aforementioned electronic device.

[0479] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0480] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of an electronic device reads computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the electronic device to perform the file processing method provided in the various alternative implementations of the above-described file processing aspect.

[0481] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store methods for performing the embodiments of this application.

[0482] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0483] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0484] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0485] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0486] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0487] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0488] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0489] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.< / id>

Claims

1. A file processing method, characterized in that, include: A file operation request is obtained on the target client, wherein the file operation request includes a target path, the target path is used to map to the corresponding storage system according to storage binding rules, the storage binding rules include the binding relationship between the path and the storage system, and the target client is mounted with at least two heterogeneous storage systems, the access methods of the at least two heterogeneous storage systems are different; In response to the file operation request, a target storage system is located on the target client according to the target path, wherein the at least two storage systems include the target storage system, and the target storage system corresponds to the target path; In response to finding the target storage system, the target client receives a file processing result returned by the target storage system, wherein the file processing result represents the result obtained by processing the target file based on the file operation request, and the target file represents the file stored in the target storage system indicated by the target path.

2. The method according to claim 1, characterized in that, The step of responding to the file operation request by locating the target storage system on the target client according to the target path includes: In response to the file operation request, the hierarchical structure of the target path is parsed, and a path matching queue is generated; Based on the path binding priority in the storage binding rules, the storage system with the highest matching degree is selected from the path matching queue as the target storage system; When multiple storage systems are bound to the same path, at least one storage system is selected as the target storage system according to a preset conflict policy.

3. The method according to claim 2, characterized in that, The step of selecting the storage system with the highest matching degree from the path matching queue as the target storage system according to the path binding priority in the storage binding rules includes: Select a binding path that is the same as the target path, wherein the binding path represents the path bound to the storage system in the storage binding rule; If no binding path exists that is identical to the target path, select the binding path corresponding to the longest prefix of the target path; In the case where there are multiple binding paths corresponding to the longest prefix, the binding path is selected according to a preset weight; The storage system corresponding to the binding path is used as the target storage system.

4. The method according to claim 1, characterized in that, The method further includes: In response to a storage system release command, the binding relationship between the target storage system and the corresponding path is released, and the data corresponding to the corresponding path is migrated to another storage system, wherein the target path includes the corresponding path; or In response to the storage system release command, the binding relationship between the target storage system and the corresponding path is suspended, while the data corresponding to the corresponding path is retained. During the removal process, if there are incomplete file operation requests in the target storage system, the removal operation is delayed until the request is completed or a preset timeout period is reached. The target path includes the corresponding path.

5. The method according to claim 4, characterized in that, The step of responding to a storage system release command, releasing the binding relationship between the target storage system and the corresponding path, and migrating the data corresponding to the path to another storage system includes: In response to the storage system release command, the amount of file data corresponding to the corresponding path is detected; The file data corresponding to the corresponding path is migrated in batches according to the preset migration rate threshold; After the migration is complete, delete the binding relationship between the target storage system and the corresponding path.

6. The method according to claim 1, characterized in that, The method further includes: If the file operation request includes a file write request, verify the target client's write permission to the target path; If the target path is bound to multiple storage systems, at least one storage system is selected as the target storage system according to a preset write policy, wherein the write policy includes: writing data to the storage system with the highest priority; and synchronously writing data to the storage system with write permissions. When a write operation fails, error handling is performed according to the preset number of retries or rollback rules.

7. The method according to claim 1, characterized in that, In response to locating the target storage system, receiving the file processing result returned by the target storage system on the target client includes: In response to finding the target storage system, the file data returned by each target storage system is converted into a uniform format, wherein different target nodes of the target path are mounted with different storage systems; Conflict detection is performed on file data under the same path returned by multiple target storage systems, and the target file is determined by filtering according to preset conflict rules. The target client receives the target file, which is encapsulated in a standard file system interface format, as the result of the file processing.

8. The method according to claim 1, characterized in that, The method further includes: Obtain file data migration instructions on the target client, wherein the file data migration instructions include the source storage system, the target storage system, and the migration path; During the migration process, file data write requests associated with the migration path are simultaneously distributed to both the source storage system and the target storage system; Once the migration is complete, the binding relationship of the migration path will be updated to point to the target storage system.

9. The method according to claim 1, characterized in that, The method further includes: The storage binding rules are read from the configuration file, which includes path regular expressions, storage system identifiers, and permission configurations. The configuration file is parsed and an in-memory routing policy table is generated; When a configuration file update is detected, the updated storage binding rules are dynamically loaded and the routing policy table is updated simultaneously.

10. The method according to claim 1, characterized in that, The method further includes: Convert standard file operation commands into native protocol commands of the target storage system; Standardize the mapping of error codes returned by the native protocol; When the target storage system does not support the current operation, a compatibility result is generated by simulating the operation.

11. The method according to claim 1, characterized in that, The method further includes: When directories with the same path are detected in multiple storage systems, a virtual merged directory is generated; The virtual merge directory is updated by performing deduplication according to the preset conflict resolution strategy; the update operation of the virtual merge directory is then synchronized to the corresponding multiple storage systems.

12. The method according to claim 11, characterized in that, The step of deduplication according to a preset conflict resolution strategy and updating the virtual merge directory includes: If files with the same name have identical metadata attributes, the file in the storage system bound to the same path will be retained; If the metadata attributes of files with the same name are inconsistent, the corresponding file is selected as the target file according to a predefined conflict resolution strategy, wherein the conflict resolution strategy is based on at least one of storage system priority, user configuration rules, or file version status.

13. The method according to claim 11, characterized in that, The step of synchronizing the update operation of the virtual merged directory to the corresponding multiple storage systems includes: When a file is created in the virtual merge directory, data is written to the storage system associated with the virtual merge directory in the storage binding rule; When a file in the virtual merge directory is deleted, the deletion operation is performed in the storage system to which the file is bound.

14. A document processing device, characterized in that, include: The acquisition module is used to acquire file operation requests on the target client, wherein the file operation request includes a target path, the target path is used to map to the corresponding storage system according to storage binding rules, the storage binding rules include the binding relationship between the path and the storage system, and the target client is mounted with at least two heterogeneous storage systems, the at least two heterogeneous storage systems have different access methods; A search module is configured to, in response to the file operation request, search for a target storage system on the target client according to the target path, wherein the at least two storage systems include the target storage system, and the target storage system corresponds to the target path; A receiving module is configured to receive, on the target client, a file processing result returned by the target storage system in response to finding the target storage system, wherein the file processing result represents the result obtained by processing the target file based on the file operation request, and the target file represents the file stored in the target storage system indicated by the target path.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed by a processor, performs the method described in any one of claims 1 to 13.

16. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1 to 13.

17. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 13 through the computer program.

Images