Email file transfer task management methods, devices, computer equipment, and storage media

By dividing the attachment file into multiple data fragments and storing them in a distributed storage system, and combining the parameters of the receiving end and the storage node to determine the transmission strategy, the file size limit and transmission stability issues in the email service system are solved, and efficient and stable large file transmission across terminals is achieved.

CN123123310APending Publication Date: 2026-11-17CHINA MOBILE INTERNET CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611324538.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-28
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing email service systems suffer from problems such as file size limitations, single transmission paths, difficulty in guaranteeing speed and stability when multiple terminals download concurrently, lack of adaptive scheduling capabilities, insufficient fault tolerance, and imperfect fragmentation verification and reassembly mechanisms during file transfer.

Method used

The attachment file is divided into multiple data fragments and stored in multiple target storage nodes of a pre-set distributed storage system. The optimal data transmission strategy is determined based on the physical operating parameters of the receiving end and the performance parameters of the target storage nodes, so as to achieve efficient and stable transmission in cross-terminal, high-concurrency scenarios.

Benefits of technology

Bypassing the file size limitations of traditional mail servers, it improves the balanced utilization of storage resources and data reliability, enables efficient and stable large file transfer across terminals, and reduces transmission failure rate and bandwidth waste.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123310A_ABST
    Figure CN123123310A_ABST
Patent Text Reader

Abstract

This disclosure relates to a method, apparatus, computer device, and storage medium for managing email file transfer tasks. It includes responding to an upload operation for an attachment file from an initiating end, obtaining the file size of the attachment file, and, if the file size exceeds an attachment upload threshold, splitting the attachment file into multiple data fragments; storing the multiple data fragments into multiple target storage nodes within a preset distributed storage system; responding to a transfer request for the attachment file from a receiving end, obtaining the physical operating parameters of the receiving end and the node performance parameters of each target storage node; determining a data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node, and controlling the multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy. This allows the data transmission strategy to dynamically adapt to the hardware capabilities of different terminals and the real-time status of nodes, ultimately achieving efficient and stable transmission in cross-terminal, high-concurrency scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of file transfer, and in particular to a method, apparatus, computer device, and storage medium for managing email file transfer tasks. Background Technology

[0002] With the development of information technology, email has become an important way of file transfer. However, most email service systems in related technologies adopt a centralized architecture, which has multiple bottlenecks. For example, the size of attachment files is limited (usually about 25MB), which cannot meet the needs of large-sized file transfer such as high-definition videos and design documents. In addition, relying on a single storage and transmission path makes it difficult to guarantee transmission speed and stability when the network fluctuates or the node load is uneven, and the defects are more prominent when multiple terminals download concurrently.

[0003] To overcome file size limitations, third-party cloud storage solutions have been introduced. However, while these solutions alleviate the size issue, they introduce new problems such as cumbersome operation, broken links, limited storage space, and privacy leaks. Furthermore, they fail to address inherent limitations such as single transmission paths and multi-device compatibility. In addition, existing solutions generally lack adaptive scheduling capabilities for heterogeneous devices and dynamic network conditions, lack real-time monitoring and evaluation mechanisms for node performance, have insufficient fault tolerance, and have inadequate fragment verification and reassembly mechanisms, making it difficult to handle abnormal situations such as node failures, fragment loss, or damage. Summary of the Invention

[0004] In view of this, the present disclosure provides a method, apparatus, computer device, and storage medium for managing email file transfer tasks, in order to solve the problems existing in the related technologies.

[0005] A first aspect of this disclosure provides a method for managing email file transfer tasks. The method includes: in response to an upload operation of an attachment file by an initiating end, obtaining the file size of the attachment file, and when the size of the attachment file exceeds an attachment upload threshold, dividing the attachment file into multiple data fragments; storing the multiple data fragments into multiple target storage nodes within a preset distributed storage system; in response to a transfer request for the attachment file by a receiving end, obtaining the physical operating parameters of the receiving end and the node performance parameters of each target storage node; determining a data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node, and controlling the multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy.

[0006] A second aspect of this disclosure provides an email file transfer task management apparatus, applied to the email file transfer task management method of the first aspect. The apparatus includes: a segmentation module, configured to, in response to an upload operation of an attachment file by an initiating end, obtain the file size of the attachment file and, when the file size of the attachment file exceeds an attachment upload threshold, segment the attachment file into multiple data fragments; a storage module, configured to store the multiple data fragments into multiple target storage nodes within a preset distributed storage system; an acquisition module, configured to, in response to a transmission request for an attachment file by a receiving end, acquire the physical operating parameters of the receiving end and the node performance parameters of each target storage node; and a transmission module, configured to, based on the physical operating parameters and the performance parameters of each target storage node, determine a data transmission strategy and, based on the data transmission strategy, control the multiple target storage nodes to transmit the corresponding data fragments to the receiving end.

[0007] A third aspect of this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described email file transfer task management method.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium having a computer program / operation instructions stored thereon, which, when executed by a processor, implement the steps of the above-described email file transfer task management method.

[0009] A fifth aspect of this disclosure provides a computer program product that, when executed by a processor, implements the steps of the above-described email file transfer task management method.

[0010] The above-mentioned at least one technical solution adopted in the embodiments of this disclosure can achieve the following beneficial effects: by dividing attachment files exceeding the attachment upload threshold into multiple data fragments and distributing them to multiple target storage nodes in a preset distributed storage system, the file content is no longer stored locally on the mail server, but relies on the preset distributed storage system to achieve reliable storage of large files; on this basis, when the receiving end initiates a transmission request, by obtaining the physical operating parameters of the receiving end and the node performance parameters of each target storage node, the optimal data transmission strategy is determined for the receiving end based on the bidirectional sensing results, and multiple target storage nodes are controlled to transmit the corresponding data fragments to the receiving end according to the data transmission strategy, so that the data transmission strategy can be dynamically adapted to the hardware capabilities of different terminals and the real-time status of nodes, ultimately achieving efficient and stable transmission of large files in cross-terminal, high-concurrency scenarios. Attached Figure Description

[0011] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is a schematic diagram of a process for managing email file transfer tasks according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram of the structure of an email file transfer task management device provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of a computer system provided in an embodiment of the present disclosure; Figure 5 A schematic diagram of a computer program product provided according to an embodiment of this disclosure. Detailed Implementation

[0013] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0014] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0015] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] With the rapid development of information technology, email has become an indispensable part of daily communication, especially in the field of file transfer, where it is widely used in data exchange scenarios between individuals and enterprises. Email service systems in related technologies typically employ a centralized architecture, storing and forwarding attachments through built-in servers. However, this architecture suffers from multiple technical bottlenecks in file transfer. First, most email service systems, considering server load and storage costs, impose strict limits on the size of attachments, usually around 25MB, making it impossible to directly send large files such as high-definition videos, design documents, and large datasets as attachments. Second, existing email service systems typically forward attachments based on a single storage and transmission path. When network bandwidth fluctuates or node load varies significantly, file transfer speed and stability are difficult to guarantee effectively. Especially when multiple terminals concurrently download the same file, the differences in hardware performance between different terminals and uneven network bandwidth exacerbate the transmission deficiencies of the single-path architecture, easily leading to sudden drops in download speed, transmission interruptions, or even task failures.

[0019] To address the aforementioned file size limitation issue, third-party cloud storage services offer an alternative: users can first upload attachments to the third-party cloud storage, then paste the shared link returned by the cloud storage into the email body. The recipient can then click the link to be redirected to the third-party cloud storage page to download the attachments. However, this solution only alleviates the surface-level problem of file size limitations and does not address the inherent shortcomings of email service systems in terms of single transmission paths and multi-terminal concurrency. Instead, it introduces new risks such as cumbersome operation procedures, easily invalidated links, limited storage space, and data privacy leaks.

[0020] In addition, existing email file transmission solutions still have significant shortcomings in the following aspects: First, they lack the ability to adaptively schedule based on terminal heterogeneity and network dynamics, and cannot dynamically adjust transmission strategies according to differences in terminal performance and changes in the network environment; Second, they lack a real-time monitoring and evaluation mechanism for storage node performance, making it difficult to detect and avoid anomalies such as node latency and packet loss in a timely manner; Third, they lack fault tolerance, and when storage nodes fail or their performance deteriorates, they lack dynamic path switching and automatic recovery mechanisms, which can easily lead to transmission interruptions; Fourth, the fragmentation verification and reassembly mechanism is imperfect, and it lacks the ability to detect and automatically repair anomalies such as fragment loss or damage during transmission in real time.

[0021] To address the aforementioned issues, this disclosure provides an email file transfer task management method. By segmenting attachments exceeding an attachment upload threshold into multiple data fragments and distributing them across multiple target storage nodes within a pre-defined distributed storage system, the file content is no longer stored locally on the email server but is reliably stored using the pre-defined distributed storage system. Furthermore, when the receiving end initiates a transfer request, the system obtains the physical operating parameters of the receiving end and the node performance parameters of each target storage node. Based on bidirectional sensing results, it determines the optimal data transfer strategy for the receiving end and controls multiple target storage nodes to transfer the corresponding data fragments to the receiving end according to this strategy. This allows the data transfer strategy to dynamically adapt to the hardware capabilities of different terminals and the real-time status of nodes, ultimately achieving efficient and stable transfer of large files across terminals and in high-concurrency scenarios.

[0022] The email file transfer task management method provided in this disclosure can be executed by a terminal or by a chip applied to the terminal.

[0023] For example, the aforementioned terminals may include one or more of the following: mobile phones, tablets, wearable devices, in-vehicle devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, handheld computers (PDAs), and wearable devices based on augmented reality (AR) and / or virtual reality (VR) technologies. They may also include, but are not limited to, remote control devices, wearable devices, streetlights, home appliances, and other smart terminals. This disclosure does not impose specific limitations on these aspects.

[0024] Figure 1 This is a schematic diagram illustrating a process for managing email file transfer tasks according to an embodiment of this disclosure. Figure 1 As shown, the email file transfer task management method, applied to an email service system, includes: S101: In response to the initiator's upload operation for an attachment file, the file size of the attachment file is obtained. If the file size of the attachment file exceeds the attachment upload threshold, the attachment file is split into multiple data fragments. The attachment upload threshold is the maximum attachment file size supported by the email service provider, and the initiator is the sender's email client.

[0025] In one embodiment, in response to a user's upload operation for an attachment file triggered in the email client (i.e., the initiating end), the file size of the attachment file can be obtained, and the file size of the attachment file can be compared with the attachment upload threshold to determine whether the upload conditions of the email service system are met. If the determination result is not met, the file uploading process is triggered, that is, the attachment file is divided into multiple data fragments according to the file size of the attachment file.

[0026] For example, when a user uploads a large attachment (such as a video file named "large_video.mp4" with a file size of 150 MB) in an email client, the email service system will first check the file size. If the value exceeds the attachment upload threshold set by the email service system (such as 25 MB), it is determined that the file does not meet the conditions for direct upload, and the file upload process is then triggered.

[0027] During the on-chain process, based on the 150MB file size of the attachment, an adaptive sharding algorithm can be used to divide the file into multiple data shards. Specifically, the adaptive sharding algorithm can calculate the number of shards and determine the size of each data shard based on a preset maximum shard granularity and the file size, ensuring that the size of each data shard does not exceed the preset maximum shard granularity. The maximum shard granularity refers to the upper limit of the maximum byte size allowed for a single data shard, i.e., the upper limit of the size of each data shard when splitting the attachment file.

[0028] For example, the maximum granularity of the fragment can be set to 10MB. For a 150MB attachment file, the adaptive fragmentation algorithm can divide the attachment file into 15 data fragments, each with a size of 10MB. If the attachment file size is 156MB, it will be divided into 15 10MB data fragments and 1 6MB data fragment.

[0029] As can be seen, this embodiment of the disclosure compares the file size with the attachment upload threshold and triggers adaptive fragmentation upload only when the attachment file exceeds the threshold. This not only supports direct upload of regular attachments but also provides differentiated transmission paths for large files, eliminating the need for users to use third-party cloud storage or manually split files, thus improving operational convenience. Simultaneously, the adaptive fragmentation algorithm dynamically calculates the number of data fragments and the byte size of each data fragment based on the preset maximum fragmentation granularity and the actual file size of the attachment file. This ensures that each data fragment does not exceed the granularity limit, making the data fragment size controllable and consistent. This avoids transmission timeouts or node overload caused by excessively large data fragments, as well as the expansion of scheduling overhead caused by excessively fragmented data fragments. Therefore, it achieves a balance between storage efficiency and transmission stability, making it particularly suitable for scenarios involving high-frequency transmission of large attachments in enterprise email service systems, effectively reducing transmission failure rates and bandwidth waste.

[0030] S102, multiple data fragments are stored into multiple target storage nodes within a preset distributed storage system. The preset distributed storage system consists of multiple storage nodes, and the target storage node is the node allocated to store the data fragments of the current attachment file. The preset distributed storage system is the distributed storage system that the email service system interfaces with. The email service system, by calling its data write interface, stores the data fragments of the attachment file into the preset distributed storage system, thereby bypassing the file size limitations of traditional email servers.

[0031] In some embodiments, after dividing the attachment file into multiple data fragments according to its size, the email service system can call the data writing interface (e.g., a distributed content addressing system interface) provided by a preset distributed storage system to write the multiple data fragments into multiple target storage nodes within the preset distributed storage system. Specifically, the email service system can adopt a distributed content addressing storage mechanism, distributing the data fragments to different target storage nodes based on their real-time availability and load. Each target storage node stores the data fragment it is responsible for and its corresponding storage location, thereby achieving distributed storage of the attachment file.

[0032] For example, after entering the on-chain process, the email service system can divide the attachment file into multiple data fragments and call the distributed content addressing system interface to write these data fragments into the cache of the intelligent cloud storage gateway. Then, the intelligent cloud storage gateway distributes the data fragments to multiple target storage nodes in the preset distributed storage system for persistent storage, so that the content of the attachment file is no longer carried on the local storage of the email service system, but is stored in a distributed manner on multiple target storage nodes in the preset distributed storage system.

[0033] Here, the intelligent cloud storage gateway is an access layer node deployed on the email service system side, used for temporarily caching data fragments and providing an HTTP access entry point; the preset distributed storage system is a remote persistent storage layer, used for ultimately storing data fragments and supporting content-addressed access. The two have an upstream-downstream relationship where data is first cached by the intelligent cloud storage gateway and then distributed by the intelligent cloud storage gateway to the preset distributed storage system for persistent storage. It should be noted that the aforementioned intelligent cloud storage gateway is not a necessary pre-component for implementing the embodiments of this disclosure. In actual deployment, after completing file fragmentation, the email service system can also directly write the data fragments to multiple target storage nodes in the preset distributed storage system by calling the data write interface of the preset distributed storage system, without going through a separate gateway caching layer.

[0034] In some embodiments, after writing data fragments to a preset distributed storage system, this disclosure embodiment can further process each data fragment based on MerkleDAG encoding to generate a fragment identifier for each data fragment and a file identifier for identifying the overall content of the attachment file. Specifically, each data fragment can be hashed using the SHA-256 hash algorithm to obtain a fragment hash value corresponding to each data fragment; then, based on the MerkleDAG encoding structure, according to the storage order of each data fragment, a MerkleDAG tree is constructed using the corresponding fragment hash values ​​as leaf nodes, and the hash values ​​of parent nodes are calculated layer by layer upwards until the root hash value is obtained, which is then used as the file identifier of the attachment file. MerkleDAG encoding is a data organization method based on hash values ​​and having a directed acyclic graph structure, where each storage node stores its content and the hash digests of its child nodes, so that any data modification will change its hash value, supporting content addressing, data deduplication, and integrity verification. The file identifier is stored in multiple target storage nodes of the preset distributed storage system through the network broadcast protocol provided by the preset distributed storage system. This allows any target storage node in the preset distributed storage system to locate and obtain the data fragments of the attachment file based on the file identifier, thereby supporting parallel downloading and fault-tolerant access across multiple nodes.

[0035] In some embodiments, after obtaining the file identifier of the attachment file, this disclosure embodiment can further associate the file identifier with the mapping relationship between the data fragments stored in each target storage node, so that each target storage node in the preset distributed storage system can locate and obtain the corresponding data fragment based on the file identifier. Simultaneously, the file identifier can be inserted into a specified structure field of the email body by parsing the Multipurpose Internet Mail Extensions (MIME) encoding format, and a corresponding Hypertext Transfer Protocol (HTTP) gateway access path can be generated for the file identifier through a Uniform Resource Locator (URL) conversion module. Finally, by performing a field mapping binding operation, the mapping relationship between the file identifier and the HTTP gateway access path is stored in the index database of the email service system. For example, after dividing a 150MB video file into 38 data fragments, the hash value of each data fragment is calculated, and then the fragments are organized layer by layer based on MerkleDAG encoding to finally generate a unique file identifier for the file (such as "QmXoypiz"), and generate a corresponding HTTP gateway access path for it (such as "http: / / ipfs.io / ipfs / QmXoypiz").

[0036] After the mapping relationship is stored, when the recipient opens the email, the email client on the receiving end can parse the file identifier from the specified structure field in the email body, find and obtain the corresponding HTTP gateway access path according to the mapping relationship stored in the index database, and then download the corresponding data fragment from the preset distributed storage system through the HTTP gateway access path and reassemble it into a complete attachment file. In this way, the recipient only needs to click the link in the email client on the receiving end to access and download the attachment file, no longer hindered by the traditional email file size limit.

[0037] MIME is an internet standard used to extend email protocols to support the transmission and representation of multimedia content such as non-US Information Interchange Standard (IIS) characters, images, audio, and video. In this embodiment, it is used to structurally insert file identifiers into specified fields of the email body. URL is an address format used to identify and locate resources on the internet. In this document, it is used to generate HTTP access addresses pointing to content stored in a preset distributed storage system, allowing recipients to directly access and download attachments. HTTP is an application layer protocol used in distributed, collaborative, and hypermedia information systems. It is the foundation of data communication on the World Wide Web and is used in this document as a transmission protocol for clients to access and download data fragments in a preset distributed storage system through a gateway.

[0038] As can be seen, this embodiment of the disclosure bypasses the traditional hard limit on file size of mail servers by dividing the attachment file into multiple data fragments and storing them in a preset distributed storage system, so that the attachment file is no longer stored locally on the mail server. It adopts a distributed content-addressable storage mechanism and distributes the data fragments according to the real-time availability and load of each target storage node, avoiding single points of failure and improving the balanced utilization of storage resources and data reliability. It generates file identifiers based on MerkleDAG encoding, so that any data modification changes its hash value, realizing content addressing, data deduplication, and integrity verification. By inserting the file identifier into a specified field in the email body and generating a corresponding HTTP gateway access path, and storing the mapping relationship in an index database, the recipient can directly download the data fragments from the preset distributed storage system through this path and reassemble them into a complete attachment, reducing the storage and bandwidth pressure on the email service system and improving the convenience for users to obtain large attachments.

[0039] S103, in response to the receiving end's request to transmit the attachment file, obtains the receiving end's physical operating parameters and the node performance parameters of each target storage node. The receiving end is the email client on the recipient's terminal.

[0040] In one embodiment, when the receiving end initiates a request to transmit an attachment file, the email service system can respond to the transmission request and obtain the physical operating parameters of the receiving end and the node performance parameters of each target storage node. The physical operating parameters characterize the hardware resource configuration and current operating status of the receiving end, while the node performance parameters characterize the real-time service capabilities of each target storage node.

[0041] The physical operating parameters mentioned above may include at least one of the following: the operating system kernel version information of the receiving end, processor model and clock frequency, available physical memory capacity, remaining disk space size, disk file system type, input / output access bandwidth, processor load status, network interface protocol stack and real-time bandwidth utilization, average round-trip time of network link, network packet loss rate, and congestion window value; the node performance parameters may include at least one of the following: bandwidth status of each target storage node, average response latency, and data integrity.

[0042] Based on this, the email service system can send parameter reporting commands to the receiving end, which then collects and reports its local physical operating parameters according to the commands. Simultaneously, the email service system can send performance probe requests to all target storage nodes storing the target data shards through the node performance monitoring module. Each target storage node responds to the probe request and returns its current node performance parameters. Based on this, the email service system can aggregate and structure the acquired physical operating parameters and the node performance parameters of each target storage node, serving as the data foundation for subsequently determining the terminal transmission strategy. Here, the email service system also includes a node performance monitoring module, deployed on the email service system side and communicatively connected to each target storage node of the pre-defined distributed storage system, used to collect real-time performance status parameters of each target storage node.

[0043] As can be seen, the embodiments of this disclosure achieve bidirectional state awareness before the transmission task is executed by synchronously obtaining the hardware performance parameters of the receiving end and the node performance parameters of each storage node when the receiving end initiates a transmission request. This provides a comprehensive data foundation for subsequently formulating the optimal data transmission strategy, thereby avoiding the strategy mismatch problem caused by insufficient unilateral awareness and improving the matching degree between the transmission strategy and the actual network environment and terminal capabilities.

[0044] S104 determines the data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node, and controls multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy.

[0045] In one embodiment, after obtaining the physical operating parameters of the receiving end and the node performance parameters of each target storage node, the scheduler of the email service system inputs the physical operating parameters and node performance parameters into a pre-trained transmission strategy prediction model, and determines the data transmission strategy based on the output of the prediction model. The scheduler is the functional unit in the email service system responsible for allocating data fragment transmission tasks and scheduling download paths.

[0046] Here, the transmission strategy prediction model can evaluate the maximum number of concurrent downloads and cache processing capabilities that the receiver can currently support based on the receiver's physical operating parameters. At the same time, it can evaluate the available transmission bandwidth and response efficiency of each storage node based on the node performance parameters of each target storage node. By combining the above bidirectional evaluation results, a data transmission strategy adapted to the current state of the receiver and the target storage node is determined. This data transmission strategy may include at least one of the following: the fragment scheduling priority of each data fragment, the number of data fragments that the receiver can download simultaneously, and the cache management method.

[0047] Each data fragment is stored on a different target storage node, and the fragment scheduling priority determines the startup order of the corresponding storage node. Data fragments with higher scheduling priority are scheduled first, and their respective target storage nodes are prioritized for transmission. Data fragments with lower scheduling priority are scheduled later, and their respective target storage nodes are executed later. Subsequently, the email service system's scheduler controls multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy. For example, the transmission fragment size in the data transmission strategy controls the size of each transmitted data packet, the maximum number of concurrent connection channels controls the number of simultaneous download connections, and the cache fragment buffer size and cache refresh cycle configure the receiving end's cache management method.

[0048] For example, suppose the physical operating parameters of a receiving end indicate that its processor load is 30%, available physical memory is 16 gigabytes, network bandwidth utilization is 70%, and network transmission latency is 20 milliseconds. These parameters indicate that the receiving end has strong concurrent processing capabilities. Meanwhile, the data fragments of the attached file are stored in target storage node A, target storage node B, and target storage node C, respectively. Target storage node A has a response latency of 20 milliseconds and available bandwidth of 200 megabits per second; target storage node B has a response latency of 50 milliseconds and available bandwidth of 100 megabits per second; and target storage node C has a response latency of 10 milliseconds and available bandwidth of 150 megabits per second. Based on this, the email service system can comprehensively evaluate each target storage node according to the above parameters. It is evident that target storage node C has the best performance, followed by target storage node A, while target storage node B is relatively poor. Therefore, the data transmission strategy determined by the email service system can be as follows: data fragments stored on target storage node C are set to the highest fragment scheduling priority and transmitted first; data fragments stored on target storage node A are set to the second highest fragment scheduling priority and transmitted second best; and data fragments stored on target storage node B are set to the lowest fragment scheduling priority and transmitted last. Simultaneously, the target fragment transmission granularity is set to 4MB, and the receiving end cache refresh cycle is 5 seconds. Subsequently, the email service system controls the three target storage nodes to transmit their respective stored data fragments to the receiving end according to the above priority strategy, so that the receiving end receives data in 4MB granularities and refreshes its cache every 5 seconds.

[0049] As can be seen, the embodiments of this disclosure achieve bidirectional perception by obtaining the physical operating parameters of the receiving end and the node performance parameters of each target storage node, and determine the optimal data transmission strategy based on this. This enables the data transmission strategy to dynamically adapt to the hardware capabilities of different terminals and the real-time status of nodes, avoiding the problem of low efficiency of static configuration strategies in heterogeneous terminals and dynamic network environments. Ultimately, it achieves efficient and stable transmission of large files in cross-terminal, high-concurrency scenarios.

[0050] In some embodiments, step S104 includes inputting physical operating parameters and node performance parameters of each target storage node into a pre-trained transmission strategy prediction model to obtain a data transmission strategy; wherein the data transmission strategy includes at least one of the following: the size of each data fragment, the number of data fragments downloaded simultaneously by the receiving end, the cache management method of the receiving end, and the fragment scheduling priority of each data fragment.

[0051] Specifically, the transmission strategy prediction model can be a prediction model trained using a recurrent neural network based on historical terminal operating parameters and historical node performance parameters. The data transmission strategy includes at least one of the following: the size of each data fragment, the number of data fragments downloaded simultaneously by the receiving end, the buffer management method of the receiving end, and the fragment scheduling priority of each data fragment. Here, each data fragment has its own fragment scheduling priority, and each data fragment has a different fragment scheduling priority. The initial fragment scheduling priority is allocated by the scheduler according to the transmission data transmission strategy, and can be dynamically adjusted subsequently based on the concurrent conflict detection results.

[0052] For example, when a receiving end initiates a file transfer request, its physical operating parameters can be obtained, including that it is running a Windows 10 operating system, its processor is an "Intel Core i7-10750H" with a clock speed of 2.6GHz, its available physical memory is 16 gigabytes, its processor load is currently at 30% utilization, its network bandwidth utilization is 70%, and its average round-trip time is 20 milliseconds. Simultaneously, the node performance parameters of each target storage node are obtained through the node performance monitoring module: target storage node A has an available bandwidth of 200 megabits per second and an average response latency of 20 milliseconds; target storage node B has an available bandwidth of 100 megabits per second and an average response latency of 50 milliseconds; and target storage node C has an available bandwidth of 150 megabits per second and an average response latency of 10 milliseconds.

[0053] After the aforementioned physical operating parameters and the node performance parameters of each target storage node are input into the pre-trained transmission strategy prediction model, the transmission strategy prediction model can evaluate the upper limit of its concurrent processing based on the hardware capabilities of the receiving end. At the same time, it can comprehensively score and rank the service capabilities of each target storage node based on the bandwidth and latency data of each target storage node, and output a data transmission strategy adapted to the current state of the receiving end and the target storage node.

[0054] Continuing with the example above, we can see that target storage node C has the best performance, and its corresponding data fragment is set to the highest fragment scheduling priority; target storage node A is next, and its corresponding data fragment is set to the second highest fragment scheduling priority; target storage node B has relatively poor performance, and its corresponding data fragment is set to the lowest fragment scheduling priority. Simultaneously, the policy specifies a target fragment granularity of 4 megabytes, a maximum number of concurrent connection channels of 8, a cache fragment buffer size of 100 megabytes, and a cache refresh cycle of 5 seconds. These parameters are used to control the current file transmission process. For example, each 256-megabyte storage fragment is further split into 4-megabyte data packets during transmission to ensure that the attachment file can be transmitted from the target storage node to the receiving end at the optimal speed and efficiency. The cache fragment buffer refers to a dedicated cache area allocated in the local memory of the receiving end to temporarily store data fragments downloaded in parallel from each target storage node. Once all fragments are received or reach a certain quantity, they are then uniformly reassembled into a complete attachment file or written to disk in batches.

[0055] As can be seen, the embodiments of this disclosure can achieve bidirectional state awareness and intelligent matching between the terminal side and the target storage node side by obtaining the physical operating parameters of the receiving end and the node performance parameters of each target storage node and inputting them into the transmission strategy prediction model. The strategy prediction model evaluates the actual processing capability of the receiving end based on its physical operating parameters, and performs comprehensive sorting based on the node performance parameters of each target storage node. The evaluation results are mapped to the sharding scheduling priority of each data shard, so that the shards corresponding to the target storage node with the best performance are scheduled first, and the shards corresponding to the target storage node with poor performance are scheduled later, realizing differentiated task allocation based on real-time performance. At the same time, the fine-grained transmission parameters output by the strategy prediction model, such as sharding granularity, concurrent connection count, and caching strategy, can achieve fine matching between the receiving end capability and the target storage node status, so that the data transmission strategy can dynamically adapt to the terminal hardware capability and the real-time status of the node, significantly improving the efficiency and stability of large file transmission in cross-terminal, high-concurrency scenarios.

[0056] In some embodiments, step S101 includes obtaining the file size, file type, and file data structure of the attachment file; and dividing the attachment file into multiple data fragments using a preset fragmentation algorithm based on the file size, file type, and file data structure. The content structure characteristics are used to characterize the internal organization of the attachment file. For video files, the content structure characteristics include encoding parameters, duration, resolution, and frame data block position offsets; for document files, the content structure characteristics include chapter structure or page structure. The preset fragmentation algorithm may include at least one of a fixed-length fragmentation algorithm, a variable-length fragmentation algorithm, and a content-based fragmentation algorithm.

[0057] In one embodiment, when a user triggers an attachment file upload operation in their email client, the email service system can obtain the file size, file type, and file data structure of the attachment file. For example, if a user uploads a video file named "large_video.mp4", the email service system obtains that the attachment file is 150MB in size, an MP4 format video file, and its file data structure may include metadata such as encoding parameters, duration, and resolution, as well as recording the position offset of each frame data block and multiple video frame data blocks arranged in chronological order. The MIME type parameter of the attachment file is also obtained as video / mp4. After obtaining these parameters, the email service system can compare the obtained file size of 150MB with the attachment upload threshold of 25MB. Since 150MB is greater than 25MB, the system determines that the attachment file does not meet the conditions for direct upload and triggers the file uploading process.

[0058] In the file upload process, the email service system's gateway uses a preset fragmentation algorithm to divide the attachment file into multiple data fragments based on at least one of the following: file size, file type, and content structure characteristics. These fragments are then written to the smart cloud storage gateway's cache. For example, for video files, whose content structure includes different video segments and frame data, the email service system can segment the video by time period, such as every 5 minutes as a data fragment, ensuring that each data fragment contains a complete video segment and preventing data fragment interruptions from affecting playback. For document files such as PDFs, whose content structure includes chapters and mixed text and images, the preset fragmentation algorithm can segment based on file chapters, ensuring that each data fragment contains complete chapter content and preventing images and text from being spread across multiple fragments, thus improving the efficiency of reassembly after download.

[0059] In this way, by obtaining the file size, file type, and file data structure of the attachment files, and using a preset segmentation algorithm for adaptive segmentation, the segmentation strategy can be dynamically determined according to the different types and file data structures of the attachment files. This ensures that each data segment contains complete content units (such as complete video frames, complete chapter content, and complete text records). As a result, in the subsequent reassembly stage after transmission is completed, the problem of low reassembly efficiency or data corruption caused by critical data crossing segments is avoided, which significantly improves the rationality of file segmentation, the efficiency of reassembly, and the integrity of data content.

[0060] In some embodiments, step S102 includes performing a hash calculation on each data shard to generate a shard identifier for each data shard; determining the storage mapping relationship between each data shard and its corresponding target storage node based on the shard identifier of each data shard and the node network topology of a preset distributed storage system; and storing the multiple data shards into their respective target storage nodes according to the storage mapping relationship. Here, the node network topology refers to the connection relationship and layout structure between storage nodes in the preset distributed storage system at the physical location and network communication levels, including physical distance between nodes, network hop count, communication latency, etc., used to select the optimal node for data storage or transmission during the scheduling process.

[0061] Specifically, after completing the above data fragmentation, the email service system can perform hash calculations on each data fragment to calculate the SHA-256 hash value of each data fragment, thereby obtaining the fragment identifier of each data fragment. Based on MerkleDAG encoding, different data fragments are processed to generate file identifiers corresponding to attachment files, such as "QmXoypiz", which uniquely identifies the content of the attachment file.

[0062] Subsequently, the email service system can determine the storage mapping relationship between each data shard and its corresponding target storage node based on the shard identifier of each data shard and the node network topology of the preset distributed storage system. According to the storage mapping relationship, multiple data shards are stored in their respective target storage nodes. At the same time, through the intelligent cloud storage gateway network broadcast protocol, the file identifier corresponding to the attachment file is stored in multiple target storage nodes of the preset distributed storage system. This means that the data shards of the attachment file will be distributed and stored on different target storage nodes, without relying on a single centralized storage. These target storage nodes will keep copies of the file to ensure the high availability and fault tolerance of the attachment file.

[0063] The intelligent cloud storage gateway distributes data shards to a pre-defined distributed storage system (such as a storage system built on object storage services like AWS S3, Google Cloud Storage, or Azure Blob) for persistent storage. AWS S3 (Amazon Simple Storage Service) is an object storage service provided by a certain company and is one of the widely used cloud storage standards in the industry. Google Cloud Storage is an object storage service provided by a certain company's platform, used to store and retrieve any amount of data on Google Cloud. Azure Blob (Azure Blob Storage) is an object storage service provided by a certain company's platform specifically for storing unstructured data.

[0064] Based on this, during the email sending process, the email service system can parse the email's MIME encoding format, insert the file identifier corresponding to the attachment file into a specified field in the email body, and generate the HTTP gateway access path corresponding to that file identifier. For example, the email service system can generate a URL path for "QmXoypiz", such as "http: / / ipfs.io / ipfs / QmXoypiz". By performing a field mapping binding operation, the file identifier is mapped one-to-one with this HTTP gateway access path and stored in the email service system's index database.

[0065] In this way, by performing hash calculations on each data shard to generate a unique shard identifier and generating a file identifier based on MerkleDAG encoding, a verifiable content fingerprint is provided for attachment files. By determining the storage mapping relationship based on each data shard identifier and the node network topology, the data shards are stored in their corresponding target storage nodes. Combined with the broadcast protocol, the file identifier of the attachment file is stored on multiple target storage nodes across the network, realizing decentralized distributed storage and multi-replica redundancy of data shards, avoiding the risk of single point of failure, and improving the reliability and fault tolerance of storage. By inserting the file identifier of the attachment file into the email body and generating an HTTP gateway access path, the mapping relationship is stored in the index database of the email service system, allowing recipients to access and download the complete file simply by clicking a link. This breaks through the traditional email file size limit and improves the file transfer capability and user experience of the email service system.

[0066] In some embodiments, step S104 includes obtaining the real-time performance status parameters of each target storage node; determining an initial target storage node for downloading the initial data fragment from multiple target storage nodes based on the data transmission strategy and the real-time performance status parameters of each target storage node, and downloading the initial data fragment from the initial target storage node; if the real-time performance status parameters of the initial target storage node do not meet the preset performance status conditions during the download process, controlling the preset distributed storage system to perform a node switching operation, switching to another target storage node that meets the preset performance status conditions and stores a copy of the initial data fragment, and continuing to execute the task of downloading the initial data fragment. Here, the real-time performance status parameters not meeting the preset performance status conditions means that the real-time operating indicators of the target storage node, such as the response time and packet loss rate, exceed the qualified range preset by the email service system, indicating that the real-time performance status parameters of the target storage node are insufficient to support normal download services.

[0067] Specifically, the email service system's scheduler can obtain the current sharding task queue and the node information of each target storage node in the preset distributed storage system (node ​​information includes at least one of the target storage node's node identifier, network address, storage capacity, and list of stored data shards). It can also obtain the Directed Acyclic Graph (DAG) index path and metadata for each data shard. Here, the DAG index path refers to the addressing path of the data shard in the MerkleDAG data structure, used to uniquely identify the position of the data shard in the entire attachment file DAG and its hierarchical dependencies with other data shards. The metadata includes the shard size, storage location, parent node references, and shard hash value for each data shard. The DAG describes the hierarchical relationships and dependency structure between data shards. Each data shard acts as a node, linked by hash values ​​to form a tree structure, ultimately generating a root hash value as the unique file identifier for the entire file.

[0068] Next, the scheduler can obtain the real-time performance status parameters of each target storage node. These parameters include at least one of the following: response time, effective bandwidth, feedback reporting latency, instantaneous packet loss rate, and block-level download success rate at the current moment. Then, based on the data transmission strategy and the real-time performance status parameters of each target storage node, the scheduler determines the initial target storage node for downloading the initial data fragments from multiple target storage nodes, and downloads the initial data fragments from the initial target storage node. The data transmission strategy includes a scheduling algorithm based on the bandwidth-delay product and historical stability weights. This algorithm prioritizes target storage nodes whose network topology distance from the receiving end is less than a preset distance threshold and whose response stability is higher than a preset stability threshold as the initial target storage nodes.

[0069] It should be noted that, in the embodiments of this disclosure, the initial data fragment refers to the data fragment to be downloaded, which can be any one of the multiple data fragments after the attachment file has been divided (for example, it can be the first data fragment or any subsequent data fragment), and does not specifically refer to the first data fragment in the order of arrangement; the initial target storage node refers to the target storage node selected for the initial data fragment in the current download task, which can be the same as or different from the target storage node selected in the previous download task, and does not specifically refer to the target storage node selected for the first time.

[0070] During the download process, if the real-time performance status parameters of the target storage node currently providing download services do not meet the preset performance status conditions, a node switching strategy is triggered. A new target storage node that meets the preset performance status conditions and stores the initial data shard copies is selected, and a target storage node replacement instruction is sent to the preset distributed storage system to replace the current target storage node. Subsequently, after switching to the new target storage node, the scheduler re-initiates the download request for the incomplete shards to the new target storage node and updates the node allocation record and shard download path in the task scheduling table to continue downloading the incomplete data shards through the newly selected target storage node.

[0071] For example, in an attachment file transfer task, suppose the receiving end initiates a file download request for a file of 500MB. The scheduler obtains the current fragment task queue. The attachment file is divided into 50 data fragments, each with a size of 10MB. The fragment task queue contains the DAG index path and corresponding metadata for each data fragment. The metadata includes the storage location, hash digest, and parent node reference relationship for each data fragment. Taking the first data fragment as an example, its DAG index path is "Qm12345abc", and its metadata includes: the fragment size is 10MB, the storage location is the target storage node A, the parent node reference relationship is "Qm67890xyz", and the hash digest is "abc123hash". Based on this, after obtaining the complete metadata information for each data fragment, the email service system prepares to start the data transmission.

[0072] Before initiating data transmission, the email service system obtains real-time performance status parameters of multiple target storage nodes. In this embodiment, it is assumed that the target storage nodes storing data fragments containing attachment files include target storage node A, target storage node B, and target storage node D. Target storage node A stores the initial data fragments, target storage node D also stores copies of the initial data fragments, and target storage node B also stores some data fragments. Although target storage node C is also a storage node in the preset distributed storage system, it does not store data fragments containing attachment files and is therefore not among the candidate target storage nodes.

[0073] Assume the performance parameters of the target storage nodes are as follows: Target storage node A has a response latency of 20ms, an effective bandwidth of 200Mbps, and a historical stability of 95%; target storage node B has a response latency of 50ms, an effective bandwidth of 100Mbps, and a historical stability of 85%; and target storage node D has a response latency of 12ms, an effective bandwidth of 180Mbps, and a historical stability of 92%. Based on this, the scheduler can, according to the data transmission strategy and the performance parameters of the target storage nodes, comprehensively consider the bandwidth-latency product and historical stability weights, and prioritize selecting nodes whose network topology distance from the receiving end is less than a preset distance threshold and whose response stability is higher than a preset stability threshold as the initial target storage nodes. After evaluation, target storage node A meets the above conditions and stores initial data fragments; therefore, the scheduler selects target storage node A as the initial target storage node and begins downloading the initial data fragments.

[0074] During transmission, the email service system collects real-time performance status parameters of each target storage node through the scheduler. In this embodiment, the preset performance status conditions are: transmission latency less than 50ms, effective bandwidth not less than 150Mbps, packet loss rate less than 1%, and block-level download success rate greater than 98%. Based on this, when the scheduler detects that the real-time performance status parameters of the target storage node A currently providing download services do not meet the above preset performance status conditions (for example, the transmission latency of target storage node A is 65ms, exceeding the 50ms threshold), a node switching strategy is triggered.

[0075] The scheduler then selects the best-performing target storage node from other target storage nodes that meet the preset performance conditions and store copies of the initial data fragments as a backup node. Evaluation shows that target storage node D currently has a transmission latency of 12ms, an effective bandwidth of 180Mbps, a packet loss rate of 0.1%, and a block-level download success rate of 98.5%, meeting the preset performance conditions. Furthermore, target storage node D stores the same data copies as target storage node A. Therefore, the remaining data fragment transmission tasks that are not yet completed on target storage node A can be switched to target storage node D for continued execution.

[0076] It should be noted that the effective execution of the node switching strategy relies on a crucial prerequisite: the replacement target storage node already stores a data copy identical to the fragment to be switched. In this embodiment, although target storage node C has excellent network performance (assuming a transmission latency of 10ms, effective bandwidth of 150Mbps, and historical stability of 90%), it stores data fragments of other attachment files and does not include a copy of the initial data fragment, thus failing to meet the switching conditions. Target storage node D, on the other hand, stores a data copy identical to the initial data fragment, and its current real-time performance parameters meet the preset performance status conditions, therefore it is selected as the replacement node. Since the preset distributed storage system employs a multi-replica redundancy mechanism, the same data fragment is typically stored across multiple target storage nodes. Therefore, when the performance of a target storage node deteriorates, the email service system can select the best-performing target storage node from those storing copies of the same fragment to continue transmission, thereby achieving a smooth migration of the task.

[0077] After the node switch is completed, the scheduler updates the task scheduling table, re-registers the download requests of the incomplete data fragments from target storage node A to target storage node D, and synchronously updates the download path information of each data fragment to ensure that subsequent download requests can be correctly routed to the new target storage node, thereby achieving smooth migration and continuous execution of the transmission task.

[0078] In summary, by acquiring real-time performance status parameters of each target storage node through the email service system's scheduler and dynamically scheduling data transmission strategies, the system can adaptively select the optimal node as the initial target storage node based on network topology distance and stability thresholds. This fully leverages the bandwidth and response advantages of multiple nodes, improving the startup efficiency of fragment downloads. During the download process, when the real-time performance status parameters of the current target storage node deteriorate to the point where they no longer meet preset performance conditions, the email service system automatically triggers a node switching strategy. It selects the best-performing node from other target storage nodes storing the same fragment replicas to continue transmission, and simultaneously updates the task scheduling table and fragment download path. This effectively avoids download interruptions or timeouts caused by single-point performance fluctuations, ensuring the continuity and stability of download tasks. Furthermore, by setting reasonable performance status parameter thresholds and node selection conditions, the distributed storage system can proactively switch nodes in the early stages of target storage node performance degradation, reducing the impact of transmission latency and packet loss on user experience and improving the overall efficiency and reliability of attachment file downloads.

[0079] In some embodiments, if the real-time performance status parameters of the initial target storage node do not meet the preset performance status conditions during the download process, the process switches to another target storage node that meets the preset performance status conditions and stores a copy of the initial data shard, and continues to execute the task of downloading the initial data shard, including: The system acquires the feedback reporting time of each target storage node and determines the feedback time difference based on the feedback reporting times of any two target storage nodes. It calculates the feedback interference index based on the difference in real-time performance status parameters between any two target storage nodes and the feedback time difference. If the feedback interference index is greater than a preset oscillation threshold, the system is determined to be in a critical state of scheduling path oscillation. When the system is determined to be in a critical state of scheduling path oscillation, the node switching operation is suspended. Parameter adjustment operations are performed on the two target storage nodes with the largest difference in real-time performance status parameters, and the corresponding feedback interference index is recalculated. If the recalculated feedback interference index is less than the preset oscillation threshold for multiple consecutive detection periods, the node switching operation is resumed.

[0080] Specifically, the scheduler can collect real-time performance status parameters of each target storage node. These parameters include at least one of the following: average response time, effective bandwidth, feedback reporting latency, real-time packet loss rate, and block-level download success rate. Based on this, the feedback reporting time of each target storage node can be obtained, and the feedback phase difference can be determined based on the feedback reporting times of any two target storage nodes. Subsequently, based on the difference in real-time performance status parameters between any two target storage nodes and the feedback phase difference, the feedback interference index is calculated. The feedback disturbance response formula is:

[0081] Where t represents the moment when the current email service system calculates the feedback interference index; Indicates the total number of target storage nodes; i and j both represent the target storage node identifiers; This represents the feedback time difference between target storage node i and target storage node j at time t; This indicates the feedback reporting latency of target storage node i; This indicates the feedback reporting delay of the target storage node j; This represents the average response latency of target storage node i at time t; This represents the real-time packet loss rate of the target storage node j at time t; This represents the effective bandwidth of target storage node i at time t; This represents the effective bandwidth of the target storage node j at time t.

[0082] In some embodiments, if the feedback interference index is greater than a preset oscillation critical threshold, the preset distributed storage system is determined to be in a critical state of oscillation in the multi-node scheduling path. At this time, the scheduler suspends the execution of the regular node switching strategy, freezes the switching status of the currently selected target storage nodes, and delays the update of the feedback status sampling window to achieve short-term locking of the scheduling path. In the frozen state, the scheduler can identify the two target storage nodes that cause the most severe interference (i.e., the two with the greatest degree of misalignment in feedback reporting time) based on the changes in the feedback time difference of each target storage node. The scheduler extends the feedback reporting cycle of these two target storage nodes respectively, further staggering their feedback reporting times to reduce resource contention and scheduling conflicts caused by simultaneous reporting of feedback data, thereby stabilizing the response performance of each target storage node in the preset distributed storage system. During the frozen period, if the feedback interference index is detected to have dropped below the preset oscillation critical threshold for multiple consecutive detection cycles, the frozen state is automatically lifted, the dynamic node selection strategy is restored, and the feedback time difference sequence, node switching records, and scheduling response behavior during this oscillation period are written to the feedback interference log database. Here, the feedback status sampling window refers to the time range within which the scheduler collects performance feedback data from each target storage node. In other words, the scheduler does not process feedback data every moment, but collects data within a fixed time window and processes it uniformly.

[0083] For example, within the scheduling period of a certain sharding task, the scheduler retrieves the current sharding task queue and synchronously obtains information on five target storage nodes in a preset distributed storage system associated with the task. First, it extracts the historical performance feedback records of each target storage node and the performance status parameters monitored in real-time during the current period. Specifically, this includes the average response time, instantaneous effective bandwidth, feedback reporting latency, packet loss rate, and block-level download success rate of each target storage node. For instance, if target storage node A and target storage node B are the two most active storage nodes in the current schedule, where target storage node A has a current average response time of 210ms, an effective bandwidth of 12Mbps, and a feedback reporting latency of 300ms; and target storage node B has a response time of 150ms, a bandwidth of 18Mbps, a feedback reporting latency of 200ms, and a current instantaneous packet loss rate of 2.1% for target storage node B.

[0084] Based on this, the scheduler can include the feedback reporting delay difference between target storage node A and target storage node B, i.e., the time difference between 300ms and 200ms, in the feedback phase offset calculation process to obtain the feedback time difference. The time interval is 60ms. Combining this with the data change rate from the two target storage nodes over the past two cycles, and based on the feedback disturbance response formula, the feedback disturbance index at the current time t is calculated. The value is 0.684. Since the current preset path oscillation critical threshold of the distributed storage system is 0.6, and Ω(t) has exceeded this threshold, it is judged that the scheduling path is in a critical state of multi-node scheduling path oscillation, which poses a risk of repeated switching of the scheduling path between nodes, amplified response latency, and sharp fluctuations in bandwidth utilization.

[0085] When the scheduling path is detected to be in a critical state of multi-node scheduling path oscillation, the mail service system can immediately suspend the regular node switching mechanism, freeze the scheduling path of the selected target storage node in the current sharding task, and synchronously delay the performance status sampling update window to keep the scheduling path in a short-term locked state to avoid further scheduling oscillations. During the frozen state, the scheduler determines that the interference-dominant pair is AB based on the slope change trend of the feedback time difference curves of target storage node A and target storage node B. The scheduler extends the feedback reporting period of target storage node A from the original 200ms to 350ms, and the feedback period of target storage node B from 180ms to 300ms. A phase delay strategy is implemented for the feedback merging window of this pair of target storage nodes to ensure that the time when their feedback values ​​enter the scheduling calculation module is non-overlapping, thereby reducing the instantaneous coupling strength between their feedback disturbances and stabilizing the system response matrix.

[0086] Subsequently, the feedback interference index can be calculated for three consecutive cycles. When the monitoring results show that Ω(t) is 0.59, 0.51, and 0.43 respectively, it can be confirmed that the feedback interference index has been continuously below the preset oscillation critical threshold of 0.6. Based on this, it can be judged that the feedback interference has reached a stable state, the freeze state can be randomly lifted, and the dynamic node selection strategy can be restored. The feedback time difference sequence, interference index change trajectory, node switching delay response, and scheduling behavior within the locked window involved in this path oscillation event are completely recorded in the feedback interference log database.

[0087] In this way, by constructing a feedback time difference sequence between target storage nodes and quantifying the feedback interference index based on the feedback disturbance response formula, the scheduler can accurately identify the critical state of oscillation in the multi-node scheduling path. If the feedback interference index is greater than the preset oscillation critical threshold, the email service system is determined to be in the critical state of oscillation in the multi-node scheduling path. The execution of the regular node switching strategy is suspended. The switching status of the currently selected target storage nodes is frozen and the update of the feedback status sampling window is delayed to achieve short-term locking of the scheduling path. Then, based on the changes in the feedback time difference of each target storage node, the main interference target storage node pair is identified. The feedback reporting cycle of the main interference target storage node pair is extended to stagger the feedback reporting times of the two, thereby reducing resource contention and scheduling conflicts caused by simultaneous reporting of feedback data, thus reducing the nonlinear feedback coupling strength and stabilizing the system node response matrix characteristics. If the feedback interference index is less than the preset oscillation critical threshold for multiple consecutive detection cycles, the frozen state is automatically lifted and the dynamic node selection strategy is restored. Therefore, in the scenario of multi-node concurrent scheduling of a pre-defined distributed storage system, the embodiments of this disclosure can effectively suppress node switching oscillations, avoid amplified response latency and drastic fluctuations in bandwidth utilization, and improve the stability of task scheduling and resource utilization when multiple terminals are downloading concurrently.

[0088] In some embodiments, after step S104, the method further includes performing fragment verification on each data fragment that has been transmitted; storing the data fragments that pass the verification into the local cache of the receiving end, and assigning a position index to each data fragment that passes the verification; discarding the data fragments that fail the verification, and re-downloading the data fragments from other target storage nodes that store the data fragments that fail the verification, and re-performing the fragment verification; after all data fragments pass the verification, concatenating all the data fragments according to the position index of each data fragment to obtain the attachment file at the receiving end.

[0089] Specifically, a hash value can be calculated for each downloaded data fragment to obtain its actual hash value. This hash value is then compared with the content address hash in the DAG node records to determine if the current data fragment's hash value matches. If they match, the data fragment is marked as verified and stored in the receiver's local cache directory. Simultaneously, a unique location index is assigned to each verified data fragment, and it is added to the reassembled block table. If they do not match, the current target storage node is recorded as the source of failure, and the failed data fragment is discarded. The data fragment is then downloaded again from other target storage nodes, and its hash verification is repeated until all data fragments pass the hash verification.

[0090] In some embodiments, after all data fragments have passed verification, all data fragments can be concatenated and merged according to the position index of each data fragment to obtain the attachment file at the receiving end. Specifically, based on all data fragments that have passed verification in the reassembly block table, the verified data fragments can be concatenated and merged according to the preset order and position index of each data fragment to reconstruct the complete attachment file; then, a hash check is performed on the merged attachment file to determine whether the attachment file is consistent with the original file identifier. If they are consistent, the reassembled attachment file is marked as completed and saved to the target storage location. If they are inconsistent, a global rollback mechanism is triggered, that is, the reassembly cache is cleared and the process rolls back to the last valid verification point, and then the data fragments that failed verification are fetched again until all data fragments have been downloaded and passed the hash check.

[0091] For example, after downloading the first 10MB data fragment on the terminal, its SHA-256 hash value is calculated as "abc123def456". This hash value is then compared with the hash value in the DAG record. A match is found, so the verification passes. A location index "block1_index" is then assigned, and the fragment is added to the reassembled block table. When downloading the second data fragment, the hash value is calculated as "xyz789ghi012", but it does not match "abc987def654" in the DAG record. Therefore, the current target storage node is marked as the source of failure, and the data fragment is re-downloaded from another node. After the re-download, the hash value is "abc987def654", which matches the DAG record, and the verification passes. A location index "block2_index" is then assigned, and the fragment is added to the reassembled block table. After all fragments pass verification, they are concatenated and merged according to their location indices. Then, a hash check is performed on the complete file. If it matches the original file identifier, it is saved to the target storage location.

[0092] If the hash value of the merged complete file does not match the original file identifier, a global rollback mechanism is triggered. This involves clearing the reassembly cache and rolling back to the last valid verification point, then re-fetching the failed fragments. For example, if data fragments 4 and 5 fail verification, with data fragment 4 stored on target storage node A and data fragment 5 stored on target storage node B, then data fragment 4 can be retrieved again from target storage node A (response latency 15ms, bandwidth 1Gbps), and data fragment 5 can be retrieved again from target storage node B (response latency 20ms, bandwidth 800Mbps). After the re-download is complete, each data fragment is re-hash-verified. If the verification passes, the file is reassembled, and then the complete file hash verification is performed again until it matches the original file identifier. Finally, the file is marked as successfully transmitted and saved to the target storage location.

[0093] In this way, by calculating the hash value of each downloaded data fragment and comparing it with the content address hash in the DAG node record, precise verification of data fragments is achieved. This ensures that only fragments with consistent hash values ​​are stored in the local cache and assigned a position index to be added to the reassembled block table, while fragments with inconsistent hash values ​​are discarded and re-downloaded from other target storage nodes. This effectively avoids data corruption or accumulation of fragment errors caused by node failures or network transmission anomalies. After all fragments pass verification, they are concatenated according to their position indexes, and a hash verification is performed on the complete file to confirm consistency with the original file identifier, further ensuring the integrity of the reassembled file. When the hash verification of the complete file is inconsistent, a global rollback mechanism is triggered to clear the reassembled cache and roll back to the last valid verification point. Only the fragments that failed verification are re-fetched, not all fragments. Combined with node performance, the optimal target storage node is selected for re-downloading, which significantly improves fault tolerance and recovery efficiency, reduces unnecessary retransmission overhead, and thus ensures both data integrity and transmission reliability when multiple terminals are downloading concurrently in a distributed storage environment.

[0094] In some embodiments, step S104 includes, for any data shard, obtaining the access frequency of the data shard, the number of receivers requesting the data shard, the shard size of the data shard, and the average response cost of all target storage nodes storing the data shard; for the data shard, determining the concurrent conflict density value of the data shard based on the access frequency of the data shard, the number of receivers requesting the data shard, the shard size of the data shard, and the average response cost of all target storage nodes storing the data shard; if the concurrent conflict density value is greater than or equal to a preset conflict density threshold, then marking the data shard as a conflict segment; according to the node network topology of the preset distributed storage system, generating temporary copies of the data shard marked as a conflict segment among other target storage nodes whose network distance to the target storage node where the data shard marked as a conflict segment is currently located meets a preset constraint condition and is in an idle state; and reducing the shard scheduling priority of the data shard.

[0095] Specifically, based on the current transmission task request information collected by the scheduler before the allocation of data sharding tasks, the access request frequency, the number of receivers requesting the data shard, the shard size, and the average response cost of all target storage nodes storing the data shards can be extracted for each data shard. The access request frequency includes the number of receivers requesting the same data shard within a preset time window, and the average response cost is the ratio of response time to effective bandwidth. Subsequently, a conflict density evaluation function is called to determine the concurrent conflict density value for each data shard based on its access request frequency, the number of receivers requesting the data shard, the shard size, and the average response cost of all target storage nodes storing the data shards. If the concurrent conflict density value is greater than or equal to a preset conflict density threshold, the corresponding data shard is marked as a conflict segment, and a temporary copy of the marked conflict segment is generated in the preset distributed storage system, and the sharding scheduling priority of the data shard is reduced.

[0096] Here, the expression for the conflict density assessment function is:

[0097] in, This represents the concurrent conflict density value; T represents the current time window; This represents the b-th available data fragment of the a-th file; Request this data shard within the current time window. The number of terminals; Data sharding The size of the fragment; The average response cost for all target storage nodes in the data sharding scheme, where the response cost is defined as the ratio of a node's response time to its effective bandwidth.

[0098] In some embodiments, after calculating the concurrent conflict density value, the concurrent conflict density value can be compared with a preset conflict density threshold. If the concurrent conflict density value is greater than the preset conflict density threshold, the corresponding target segment is marked as a conflict segment, and a pre-schedule adjustment process is triggered.

[0099] Specifically, the pre-scheduling adjustment process includes: First, performing hot distribution scheduling of replicas for the shard identifier corresponding to the conflict segment. Based on the preset distributed storage system's node network topology, generating temporary replicas among target storage nodes whose physical distance and network hop count meet preset constraints. These preset constraints include a logical hop count not exceeding a preset hop count threshold and a physical distance latency less than a preset latency threshold. Among all candidate target storage nodes that meet these preset constraints, the target storage node with the fewest logical hops and the lowest physical distance latency is selected as the target storage node for generating the temporary replica. In this way, by generating temporary replicas, the access channels for the data shard are expanded, and the number of schedulable nodes is increased, thereby reducing the impact of concentrated access to the data shard under high concurrency.

[0100] After completing the hot distribution scheduling of replicas, the preset distributed storage system can update the shard scheduling priority index parameter in the current shard task queue, apply priority delay control to the target shard marked as a conflict segment, and lag its scheduling order behind one or more shards whose concurrent conflict density value is lower than a preset low threshold.

[0101] For example, before a file transfer task starts, the scheduler can collect request information for each data segment within the current scheduling cycle. Taking the currently processed data segment as an example, the record shows that in the past 15 seconds, a total of 95 terminals initiated access requests to this data segment. These terminals include enterprise user desktop terminals, mobile devices, and IoT data collection nodes. Based on this, the scheduler extracts the access request frequency of this segment as 95 times / 15 seconds, and combines it with the terminal operating parameters and historical scheduling trajectories of each initiating terminal to determine that this data segment is a high-frequency access target in the current scheduling cycle.

[0102] Meanwhile, the scheduler obtains the real-time performance status of the six target storage nodes currently storing the data shard through a preset node performance acquisition module. The average response time is 180ms, and the average effective bandwidth is 8Mbps, resulting in an average response cost of 22.5ms / Mbps for this data shard. The above number of requesting terminals... Fragment size in bytes Average response cost Input the conflict density evaluation function and calculate the concurrent conflict density value μ of the partition at the current task scheduling time as 427.5.

[0103] Since the preset conflict density threshold set by the distributed storage system is 350, and the concurrent conflict density value μ=427.5 of this data shard exceeds the threshold, it is determined that the data shard is in a high-concurrency conflict state and is marked as a conflict segment. After marking, a pre-scheduling adjustment process can be automatically executed: First, based on the current node network topology, four idle target storage nodes are selected from the set of nodes with a logical hop count of no more than 3 and a physical distance latency of less than 30ms. Based on the replica hot distribution strategy, temporary replicas are generated for the data shard in the four selected idle target storage nodes to expand the access path and alleviate the single-node load concentration phenomenon. After the temporary replicas are generated, the preset distributed storage system updates the shard scheduling priority index parameter of the data shard in the current shard task queue, downgrading its original shard scheduling priority by one level, so that its scheduling order lags behind other data shards with low conflict density, preventing the scheduler from continuing to schedule the conflict segment shard in the current cycle, thereby reducing access resource contention across the entire network.

[0104] By introducing a concurrent conflict density quantification evaluation function, and taking the number of requesting terminals, fragment size, and average node response cost as joint consideration factors, the system can accurately identify high-population and high-conflict-risk target fragments before scheduling execution, overcoming the shortcomings of traditional static scheduling strategies that cannot perceive the degree of real-time concurrent contention. When the conflict density exceeds the threshold, the preset distributed storage system automatically triggers hot replica distribution scheduling, dynamically generating temporary replicas among redundant nodes that meet the physical distance and hop count conditions based on network topology constraints. This effectively expands the access channels for high-conflict fragments and distributes the load pressure on single points, significantly reducing the risk of node overload and access conflict probability during concurrent downloads by multiple terminals. At the same time, by implementing priority delay control on conflict segments, their scheduling order is delayed compared to low-conflict fragments, avoiding centralized scheduling of the same hot fragment by the scheduler during resource-scarce periods, further alleviating network-wide bandwidth competition and node response congestion from a time perspective. Therefore, the embodiments of this disclosure can significantly improve the overall throughput of file transmission, reduce the download failure rate and retransmission overhead in dynamic network environments and heterogeneous terminal concurrent scenarios, and ensure the stability and fairness of high-concurrency task scheduling. They are especially suitable for the actual deployment needs of enterprise-level email service systems for simultaneous download of large attachments on multiple terminals.

[0105] In some embodiments, after controlling multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy, the method further includes: acquiring key transmission performance indicators during the file transmission process, storing the key transmission performance indicators in a transmission performance statistics database, and generating a node availability evaluation report; the key transmission performance indicators include file download rate, reassembly efficiency, node response latency, packet loss rate, retransmission count, node stability, bandwidth utilization, and throughput; the node availability evaluation report includes response time, bandwidth fluctuation, and failure rate information of each target storage node; based on the historical key transmission performance indicator data stored in the transmission performance statistics database, terminal performance level labels are marked, and a random forest algorithm is used for model training to construct a terminal performance level evaluation model to evaluate the performance level of the terminal device; the performance level includes high performance, medium performance, and low performance; if the performance level of the terminal device is low performance, the transmission strategy parameters of the terminal device are adjusted until the performance of the terminal device reaches a preset performance standard; the transmission strategy parameters include fragment size, download concurrency, caching strategy, and download path priority.

[0106] Specifically, firstly, based on the transmission log data during file transfer, key transmission performance indicators are obtained and stored in a transmission performance statistics database, along with a node availability evaluation report. These key transmission performance indicators include file download speed, reassembly efficiency, node response latency, packet loss rate, retransmission count, node stability, bandwidth utilization, and throughput. The node availability evaluation report includes information on each node's response time, bandwidth fluctuations, failure rate, and the reliability and stability of each target storage node.

[0107] Then, historical key transmission performance index data can be obtained from a transmission performance statistics database and labeled with terminal performance level tags. A random forest algorithm is then used to train a model to construct a terminal performance level evaluation model, which assesses the terminal's performance level, including high, medium, and low performance. After the terminal performance level evaluation model is trained, the real-time transmission performance index of the current terminal is input into the model, and the model outputs the terminal's performance level. If the performance level output by the model is high or medium, the terminal's performance is determined to meet the preset performance standard; if the performance level output by the model is low, the terminal's performance is determined to not meet the preset performance standard. Next, if the terminal performance does not meet the preset performance standard, the terminal transmission strategy is optimized, i.e., the terminal transmission strategy parameters are adjusted until the terminal performance reaches the preset performance standard, thus obtaining an updated terminal transmission strategy.

[0108] For example, during file transfer, a user is downloading a 1GB file named "large_file.zip". The email service system records the entire download process, obtaining key transmission performance indicators. Assuming the download speed is 50Mbps, the reassembly efficiency is 95%, the node response latency is 40ms, the packet loss rate is 0.2%, the number of retransmissions is 3, the bandwidth utilization is 85%, and the throughput is 45Mbps, these indicators are stored in a transmission performance statistics database. Based on this, the email service system generates a node availability evaluation report, analyzing the performance of each target storage node involved in the download. Target storage node A has a response time of 40ms, bandwidth fluctuation of 5%, and a failure rate of 0.1%; target storage node B has a response time of 55ms, bandwidth fluctuation of 10%, and a failure rate of 0.3%. Therefore, target storage node A is rated as a more reliable target storage node due to its better performance stability and lower failure rate.

[0109] Meanwhile, the email service system uses historical data to retrieve key transmission performance indicators from the transmission performance statistics database and labels terminal performance levels based on this data. For example, Terminal 1's past transmission performance indicators show a download speed of 60Mbps, bandwidth utilization of 90%, low packet loss rate, and high node stability, thus it is rated as a high-performance terminal; Terminal 2's download speed is only 20Mbps, bandwidth utilization of 50%, high packet loss rate, and long node response time, thus it is rated as a low-performance terminal.

[0110] Based on the labeled data, the email service system uses the random forest algorithm to train a terminal performance level evaluation model using historical data to predict the terminal's performance level. For terminal 2, after the performance level evaluation model predicts it to be a low-performance terminal, the email service system optimizes its terminal transmission strategy. Specifically, it adjusts terminal 2's transmission strategy parameters: reducing the data fragment size from 10MB to 5MB, reducing download concurrency from 5 connections to 3, adjusting the caching strategy to a higher frequency of cache refresh, and re-prioritizing download paths, prioritizing target storage nodes with higher bandwidth as download sources. After the optimized terminal transmission strategy is applied to terminal 2, its performance is evaluated again to ensure it meets preset standards. Finally, terminal 2's transmission strategy is updated to better suit its hardware conditions and network environment, thereby ensuring the efficiency of its file transfer process.

[0111] In this way, by collecting key transmission performance indicators during file transfer and storing them in a transmission performance statistics database, a historical performance data accumulation mechanism is constructed, providing a data foundation for node availability evaluation and terminal performance assessment. Based on this, a node availability evaluation report is generated to quantitatively analyze the response time, bandwidth fluctuation, and failure rate of each target storage node, enabling accurate identification of node reliability and helping to prioritize nodes with higher stability in subsequent download tasks. At the same time, a random forest algorithm is used to train a model on historical performance indicator data to construct a terminal performance level evaluation model, classifying terminals into high-performance, medium-performance, and low-performance levels. This allows for adaptive adjustment of transmission strategy parameters for terminals of different performance levels, achieving optimal matching between the transmission strategy and terminal hardware conditions and network environment, effectively avoiding low transmission efficiency caused by insufficient terminal performance or inappropriate strategies. Therefore, based on real-time dynamic scheduling, this embodiment of the present disclosure further forms a closed-loop feedback mechanism of "transmission-evaluation-optimization-retransmission" through statistical analysis of historical data and model training. This mechanism can continuously optimize terminal transmission strategies, significantly improve the overall efficiency and terminal adaptability of subsequent file transmission tasks, and is especially suitable for complex deployment environments with diverse terminals in enterprise email service systems. It effectively reduces transmission bottlenecks and bandwidth waste caused by differences in terminal performance.

[0112] The foregoing primarily describes the solutions provided by the embodiments of this disclosure from the perspective of the server. It is understood that, in order to implement the above functions, the server includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0113] This disclosure embodiment can divide the server into functional units according to the above method example. For example, it can divide each function into separate functional modules, or it can integrate two or more functions into one management module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this disclosure embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0114] By dividing each function into corresponding functional modules, an exemplary embodiment of this disclosure provides an email file transfer task management device, which can be a server or a chip applied to a server. Figure 2 This is a schematic diagram of the structure of an email file transfer task management device provided in one embodiment of the present disclosure. Figure 2 As shown, the email file transfer task management device 200 includes: The segmentation module 201 is used to respond to the upload operation of the attachment file by the initiating end, obtain the file size of the attachment file, and when the size of the attachment file is greater than the attachment upload threshold, segment the attachment file into multiple data fragments; Storage module 202 is used to store multiple data fragments into multiple target storage nodes within a preset distributed storage system; The acquisition module 203 is used to acquire the physical operating parameters of the receiving end and the node performance parameters of each target storage node in response to the receiving end's transmission request for the attachment file. The transmission module 204 is used to determine a data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node, and to control multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy.

[0115] In an optional manner, the transmission module 204 is further configured to input the physical operating parameters and the node performance parameters of each of the target storage nodes into a pre-trained transmission strategy prediction model to obtain the data transmission strategy; wherein, the data transmission strategy includes the sharding scheduling priority of each of the data shards.

[0116] In an optional manner, the segmentation module 201 is further configured to obtain the file size, file type, and file data structure of the attachment file; and to segment the attachment file into multiple data fragments using a preset segmentation algorithm based on the file size, file type, and file data structure.

[0117] In an optional manner, the storage module 202 is further configured to perform a hash calculation on each of the data shards to generate a shard identifier for each of the data shards; determine the storage mapping relationship between each of the data shards and the corresponding target storage node based on the shard identifier of each of the data shards and the node network topology of the preset distributed storage system; and store the multiple data shards into the corresponding target storage node according to the storage mapping relationship.

[0118] In an optional manner, the transmission module 204 is further configured to acquire the real-time performance status parameters of each of the target storage nodes; determine an initial target storage node for downloading the initial data fragment from among the multiple target storage nodes according to the data transmission strategy and the real-time performance status parameters of each target storage node, and download the initial data fragment from the initial target storage node; if the real-time performance status parameters of the initial target storage node do not meet the preset performance status conditions during the download process, control the preset distributed storage system to perform a node switching operation, switch to another target storage node that meets the preset performance status conditions and stores a copy of the initial data fragment, and continue to execute the task of downloading the initial data fragment.

[0119] In an optional manner, the transmission module 204 is further configured to acquire the feedback reporting time of each of the target storage nodes, and determine the feedback time difference based on the feedback reporting times of any two of the target storage nodes; calculate the feedback interference index based on the difference in real-time performance status parameters between any two target storage nodes and the feedback time difference; if the feedback interference index is greater than a preset oscillation critical threshold, it is determined that the preset distributed storage system is currently in a scheduling path oscillation critical state; when it is determined that the preset distributed storage system is in a scheduling path oscillation critical state, the node switching operation is suspended; parameter adjustment operations are performed on the two target storage nodes with the largest difference in real-time performance status parameters among the multiple target storage nodes, and the corresponding feedback interference index is recalculated; if the recalculated feedback interference index is less than the preset oscillation critical threshold for multiple consecutive detection cycles, the node switching operation is resumed.

[0120] In one alternative approach, fragment verification is performed on each data fragment that has been transmitted; the data fragments that pass verification are stored in the local cache of the receiving end, and a position index is assigned to each data fragment that passes verification; the data fragments that fail verification are discarded, and the data fragments are downloaded again from the target storage node that stores the data fragments that fail verification, and fragment verification is performed again; after all the data fragments pass verification, all the data fragments are concatenated and merged according to the position index of each data fragment to obtain the attachment file at the receiving end.

[0121] In an optional manner, the transmission module 204 is further configured to, for any data fragment, obtain the access frequency of the data fragment, the number of receivers requesting the data fragment, the fragment size of the data fragment, and the average response cost of all target storage nodes storing the data fragment; for the data fragment, determine the concurrent conflict density value of the data fragment based on the access frequency of the data fragment, the number of receivers requesting the data fragment, the fragment size of the data fragment, and the average response cost of all target storage nodes storing the data fragment; if the concurrent conflict density value is greater than or equal to a preset conflict density threshold, mark the data fragment as a conflict segment; according to the node network topology of the preset distributed storage system, generate a temporary copy of the data fragment marked as a conflict segment from other target storage nodes whose network distance to the target storage node where the data fragment marked as a conflict segment is currently located meets a preset constraint condition and is in an idle state; and reduce the fragment scheduling priority of the data fragment.

[0122] This disclosure also provides an electronic device, including: at least one processor; a memory for storing at least one processor-executable instruction; wherein the at least one processor is used to execute the instruction to implement the steps of the method disclosed in this disclosure.

[0123] Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present disclosure. Figure 3 As shown, the electronic device 300 includes at least one processor 301 and a memory 302 coupled to the processor 301, which can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0124] The processor 301 described above can also be called a Central Processing Unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 301 or by software instructions. The processor 301 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 302, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 301 reads information from the memory 302 and, in conjunction with its hardware, completes the steps of the method described above.

[0125] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, for example, Figure 4 The computer system 400 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including functions such as those mentioned above. Figure 4 This is a schematic diagram of the structure of a computer system provided in an embodiment of the present disclosure.

[0126] Computer system 400 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0127] like Figure 4As shown, the computer system 400 includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the computer system 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0128] Multiple components in the computer system 400 are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device capable of inputting information into the computer system 400. The input unit 406 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 407 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 408 may include, but is not limited to, a hard disk and an optical disk. The communication unit 409 allows the computer system 400 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, a modem, network card, infrared communication device, wireless communication transceiver, and / or chipset, such as Bluetooth™ device, WiFi device, WiMax device, cellular communication device, and / or the like.

[0129] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0130] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0131] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0133] Figure 5 This is a schematic diagram of a computer program product provided according to an embodiment of the present disclosure. Figure 5 As shown, the computer program product 500 includes a computer program 501, which, when executed by a processor, implements the methods disclosed in the embodiments of this disclosure.

[0134] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0137] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0138] The above description is merely an illustration of some embodiments of this disclosure and the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0139] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for managing email file transfer tasks, characterized in that, include: In response to the upload operation of the attachment file by the initiating end, the file size of the attachment file is obtained, and when the file size of the attachment file is greater than the attachment upload threshold, the attachment file is divided into multiple data fragments; The data fragments are stored in multiple target storage nodes within a pre-defined distributed storage system; In response to the receiving end's request to transmit the attached file, the physical operating parameters of the receiving end and the node performance parameters of each of the target storage nodes are obtained; A data transmission strategy is determined based on the physical operating parameters and the performance parameters of each target storage node, and the data transmission strategy is used to control multiple target storage nodes to transmit the corresponding data fragments to the receiving end.

2. The method according to claim 1, characterized in that, The method for determining a data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node includes: The physical operating parameters and the node performance parameters of each target storage node are input into a pre-trained transmission strategy prediction model to obtain the data transmission strategy; The data transmission strategy includes the sharding scheduling priority for each data shard.

3. The method according to claim 1, characterized in that, The method for dividing the attachment file into multiple data fragments includes: Obtain the file size, file type, and file data structure of the attached file; Based on the file size, file type, and file data structure, the attachment file is divided into multiple data fragments using a preset fragmentation algorithm.

4. The method according to claim 1, characterized in that, The method of storing multiple data shards into multiple target storage nodes within a preset distributed storage system includes: Perform a hash calculation on each of the data shards to generate a shard identifier for each of the data shards; Based on the fragment identifier of each data fragment and the node network topology of the preset distributed storage system, the storage mapping relationship between each data fragment and the corresponding target storage node is determined; According to the storage mapping relationship, the multiple data shards are stored in the corresponding target storage nodes respectively.

5. The method according to claim 1, characterized in that, The method of controlling multiple target storage nodes to transmit corresponding data fragments to the receiving end according to the data transmission strategy includes: Obtain the real-time performance status parameters of each of the target storage nodes; Based on the data transmission strategy and the real-time performance status parameters of each target storage node, an initial target storage node for downloading the initial data fragment is determined from the plurality of target storage nodes, and the initial data fragment is downloaded from the initial target storage node; If the real-time performance status parameters of the initial target storage node do not meet the preset performance status conditions during the download process, the preset distributed storage system is controlled to perform a node switching operation, switching to another target storage node that meets the preset performance status conditions and stores a copy of the initial data shard, and continuing to execute the task of downloading the initial data shard.

6. The method according to claim 5, characterized in that, If the real-time performance status parameters of the initial target storage node do not meet the preset performance status conditions during the download process, the process switches to another target storage node that meets the preset performance status conditions and stores a copy of the initial data shard, and continues to execute the task of downloading the initial data shard. The method includes: The feedback reporting time of each target storage node is obtained, and the feedback time difference is determined based on the feedback reporting times of any two target storage nodes. The feedback interference index is calculated based on the difference in real-time performance status parameters between any two target storage nodes and the feedback time difference. If the feedback interference index is greater than the preset oscillation critical threshold, then the preset distributed storage system is determined to be in a scheduling path oscillation critical state. When it is determined that the preset distributed storage system is in a critical state of scheduling path oscillation, the node switching operation is suspended. For the two target storage nodes with the largest difference in real-time performance status parameters among the multiple target storage nodes, parameter adjustment operations are performed respectively, and the corresponding feedback interference index is recalculated; If the recalculated feedback interference index is less than the preset oscillation critical threshold for multiple consecutive detection cycles, then the node switching operation will resume.

7. The method according to claim 1, characterized in that, After controlling multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy, the method further includes: Perform fragment verification on each of the data fragments that has been transmitted; The data fragments that pass the verification are stored in the local cache of the receiving end, and a position index is assigned to each data fragment that passes the verification. The data fragments that fail the verification are discarded, and the data fragments are downloaded again from other target storage nodes that store the data fragments that fail the verification, and the fragment verification is re-executed. After all the data fragments have passed verification, all the data fragments are concatenated and merged according to the position index of each data fragment to obtain the attachment file at the receiving end.

8. The method according to claim 1, characterized in that, After determining the data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node, and before controlling the multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy, the method further includes: For any of the data shards, obtain the access frequency of the data shard, request the number of receivers for the data shard, the shard size of the data shard, and the average response cost of all target storage nodes storing the data shard; For the data shard, the concurrent conflict density value of the data shard is determined based on the access frequency of the data shard, the number of receivers requesting the data shard, the shard size of the data shard, and the average response cost of all target storage nodes storing the data shard. If the concurrent conflict density value is greater than or equal to the preset conflict density threshold, the data fragment is marked as a conflict segment. Based on the node network topology of the preset distributed storage system, among other target storage nodes that meet the preset constraint conditions in terms of network distance between the data fragment marked as the conflict segment and the target storage node where it is currently located, and are in an idle state, a temporary copy is generated for the data fragment marked as the conflict segment. Reduce the scheduling priority of the data shards.

9. A mail file transfer task management device, characterized in that, include: The segmentation module is used to respond to the upload operation of the attachment file by the initiating end, obtain the file size of the attachment file, and when the file size of the attachment file is greater than the attachment upload threshold, segment the attachment file into multiple data fragments; The storage module is used to store multiple data fragments into multiple target storage nodes within a preset distributed storage system; The acquisition module is used to acquire the physical operating parameters of the receiving end and the node performance parameters of each of the target storage nodes in response to the receiving end's transmission request for the attached file. The transmission module is used to determine a data transmission strategy based on the physical operating parameters and the performance parameters of each target storage node, and to control multiple target storage nodes to transmit the corresponding data fragments to the receiving end according to the data transmission strategy.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.