Data downloading method and device, electronic equipment and medium

Abstract

The invention provides a data downloading method. The data downloading method comprises the following steps: uniformly fragmenting data to be downloaded according to a network condition; distributing downloading threads according to the number and the size of the fragments; the network condition is monitored in real time, when the network condition changes, the sizes of the fragments are dynamically adjusted, and downloading threads are redistributed according to the adjusted sizes of the fragments to complete downloading of all the fragments; and splicing the downloaded fragments into a target file according to a sequence, completing the integrity verification of the target file, storing the target file into a specified directory, and notifying a user that the downloading is successful. According to the invention, the downloading speed and stability are improved, the uncertainty and risk of the user in the using process are reduced, and smoother and more convenient downloading experience is brought to the user.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data downloading method, device, electronic equipment and medium. Background Art

[0002] In the tide of modern information society, the rise of multi-threaded data download technology has profoundly responded to the urgent need for large-scale data transmission efficiency and reliability. With the rapid development of Internet technology, especially the booming cloud computing technology, global data exchange and sharing have reached an unprecedented scale and speed. From commercial data analysis to the rapid acquisition of scientific research data sets, to the instant download of high-definition videos, large-scale games and other content for individual users, the efficiency and stability of data transmission have become key indicators for measuring technological advancement and user experience.

[0003] The traditional single-thread download mode has gradually shown its limitations in the face of growing data volumes and complex network environments. It cannot effectively tap and utilize the potential of network bandwidth, resulting in low download efficiency, especially when dealing with large files or facing high concurrent access. In addition, the single-thread mechanism is highly sensitive to network fluctuations or short interruptions. Once encountered, it often needs to start downloading from the beginning, which greatly affects the continuity and efficiency of data transmission.

[0004] In view of this, multi-threaded data download technology came into being and became an effective means to solve the above problems. The core of this technology is to intelligently divide the data file to be downloaded into multiple data blocks and assign them to different threads for parallel processing. This parallelization strategy not only greatly improves the data transmission rate, can make more full use of existing network resources, but also significantly shortens the user's waiting time. At the same time, the redundant design of multi-threading enhances the robustness of the system. Even if a thread is interrupted due to network problems, other threads can continue to work until the task is completed, thereby effectively ensuring the success rate and stability of the download task.

[0005] With the continuous integration and development of cutting-edge technologies such as big data, artificial intelligence, and the Internet of Things, the application field of multi-threaded data download technology has continued to expand, becoming an important force in promoting the efficient operation of the digital society. It not only promotes the rapid circulation and sharing of data resources, but also provides solid technical support for building a more intelligent and stable network environment. Summary of the invention

[0006] In view of this, the embodiments of the present disclosure provide a data downloading method, device, electronic device and medium, which at least partially solve the problems existing in the prior art.

[0007] In a first aspect, an embodiment of the present disclosure provides a data downloading method, which includes: Evenly shard the data to be downloaded according to the network conditions; Allocate download threads based on the number and size of shards; The network status is monitored in real time, and when the network status changes, the size of the fragments is dynamically adjusted and the download threads are reallocated according to the adjusted fragment sizes to complete the download of all fragments; The downloaded segments are sequentially spliced ​​into the target file, the integrity verification of the target file is completed, the target file is stored in the specified directory, and the user is notified of the successful download.

[0008] According to a specific implementation of the embodiment of the present disclosure, the data is evenly segmented according to the maximum reasonable segment size, network bandwidth and parallel download capability.

[0009] According to a specific implementation method of the embodiment of the present disclosure, if a segment fails to be downloaded, the failed segment is recorded, and a download thread is reallocated to the failed segment until the failed segment is successfully downloaded or a preset retry condition is met.

[0010] According to a specific implementation manner of the embodiment of the present disclosure, a client sends a download request, the client connects to a server and prepares a path or link of a file to be downloaded.

[0011] According to a specific implementation manner of the embodiment of the present disclosure, before downloading data, it is necessary to detect the network connection and measure and obtain the bandwidth parameters of the current network.

[0012] According to a specific implementation manner of the embodiment of the present disclosure, network parameters are initialized based on the detection result of the network connection, and the size of the fragment and the allocation of the download thread are determined based on the network parameters.

[0013] According to a specific implementation manner of the embodiment of the present disclosure, the preset retry condition is an upper limit on the number of retries or a waiting time.

[0014] In a second aspect, an embodiment of the present disclosure provides a data downloading device, comprising: Data sharding module: used to evenly shard the data to be downloaded according to the network conditions; Thread allocation module: used for receiving the sharding information of the data sharding module and allocating download threads according to the sharding information; Network monitoring and adjustment module: used to monitor network status in real time, dynamically adjust the size of the fragments according to network changes, and reallocate download threads based on the adjusted fragment size until all fragments are downloaded; File splicing and verification module: used to splice all fragments into the target file in order after all fragments are downloaded, complete the integrity verification of the target file, save the target file in the specified directory, and notify the user that the download is successful.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned data downloading method.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned data downloading method.

[0017] In summary, compared with the prior art, this embodiment has the following advantages: 1. Intelligent sharding algorithm: Dynamically adjust the size and number of each shard according to the file size and current network conditions. This intelligent sharding strategy can adapt to different network environments to the greatest extent possible, ensuring that the download process is both efficient and stable. By reducing network congestion and bandwidth waste, the algorithm significantly optimizes download performance, allowing users to complete the download of large files in the shortest possible time.

[0018] 2. Multi-threaded scheduling mechanism: It can reasonably allocate thread tasks to ensure that each thread can fully utilize system resources, while avoiding resource competition and conflicts between threads. By processing multiple shards in parallel, the multi-threaded scheduling mechanism greatly shortens the overall download time and improves download efficiency.

[0019] 3. Error recovery mechanism: During the download process, due to network fluctuations, server failures, etc., the download of a segment may sometimes fail. To this end, this embodiment provides an error recovery mechanism. The mechanism can automatically identify the segment that failed to download, and automatically re-add it to the download queue, and other idle threads will retry it. By continuously trying until the download is successful, the mechanism ensures the reliability and stability of the download process. In addition, the mechanism also has error recording and logging functions, which makes it easy for users to view and analyze the reasons for download failures, further improving the maintainability and user satisfaction of the download.

[0020] In summary, this embodiment improves download efficiency, resource utilization, and reliability, while improving user experience and addressing deficiencies of prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific implementation methods and drawings. Here, the exemplary implementation methods of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0022] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0023] Figure 1 A schematic diagram of a flow chart of a data downloading method provided by a first embodiment of the present invention; Figure 2 A schematic diagram of the data download system architecture provided by the first embodiment of the present invention; Figure 3 A schematic diagram of the structure of a data downloading device provided by a second embodiment of the present invention; Figure 4 An exemplary structural diagram of a device capable of implementing the method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0025] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0026] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0027] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0028] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0029] See also Figure 1 The first embodiment of the present invention provides a data downloading method, which can be executed by an electronic device, and in particular, by one or more processors in the electronic device, to implement the following steps: S101, evenly sharding the data to be downloaded according to the network conditions.

[0030] In this embodiment, the electronic device may be a device with computing capabilities such as a laptop computer, a desktop computer, a workstation, a server, a field programmable gate array (FPGA), etc.

[0031] Among them, network speed measurement technology (such as HTTP speed test, TCP throughput test, etc.) can be used to regularly evaluate the available bandwidth of the current network. The stability and quality of the network can be evaluated by sending probe packets and calculating the round-trip time (RTT) and packet loss rate. The file meta information can be directly read to obtain the total size of the data to be downloaded. Identify the file type (such as video, audio, document, etc.), because different types of data may have different optimal choices for the fragment size.

[0032] Calculate the appropriate shard size based on network bandwidth and file size. For example, shard size = min(maximum reasonable shard size, total file size / ideal number of shards), where the maximum reasonable shard size is determined based on bandwidth and expected download time, and the ideal number of shards is based on parallel download capabilities (e.g., the number of parallel download threads) and network stability.

[0033] According to the calculated shard size, the number of shards is automatically determined, while considering the maximum number of parallel connections supported by the network. It should be noted that in the embodiment of the present application, the data to be downloaded is evenly sharded, that is, the size of each shard is the same.

[0034] It should be noted that in the embodiment of the present application, the user specifies the file path or link to be downloaded through the client interface and triggers the download operation. The client software parses this information and prepares to send a download request.

[0035] The request contains the file identifier (such as URL, file ID, etc.) and is sent to the download server. At the same time, the client starts a listening mechanism to wait for the server's response and subsequent download data.

[0036] Before officially initiating data transfer for file download, the client first performs a network connection test. This step is to ensure that the network connection is active and to evaluate the performance of the current network environment.

[0037] By sending a series of small probe packets to the server (or a public speed test server), the client calculates the round-trip time (RTT), packet loss rate, and estimated available bandwidth. These parameters are critical for subsequent optimization of the download process. The detected network bandwidth parameters are fed back to the download control logic for subsequent download strategy adjustments.

[0038] Based on the results of the network connection detection, the client initializes a series of network parameters. These parameters include but are not limited to: calculating the optimal size of each shard based on the estimated bandwidth and total file size. The shard size should be able to fully utilize the network bandwidth while avoiding memory or disk IO bottlenecks caused by overly large shards.

[0039] The number of shards to be downloaded in parallel, i.e. the number of download threads, is determined based on system resources (CPU core count, memory) and network bandwidth. The number of threads should maximize the use of network resources while avoiding performance degradation caused by excessive thread competition.

[0040] Set the timeout and retry logic for network requests to cope with network fluctuations or server response delays.

[0041] The calculated parameters such as the shard size and number of threads are applied to the subsequent download process to guide operations such as shard generation, thread allocation, and data transmission.

[0042] Through the above steps, the client can prepare an optimized download strategy before downloading begins, so that efficient download performance can be maintained even when network conditions are changing.

[0043] S102, allocating download threads according to the number and size of the fragments.

[0044] In this embodiment, a download thread allocation scheme is formulated based on the number and size of shards and the results of system resource evaluation. The goal is to enable each thread to maintain a high working efficiency while avoiding performance degradation caused by excessive thread competition. Each shard or a group of shards can be assigned to a download thread. It can be static allocation (each thread processes a fixed group of shards) or dynamic allocation (adjusted in real time according to the download progress and thread performance). According to the allocation scheme, a corresponding number of download threads are started. Each thread receives the shard information it is responsible for. All download threads start working concurrently, and each thread sends a download request to the server and receives data according to its assigned shard.

[0045] S103, monitoring the network status in real time, dynamically adjusting the size of the slices when the network status changes, and reallocating download threads according to the adjusted slice sizes to complete the data download.

[0046] Specifically, during the download process, the network status is continuously monitored, including key indicators such as bandwidth, latency, and packet loss rate. Real-time network data is collected and compared with historical data or preset thresholds to determine whether the network status has changed. If changes in network status are detected, such as a significant drop in bandwidth, the system will trigger a change detection mechanism. The impact of network changes on the current download task is evaluated, including the expected drop in download speed, increase in remaining download time, etc.

[0047] Dynamically adjust the size of the shards according to changes in network bandwidth. If the bandwidth drops, the shard size can be appropriately reduced to reduce download interruptions and retries caused by network fluctuations.

[0048] Re-evaluate and re-assign download threads based on the new shard size and the number of shards remaining to be downloaded. You may need to reduce the number of parallel threads to avoid too many threads competing for limited network resources.

[0049] Apply the adjusted shard size and thread allocation scheme to the ongoing download task. This may involve pausing the current thread, reallocating shards, starting a new thread, etc.

[0050] After dynamic adjustment, continue to monitor network conditions and further adjust the download strategy as needed to ensure the stability and efficiency of the download process.

[0051] During the download of each segment, the system monitors the download status and progress in real time. If an error is detected during the download of a segment (such as network timeout, connection interruption, file corruption, etc.), the system will immediately identify and record the error. For example, the system will log the error details (including error type, occurrence time, segment identifier, etc.) for subsequent analysis and tracking. The status of the segment is updated to "download failed" and it is removed from the download queue of the current thread.

[0052] Based on the preset retry conditions or user configuration, the system can set the number of times each shard can be retried or the waiting time between each retry to avoid network congestion caused by frequent retries in a short period of time.

[0053] Reassign failed download segments to one or more other idle threads for download. This can be done by randomly selecting threads or intelligently assigning them based on the current load and performance of the threads.

[0054] The thread that is assigned the failed segment will start the download process and try to continue or restart the download from the point where it failed last time.

[0055] If the shard fails again during the retry process, the system will repeat the above error handling and retry steps until the preset retry conditions (for example, the upper limit of the number of retries or the waiting time) are met or the download is successful.

[0056] Of course, if the segment still cannot be downloaded successfully after multiple retries, the system can take further measures, such as pausing the entire download task, notifying the user to handle it manually, or automatically skipping the segment and continuing to download other parts.

[0057] S104, splicing the downloaded segments into the target file in order, completing the integrity verification of the target file, storing the target file in a specified directory, and notifying the user that the download is successful.

[0058] Specifically, before reassembling the file, the system first verifies the integrity and correctness of each downloaded segment. This usually includes verification of checksums (such as MD5, SHA-256, etc.) to ensure that the segments have not been damaged during transmission. Based on the starting position or unique identifier of the segment, the system sorts the segments to ensure that they can be reassembled in the order of the original file. The downloaded segments are spliced ​​into the target file in order.

[0059] If the target file already exists, the system can choose to clear the file or verify whether its current content matches the download task. If the target file does not exist, a new file is created. The system reads the content of each shard in sequence and writes it directly to the corresponding position of the target file. The completion of the splicing of all shards marks the end of the data download. When the file is reassembled and verified, the system updates the status of the download task to "Completed".

[0060] After the file is reassembled, the system performs a final integrity verification step. This typically involves comparing the file's checksum (such as MD5, SHA-256, etc.) with the expected checksum to ensure that the file has not been tampered with or damaged during transmission and reassembly.

[0061] If the verification fails, that is, there is a problem with the file integrity, the system will record the error and perform corresponding error handling actions according to the configuration (such as trying to download again, marking the file as damaged, notifying the user, etc.).

[0062] The system checks whether the specified target directory exists. If it does not exist, it will be created automatically, depending on the configuration. Verify that the system or user has permission to write files in the target directory.

[0063] If the verification passes and the target directory is ready, the system moves the final downloaded file from the temporary storage location (such as the cache directory) to the specified target directory and notifies the user that the download is successful.

[0064] like Figure 2 As shown, the client initiates the download service, calculates the initial shard size based on the obtained network bandwidth, and first uses parallel multi-threading to download data according to the number of initialized shards. Monitor the network in real time, dynamically adjust the shard size according to the network status, and reallocate threads and download data based on the adjusted shard size. If the downloaded shard can be written to the file normally, it will be stored in the server. If it cannot be written to the file normally, reset the shard information (reallocate the download thread to the wrong shard for download). Figure 2 The preset retry condition in is the upper limit of the number of retries, and the upper limit of the number of retries is 3 times, but the embodiment of the present application does not impose any specific limitation on the number of retries.

[0065] In this embodiment, the split data blocks are input into the model for training. It should be noted here that the position marks of the data blocks in the model training do not need to be reset.

[0066] In summary, compared with the prior art, this embodiment has the following advantages: 1) Segment initialization and dynamic adjustment: By dividing a large file into multiple evenly spaced segments, it is not only easier to manage, but also allows the size of each segment to be dynamically adjusted according to real-time network conditions. This flexibility ensures that the download process can fully utilize current network resources, effectively avoiding download bottlenecks caused by network congestion or insufficient bandwidth, thereby significantly optimizing download efficiency.

[0067] 2) Efficient thread allocation and parallel processing: Utilize thread pool technology to create multiple download threads and allocate these threads to different shards for parallel downloading. This parallel processing mechanism greatly shortens the overall download time, increases the download speed, reduces user waiting time, and improves the user experience.

[0068] 3) Stable data transmission: During the download process, each thread independently requests and receives the shard data it is responsible for from the server, ensuring efficient and stable data transmission. Even when network conditions fluctuate, reasonable thread management and sharding strategies can maintain a relatively stable download rate, reducing download interruptions or failures caused by network problems.

[0069] 4) Error handling and recovery mechanism: It provides error handling logic that can detect and respond to various abnormal situations during the download process. Once a segment fails to download, the system will immediately record the error information and reallocate the segment to other idle threads for retry until the download is successful. This automatic retry and error recovery mechanism greatly enhances the reliability of the download and reduces the risk of interrupting the entire download task due to a single download failure.

[0070] In summary, this embodiment provides an efficient and reliable data multi-threaded segmented download method through a data segmentation method, a thread management mechanism, and an error handling and recovery mechanism. This method not only significantly improves the download speed and stability, but also reduces the uncertainty and risk of users during use, bringing users a smoother and more convenient download experience.

[0071] A second embodiment of the present invention provides a data downloading device (such as Figure 3 shown), which includes: Data slicing module 310: used to evenly slice the data to be downloaded according to the network conditions, ensuring that each slice has similar data volume and processing difficulty in the initial state.

[0072] Thread allocation module 320: used to receive the fragmentation information from the data fragmentation module, including the number of fragments and the size of each fragment, and allocate download threads accordingly. Its goal is to optimize resource utilization and improve download efficiency.

[0073] Network monitoring and adjustment module 330: This module continuously monitors the network status in real time. Once the network status changes (such as bandwidth fluctuations, increased network latency, etc.), it dynamically adjusts the size of the segments according to the new network conditions. Subsequently, the module reallocates download threads based on the adjusted segment size to ensure that the download process can continue to proceed efficiently until all segments are downloaded.

[0074] File splicing and verification module 340: After all the segments are downloaded, this module is responsible for splicing all the segments into a complete file in the order of the original data. After that, the module performs the integrity verification step to ensure that the spliced ​​file is completely consistent with the original data. After the verification is passed, the module saves the target file in the specified directory and notifies the user that the download is successful.

[0075] A third embodiment of the present invention further provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data downloading method of any of the aforementioned embodiments.

[0076] The fourth embodiment of the present invention further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the data downloading method described in any of the aforementioned embodiments.

[0077] The fifth embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the data download method of any of the aforementioned embodiments.

[0078] The sixth embodiment of the present invention further provides a computer program, which includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute the data downloading method of any of the above embodiments.

[0079] Figure 4 A schematic diagram of a method or device 1000 that can implement an embodiment of the present invention is shown, which may include more or fewer devices than shown in the figure in some embodiments. In some embodiments, it can be implemented using a single or multiple devices. In some embodiments, it can be implemented using cloud or distributed devices.

[0080] like Figure 4As shown, the device 1000 includes a processor 1001, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) 1002 or the program and / or data loaded from the storage part 1008 to the random access memory (RAM) 1003. Processor 1001 can be a multi-core processor or include multiple processors. In some embodiments, processor 1001 can include a general main processor and one or more special coprocessors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM 1003, various programs and data required for the operation of device 1000 are also stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0081] The processor and the memory are used together to execute the program stored in the memory. When the program is executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0082] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, a touch screen, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed. Figure 4 Only some components are schematically shown, which does not mean that the device 1000 only includes Figure 4 Components shown.

[0083] The systems, devices, modules or units described in the above embodiments may be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, a smart phone, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device, a smart TV, an Internet of Things system, a smart home, an industrial computer, a server or a combination thereof.

[0084] Although not shown, in an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the data downloading method described in Example 1 is implemented.

[0085] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be used to store information by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0086] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which implements the data downloading method described in Embodiment 1 when executed by a processor.

[0087] The methods, programs, systems, devices, etc. of the embodiments of the present invention may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0088] Those skilled in the art should understand that the embodiments of the present specification may be provided as methods, systems or computer program products. Therefore, those skilled in the art may imagine that the functional modules / units or controllers and related method steps described in the above embodiments may be implemented in software, hardware or a combination of software / hardware.

[0089] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0090] In this article, multiple embodiments of the present invention are described, but for the sake of simplicity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the embodiments may be omitted. In this article, "one embodiment", "some embodiments", "example", "specific example", or "some examples" are intended to be applicable to at least one embodiment or example of the present invention, but not all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. In the absence of contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0091] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best modes for implementing the present systems and methods. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A data downloading method, characterized in that: include: Evenly shard the data to be downloaded according to the network conditions; Allocate download threads based on the number and size of shards; The network status is monitored in real time, and when the network status changes, the size of the fragments is dynamically adjusted and the download threads are reallocated according to the adjusted fragment sizes to complete the download of all fragments; The downloaded segments are sequentially spliced ​​into the target file, the integrity verification of the target file is completed, the target file is stored in the specified directory, and the user is notified of the successful download.

2. The data downloading method according to claim 1, characterized in that: The data to be downloaded is evenly sharded according to the network conditions, including: The data is evenly sharded according to the maximum reasonable shard size, network bandwidth, and parallel download capability.

3. The data downloading method according to claim 1, characterized in that: If a segment fails to be downloaded, the failed segment is recorded, and a download thread is reallocated to the failed segment until the failed segment is successfully downloaded or a preset retry condition is met.

4. The data downloading method according to claim 1, characterized in that: The client sends a download request, connects to the server and prepares a path or link of the file to be downloaded.

5. The data downloading method according to claim 1, characterized in that: Before downloading data, the network connection needs to be tested and the bandwidth parameters of the current network need to be measured.

6. The data downloading method according to claim 5, characterized in that: Based on the detection result of the network connection, the network parameters are initialized, and the size of the slice and the allocation of the download thread are determined based on the network parameters.

7. The data downloading method according to claim 3, characterized in that: The preset retry condition is an upper limit on the number of retries or a waiting time.

8. A data downloading device, characterized in that: include: Data sharding module: used to evenly shard the data to be downloaded according to the network conditions; Thread allocation module: used for receiving the sharding information of the data sharding module and allocating download threads according to the sharding information; Network monitoring and adjustment module: used to monitor network status in real time, dynamically adjust the size of the fragments according to network changes, and reallocate download threads based on the adjusted fragment size until all fragments are downloaded; File splicing and verification module: used to splice all fragments into the target file in order after all fragments are downloaded, complete the integrity verification of the target file, save the target file in the specified directory, and notify the user that the download is successful.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data downloading method described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the data downloading method described in any one of claims 1 to 7.

Images