File fragment uploading method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a file fragment uploading method and device, electronic equipment, computer program and storage medium; the embodiments of the present application obtain the file to be uploaded and file information, and the file information at least includes file size;According to the initial network speed of current network, the file to be uploaded is cut, and a plurality of initial file fragments are obtained;Initial file fragments are sequentially uploaded to the server, and the upload time consumption and upload rate corresponding to the initial file fragments are recorded;According to the upload time consumption and upload rate of the uploaded initial file fragments, the size of the next initial file fragment is adjusted, and the target file fragment is obtained;The target file fragment is uploaded to the server. Therefore, by using the mode of "dynamic fragment uploading+network adaptation", in the file uploading process, the real-time network state is determined according to the uploading situation of the uploaded fragment, and the fragment size is dynamically adjusted accordingly, so as to improve the file uploading success rate and user experience, and be flexibly applied to various large file uploading scenes.

Description

Technical Field

[0001] This application relates to the field of computers, specifically to a method, apparatus, electronic device, and storage medium for uploading files in segments. Background Technology

[0002] With the development of computer network technology, more and more applications support users uploading their own content for playback, sharing, and other purposes. For example, many audio and video applications support users uploading their own videos and audio files for playback. For uploading large files such as audio and video files, a segmented parallel upload method is typically used, which involves dividing the file into multiple data blocks and uploading them separately.

[0003] In related technologies, files are typically fragmented into chunks of fixed size based on human experience. However, because the chunk size is fixed, this method is difficult to achieve good results in different file upload scenarios, and chunks can easily become too large or too small. Chunks that are too large result in excessively long upload times for each chunk, increasing the probability of chunk upload failures; chunks that are too small increase the number of requests for the entire file upload, increasing network request overhead and hindering subsequent file archiving and merging.

[0004] Therefore, there is an urgent need for a file chunking upload method that can be flexibly applied to different file upload scenarios. Summary of the Invention

[0005] This application provides a file chunking upload method, apparatus, electronic device, and storage medium, which can be flexibly applied to different large file upload scenarios, improving file upload success rate and user experience.

[0006] This application provides a method for uploading files in chunks, including:

[0007] Obtain the file to be uploaded and its information, wherein the file information includes at least the file size;

[0008] Based on the initial network speed, the file to be uploaded is split into multiple initial file fragments.

[0009] The initial file fragments are uploaded to the server sequentially, and the upload time and upload rate corresponding to each initial file fragment are recorded.

[0010] Based on the upload time and upload rate of the previously uploaded initial file segments, adjust the size of the next initial file segment to obtain the target file segment;

[0011] Upload the target file fragments to the server.

[0012] This application embodiment also provides a file fragment uploading device, including:

[0013] The file parsing unit is used to obtain the file to be uploaded and file information, wherein the file information includes at least the file size;

[0014] The file fragmentation unit is used to split the file to be uploaded into multiple initial file fragments based on the initial network speed of the current network.

[0015] An upload statistics unit is used to sequentially upload the initial file fragments to the server and record the upload time and upload rate corresponding to the initial file fragments.

[0016] The segment adjustment unit is used to adjust the size of the next initial file segment based on the upload time and upload rate of the already uploaded initial file segments, so as to obtain the target file segment.

[0017] The fragmented upload unit is used to upload the target file fragments to the server.

[0018] In some embodiments, the segmentation adjustment unit includes:

[0019] The sliding window determination sub-unit is used to determine the initial width of the sliding window based on the user's historical network status data;

[0020] A sliding window processing subunit is used to process the uploaded initial file fragments based on a sliding window with an initial width, and to determine the average time consumption and average rate of multiple initial file fragments within the sliding window.

[0021] The fragment size determination subunit is used to determine the target fragment size based on the average time consumption and the average rate.

[0022] The fragment size adjustment subunit is used to adjust the next initial file fragment to the target fragment size to obtain the target file fragment.

[0023] In some embodiments, the apparatus further includes:

[0024] An upload change statistics unit is used to determine the time variation coefficient based on the upload time of the K most recently uploaded initial file fragments;

[0025] A sliding window adjustment unit is used to adjust the initial width of the sliding window based on the time variation coefficient to obtain a target width;

[0026] A sliding window processing unit is used to process the uploaded initial file fragments based on a sliding window with a target width.

[0027] In some embodiments, a file fragmentation unit includes:

[0028] The network level determination subunit is used to determine the network level to which the initial network speed belongs based on a preset network level classification rule;

[0029] The initial fragmentation subunit is used to split the file to be uploaded based on the preset fragmentation size corresponding to the network level, so as to obtain multiple initial file fragments.

[0030] In some embodiments, the apparatus further includes:

[0031] The test request unit is used to send a test file retrieval request to the server and record the request initiation time;

[0032] The test response unit is used to receive the test file returned by the server and record the request end time and file size;

[0033] The initial network speed determination unit is used to determine the initial network speed based on the request initiation time, the request end time, and the file size.

[0034] In some embodiments, the apparatus further includes:

[0035] The request acquisition unit is used to acquire network requests and determine whether the network request is a critical resource request;

[0036] The fragmentation pause unit is used to pause the adjustment of the next initial file fragment size if the network request is a critical resource request.

[0037] The fragment recovery unit is used to restore the adjustment of the next initial file fragment size when the network request response is detected to be completed.

[0038] In some embodiments, the apparatus further includes:

[0039] The upload statistics unit is used to determine whether the upload is complete based on the percentage of uploaded segments.

[0040] The fragment merge request unit is used to send an archive merge request to the server after each target file fragment has been uploaded, so that the server can merge the target file fragments to obtain the target merged file.

[0041] This application also provides an electronic device, including a memory storing multiple instructions; the processor loads instructions from the memory to execute steps in any of the file fragment upload methods provided in this application.

[0042] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the file fragment upload methods provided in this application.

[0043] This application embodiment can obtain the file to be uploaded and file information, the file information including at least the file size; divide the file to be uploaded into multiple initial file fragments according to the initial network speed; sequentially upload the initial file fragments to the server and record the upload time and upload rate corresponding to the initial file fragments; adjust the size of the next initial file fragment according to the upload time and upload rate of the uploaded initial file fragments to obtain the target file fragment; and upload the target file fragment to the server.

[0044] This application adopts a "dynamic segmented upload + network adaptive" approach. During the file upload process, the real-time network status is determined based on the upload status of the initial file segments that have been uploaded, and the segment size is dynamically adjusted accordingly to improve the file upload success rate and user experience. This approach can be flexibly applied to different large file upload scenarios. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the file fragment upload method provided in an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of the process for adjusting the fragment size provided in an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the file fragment uploading device provided in the embodiments of this application;

[0049] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] This application provides a method, apparatus, electronic device, and storage medium for uploading files in segments.

[0052] Specifically, the file fragment uploading device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0053] In some embodiments, the file chunking upload device can also be integrated into multiple electronic devices. For example, the file chunking upload device can be integrated into multiple servers, and multiple servers can implement the file chunking upload method of this application.

[0054] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0055] In related technologies, files are typically divided into chunks of fixed size and uploaded based on human experience. However, during the upload process, due to the different network environments of each user and the often unstable network conditions, file uploads may experience lag, failures, or other issues, affecting the user experience.

[0056] In view of this, this application provides a file chunking upload method. By adopting the "dynamic chunking upload + network adaptive" approach, the real-time network status is determined based on the upload status of the initial file chunks during the file upload process, and the chunk size is dynamically adjusted accordingly to improve the file upload success rate and user experience. This method can be flexibly applied to different large file upload scenarios.

[0057] In this embodiment, a file chunking upload method is provided, such as... Figure 1 As shown, the specific process of uploading the file in chunks can be as follows:

[0058] S110. Obtain the file to be uploaded and its information. The file information shall include at least the file size.

[0059] In this embodiment, the file to be uploaded refers to: a collection of data selected and submitted by the user through a human-computer interaction interface, awaiting transmission to a remote server over the network. That is, a complete data unit stored locally on the user's device. For example, a video file named vacation.mp4 selected by user Zhang San in a cloud storage application.

[0060] File information is a collection of metadata describing the basic attributes of the file to be uploaded. File information includes at least the file size; for example, a video file may be 250MB in size.

[0061] In some embodiments, file information also includes the file type, file name, file unique identifier, last modified time, etc. The file type is the file format and nature identified by the file extension or MMS extension type, which determines how the system processes the file (e.g., which application opens it, how it is parsed). For example, file types based on extensions include: .pdf, .mp4, .xlsx, .jpg; those based on MMS extensions include: application / pdf, video / mp4, etc. The file name is a user-defined string used to identify and distinguish different files in the file system, typically including the filename and extension. For example: 2024 travel trailer.mp4. The file unique identifier is a string generated by a specific algorithm (e.g., a hash algorithm) that is unique to the file content. Even if file names are the same, if the content is different, their unique identifiers will be different. The file unique identifier can be used for: instant uploads (the server compares the file hash value to determine if the same file already exists; if so, there is no need to upload it again); integrity checks (verifying whether the file has been corrupted or tampered with during upload or download); and deduplication (saving space by storing only one copy of the file with the same content in the storage system). For example, the unique identifier of a file formed using an MD5 hash value is: 5d41402abc4b2a76b9719d911017c592 (corresponding to the string "hello"). The last modified time refers to the timestamp recorded by the operating system when the file content was last modified and saved. This information is used to determine the old and new versions of a file, synchronize file status, etc. For example, the last modified time of a file is: 2024-05-27 14:30:25.

[0062] In some embodiments, after a user selects a file to upload on the client, the client system parses the file to obtain file information such as file type, file name, file unique identifier, and file size, and caches it to prepare data for subsequent file uploads by the client.

[0063] For example, when a user uploads a file to Baidu Cloud through a browser (client), assuming the user clicks the "Select File" button on the webpage and selects a file named MyStyle.mp4 from their computer, the browser will capture this action and create a File object to represent the selected file. This File object is a built-in interface of the browser, containing all the basic information about the file. Subsequently, the webpage's JavaScript code (i.e., client-side logic) can access this File object and extract the file information from it.

[0064] Considering that users may not click the "upload" button immediately after selecting a file, the client will cache the file first to ensure that the client can obtain the original file data no matter when the user clicks upload, thus making sufficient data preparation for subsequent file chunk uploads.

[0065] S120. Based on the initial network speed, the file to be uploaded is split into multiple initial file fragments.

[0066] In this embodiment, the current network refers to the network environment and its state when the user device initiates an upload request. This is a dynamically changing context, including network type (such as Wi-Fi, 5G), signal strength, bandwidth capacity, and congestion level. For example, user Zhang San's mobile phone is currently connected to an 802.11ac wireless network named Home-WiFi, with a full signal strength.

[0067] Initial network speed refers to the initial value representing the current network's uplink bandwidth capacity obtained before the upload task begins. This value is used to guide the initial file splitting. In related technologies, the browser's built-in network information API, navigator.connection, is typically used to estimate the current user's network status. However, the implementation of this built-in API varies across different browsers, and older browser versions may have incomplete support for new APIs, limiting its usage scenarios.

[0068] To address this, this embodiment detects the initial network speed using the following method:

[0069] Send a test file retrieval request to the server and record the request initiation time; receive the test file returned by the server and record the request end time and file size; determine the initial network speed based on the request initiation time, request end time, and file size.

[0070] In this context, the server refers to a remote computer system or cluster that provides file upload, reception, storage, and processing services to user devices on a network. It is the receiver of upload requests and the data processor. For example, the backend server cluster of a cloud storage service provides an API interface (such as https: / / pan.baidu.com / upload) to receive user requests and uploaded files, etc.

[0071] The client is informed in advance of the URL (Uniform Resource Locator) of a test file provided by the server specifically for speed testing. This test file is typically a small, precisely calibrated static file (such as a 1KB text file or a small image).

[0072] During the testing process, the client initiates an HTTP GET request for the test file, appending a random parameter (e.g., ?t=1716800000000) to the request URL for accurate timing. Just before the request is sent to the network, the client records the request initiation time using a high-precision time API (e.g., performance.now()). Upon receiving the test file retrieval request, the server responds by sending the test file to the client.

[0073] The client receives the test file data returned by the server. When the last byte of the test file is received, the client again uses a high-precision time API to record the end time of the request. Then, the client can obtain the precise size of the test file, in bytes, from the Content-Length header of the HTTP response returned by the server.

[0074] Next, calculate the difference between the request initiation time and the request completion time to obtain the total time spent transferring the test file. Then, use the following formula to calculate the initial network speed:

[0075]

[0076] in, This indicates the file size of the test file, in bytes. Indicates the time the request was initiated, in milliseconds (ms). This indicates the request end time, in milliseconds (ms). This indicates the initial network speed, measured in bits per second (bps).

[0077] For example, suppose the request was initiated at time The request end time is 1716800000000.5000ms. The value is 1716800000001.2000ms. The size of the test file is 1024 bytes. Therefore, the initial network speed calculated using the above formula is 11.7 × 10^6 bps, which is 11.7 Mbps in megabits per second (Mbps).

[0078] In this embodiment, after detecting the initial network speed, the file to be uploaded can be segmented according to the initial network speed to obtain multiple initial file fragments. An initial file fragment refers to a sequence of data blocks obtained by logically dividing the file to be uploaded into fragments of size calculated based on the initial network speed. These fragments are the smallest transmission unit for file upload. For example, based on an initial network speed of 4.8 Mbps, if the final initial fragment size is determined to be 1 MB, then a 250 MB file to be uploaded will be divided into 250 initial file fragments, numbered sequentially from Chunk-001 to Chunk-250.

[0079] In some embodiments, the file to be uploaded is split into multiple initial file fragments based on the initial network speed, including:

[0080] Based on the preset network level classification rules, the network level to which the initial network speed belongs is determined; the file to be uploaded is split into multiple initial file fragments based on the preset fragment size corresponding to the network level.

[0081] Specifically, the network tier classification rules are as follows: Slow tier: if the initial network speed is ≤3Mbps, the default segment size is 100MB; Medium tier: if the initial network speed is >3Mbps and <7MB / s, the default segment size is 200MB; High tier: if the initial network speed is ≥7Mbps, the default segment size is 500MB.

[0082] Assuming user A's initial network speed is 45Mbps, their network level is considered high-speed. Therefore, the file to be uploaded is divided into several initial file chunks of 500MB each. Thus, based on the initial network speed level, a safe and reasonable initial chunk size can be quickly determined, ensuring that the file upload process can start quickly and run securely under any circumstances.

[0083] S130. Upload the initial file fragments to the server sequentially, and record the upload time and upload rate of the initial file fragments.

[0084] In this embodiment, upload time refers to the complete time interval from initiating a network request to upload an initial file fragment to receiving a successful response from the server. For example, uploading the initial file fragment Chunk-123 took a total of 1250 ms from sending the upload request to receiving the response.

[0085] Upload speed refers to the amount of data successfully uploaded per unit of time and is a key indicator for measuring the actual utilization efficiency of network bandwidth. The calculation formula is: Upload speed = Chunk size / Upload time. For example, if the size of Chunk-123 is 1MB (1,048,576 bytes) and the upload time is 1250ms (1.25s), then its upload speed is 1,048,576 bytes / 1.25 seconds = 838,860.8 bytes / second, approximately 6.7Mbps.

[0086] After the initial file chunks begin uploading, the upload time and upload rate of the initial file chunks are recorded to understand the real-time changes in the user's network, providing data for dynamic adjustments to subsequent chunk uploads.

[0087] S140. Based on the upload time and upload rate of the already uploaded initial file segments, adjust the size of the next initial file segment to obtain the target file segment.

[0088] In some embodiments, network conditions can change drastically in a short period (such as momentary network congestion or signal interference), making it difficult to fully utilize bandwidth under fluctuating conditions. Furthermore, when a user device switches from one network environment to another, it may take time to accurately reflect the new network conditions. During this transition period, the network information obtained may be inaccurate. Therefore, simply using a fixed-fragmentation upload method will affect file upload efficiency because it cannot adapt well to network changes.

[0089] In this embodiment, the size of subsequent initial file fragments is dynamically adjusted based on the upload time and upload rate of the already uploaded initial file fragments to adapt to the current network conditions. For details, please refer to... Figure 2 The process mainly includes:

[0090] S140-1: Determine the initial width of the sliding window based on the user's historical network status data.

[0091] Historical network status data refers to the set of performance metrics recorded by user devices over a period of time when transmitting data through the network. This data is used to quantify the stability and performance level of the user's network environment. This data typically includes, but is not limited to: file fragment upload time, actual upload rate, round-trip time (RTT) for network requests, packet loss rate, etc.

[0092] A sliding window is a data analysis technique used to process real-time data streams. It defines a fixed or variable-length observation interval that "slides" forward as new data arrives, focusing only on the latest data within the interval and discarding older data outside the interval. In this embodiment, it specifically means a queue (such as an array or linked list) for storing the time and rate data of the most recent N file chunk uploads. N is the width of the window. The window is updated as each new chunk is uploaded, adding new data and removing the oldest data.

[0093] The initial width of the sliding window refers to its initial length when dynamic adjustments begin, i.e., the initial value of N. The initial width is not a fixed value, but rather is set based on the analysis results of the user's historical network status data.

[0094] In some embodiments, if historical network state data indicates that the user network fluctuates drastically and is unstable, a smaller initial width (e.g., N=3) is set to enable the system to respond more sensitively to the latest changes. If historical network state data indicates that the user network is stable over a long period and changes gradually, a larger initial width (e.g., N=10) is set to enable the system to obtain a more robust and jitter-resistant average value.

[0095] S140-2: Process the uploaded initial file fragments based on a sliding window with an initial width, and determine the average time and average rate of multiple initial file fragments within the sliding window.

[0096] In this embodiment, after the initial file fragment upload begins, the upload time and upload rate of each initial file fragment are continuously monitored, and a window with a capacity of N (i.e., the initial width) is maintained. Whenever a new initial file fragment is uploaded, its actual upload time is recorded. and upload speed Add to the window. If the window is full (length has reached N), remove the oldest data.

[0097] Then, calculate the average upload time and upload rate of the N initial file fragments within the window to obtain the average upload time and average upload rate.

[0098] As is easily understood, the average latency reflects the recent network response efficiency, while the average rate reflects the recent network throughput. Using the sliding window mean instead of a single data point can effectively filter out occasional, transient network fluctuations, resulting in more reliable trend values.

[0099] S140-3: Determine the target fragment size based on the average time consumption and average rate.

[0100] The target chunk size refers to the new, optimized chunk size obtained by dynamically calculating and adjusting based on the upload performance (upload time, upload speed) of the initial uploaded file chunks. This chunk size aims to better adapt to real-time changes in network conditions, thereby improving overall upload efficiency and reliability.

[0101] In some embodiments, the target fragment size can be calculated using the following formula:

[0102]

[0103] in, Indicates the target fragment size. This represents the average time taken. This represents the average rate.

[0104] Thus, the measured value of network performance (rate) is directly converted into control commands (fragment size) to achieve adaptive adjustment of network fragment size.

[0105] S140-4: Adjust the next initial file fragment to the target fragment size to obtain the target file fragment.

[0106] For example, assuming the initial width of the sliding window is 5, when adjusting the size of the 7th initial file segment to be uploaded, the target segment size is calculated based on the average time and average speed of the second, third, fourth, fifth, and sixth initial file segments that have already been uploaded. If the average time of these five most recent initial file segments is 0.5s and the average speed is 2MB / s, then the target segment size is 1MB. Then, the size of the 7th initial file segment to be uploaded is adjusted from its preset segment size (e.g., 2MB) to 1MB.

[0107] As is easily understood, for the first and second initial file segments to be uploaded, they are uploaded directly at their preset segment size. For the third initial file segment to be uploaded, its size is adjusted based on the average time and average speed of the first two uploaded initial file segments. For the fourth initial file segment to be uploaded, its size is adjusted based on the average time and average speed of the first three uploaded initial file segments. For the fifth initial file segment to be uploaded, its size is adjusted based on the average time and average speed of the first four uploaded initial file segments. For the sixth initial file segment to be uploaded, its size is adjusted based on the average time and average speed of the first five uploaded initial file segments.

[0108] Furthermore, during the initial file chunk upload process, the width of the sliding window can be optimized and adjusted based on the recent initial file chunk upload data. Specifically, this process mainly includes:

[0109] S140-2-1: Determine the coefficient of variation of time based on the upload time of the K most recently uploaded initial file fragments.

[0110] The coefficient of variation (CPV) is the ratio of the standard deviation to the average upload time of the K most recently uploaded initial file fragments. A large CPV indicates significant fluctuations in the upload times of the K most recently uploaded initial file fragments, suggesting network instability; a small CPV indicates minimal fluctuations in the upload times of the K most recently uploaded initial file fragments, suggesting a more stable network. K can be set according to actual needs, for example, it can be set to 5.

[0111] S140-2-2: Based on the time variation coefficient, adjust the initial width of the sliding window to obtain the target width.

[0112] S140-2-3: Process the uploaded initial file fragments based on a sliding window with a target width.

[0113] In some embodiments, the initial width of the sliding window can be adjusted by setting a range of variation. If the current time coefficient of variation is less than the lower limit of the range, it indicates that the network is very stable. Therefore, the size N of the sliding window is set to a larger value (e.g., N=10) to allow the large window to smooth out minor fluctuations and obtain a more stable and reliable average estimate. This allows for more confident use of larger fragments and improved upload efficiency. If the current time coefficient of variation is greater than the upper limit of the range, it indicates that the network is very unstable. Therefore, the size N of the sliding window is set to a smaller value (e.g., N=3) to allow the small window to quickly reflect the latest changes in the network and rapidly reduce the fragment size.

[0114] For example, assuming the variation range is [0.2, 0.5], the initial width of the sliding window is N=5, and the times (in seconds) of the last 5 fragments are monitored as [0.5, 0.6, 0.5, 0.55, 0.5], then the coefficient of variation can be calculated to be 0.085. Since the coefficient of variation of 0.085 is less than the lower limit of the variation range of 0.2, the sliding window size can be increased to N=10 to obtain a longer-term and more stable average value to guide subsequent fragment size adjustments.

[0115] The time taken for the next few shards starts to increase: [1.2, 2.0, 1.8] (due to network congestion). The time taken for the most recent 10 shards is now: [0.5, 0.6, 0.5, 0.55, 0.5, 1.2, 2.0, 1.8, 1.7, 1.6], with a calculated coefficient of variation of 0.6. Since the coefficient of variation of 0.6 is greater than the upper limit of the variation range of 0.5, the sliding window size can be reduced to N=3, using only the most recent 3 shards ([1.2, 2.0, 1.8]) to calculate the average time. This means the algorithm can immediately detect network degradation and quickly make a decision to reduce the number of shards, rather than being "averaged" out by previously stable historical data.

[0116] After reducing the number of fragments, the network performance improved and the processing time decreased. The most recent fragment sequence was [1, 0.65, 0.6], with a coefficient of variation of 0.29. Since the coefficient of variation of 0.29 is within the range of [0.2, 0.5], the sliding window size can be increased back to N=5.

[0117] Thus, through this mechanism of dynamically adjusting the sliding window size, it no longer mechanically calculates the average value of a fixed window, but has a "smart switch" that can take the most appropriate strategy under different network conditions, thereby achieving network self-adaptation.

[0118] S150. Upload the target file fragments to the server.

[0119] In this embodiment, when uploading the next initial file segment, the size of the initial file segment is dynamically adjusted according to the real-time network conditions before uploading. Thus, through this combination of "static upload startup + dynamic real-time optimization" strategy, the startup speed, stability and final efficiency of file segment upload are taken into account.

[0120] In some embodiments, the above method further includes:

[0121] Obtain the network request and determine if it is a critical resource request; if it is, pause the adjustment of the next initial file fragment size; resume the adjustment of the next initial file fragment size once the network request response is detected.

[0122] A network request refers to a complete data communication process initiated by a client (such as a web browser, mobile application, or desktop program) to a server via a computer network in order to obtain resources, submit data, or interact with a remote service. This process follows a specific application layer protocol (such as HTTP / HTTPS) and includes the entire lifecycle of request initiation, transmission, processing, and response return.

[0123] Critical resource requests refer to a special type of network request that has an immediate and high-priority impact on user experience or core application functionality. Delays or failures in these requests can directly lead to user-perceived application lag, unresponsiveness, or functional malfunctions. Therefore, it is necessary to identify these requests and prioritize their allocation of network resources to ensure rapid processing.

[0124] Key resource requests include, but are not limited to: API requests triggered by user interactions, such as when a user clicks the "save" button or sends a chat message; message sending and receiving requests from real-time communication applications, such as real-time stock price pushes and synchronized operations in multiplayer games; and requests for core page rendering resources.

[0125] In practice, the client (such as a browser or upload tool) listens for all HTTP / HTTPS requests through a network request interception mechanism (such as an HTTP interceptor or the PerformanceObserver API) and determines whether the request is a critical resource request. If so, a global flag is set (e.g., isCriticalRequestActive=true), and any dynamic adjustment logic for fragment size is stopped. That is, subsequent file fragment uploads maintain the current fragment size (e.g., continue using 1MB fragments), proactively reducing bandwidth usage for file uploads (e.g., limiting to a maximum of 50% bandwidth) to ensure that critical resource requests receive sufficient network resources. Then, by listening to the onload or onend events of critical resource requests, it confirms that the transmission has been completed and clears the global flag (isCriticalRequestActive=false) to re-enable the dynamic fragment adjustment algorithm and calculate the next fragment size based on the latest network status.

[0126] In some embodiments, determining whether a request is a critical resource request can be done in the following ways: 1. URL (Uniform Resource Locator) matching rules, for example, / api / chat / send is a critical resource request for real-time chat, and / api / user / action is a critical resource request for user interaction. 2. Request header markings, for example, if a request carries a custom high-priority tag (X-Priority:high), it is also determined to be a critical resource request and processed with priority.

[0127] This embodiment employs dynamic priority scheduling technology. During the file chunking upload process, when a critical resource request is detected, the chunking adjustment is actively paused, and the bandwidth usage of file upload is forcibly reduced (e.g., from 10Mbps to 5Mbps) to prioritize the response to critical resource requests, ensuring low latency and high success rate for critical resource requests and guaranteeing user experience.

[0128] In some embodiments, the above method further includes:

[0129] Based on the percentage of uploaded fragments, determine whether the upload is complete; after each target file fragment is uploaded, send an archive merge request to the server so that the server can merge the target file fragments to obtain the target merged file.

[0130] In this embodiment, the client updates the counter or percentage of uploaded fragments each time a target file fragment is successfully uploaded. The client continuously checks that (number of successfully uploaded fragments / total number of fragments) * 100% >= 100%. When this condition is true, it is determined that all fragments of the entire file have been uploaded. Subsequently, the client initiates a specific, lightweight HTTP API request (such as POST / api / upload / complete) to the server to notify the server to begin merging fragments. For example, the request body carries the following key information in JSON format: file name, file's unique identifier, total number of fragments, etc.

[0131] After receiving the merge request, the server will sequentially assemble the target file fragments into a complete file according to the index order of the fragments, move the merged file to the final storage location, and update the database records.

[0132] As can be seen from the above, the embodiments of this application adopt the method of "dynamic segmented upload + network adaptive" to determine the real-time network status based on the upload status of the initial file segments during the file upload process, and dynamically adjust the segment size accordingly, so as to improve the file upload success rate and user experience, and can be flexibly applied to different large file upload scenarios.

[0133] To better implement the above methods, this application also provides a file chunking upload device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0134] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the file fragmentation upload device specifically integrated into a tablet computer as an example.

[0135] For example, such as Figure 3 As shown, the file chunking upload device may include a file parsing unit 310, a file chunking unit 320, an upload statistics unit 330, a chunking adjustment unit 340, and a chunking upload unit 350, as follows:

[0136] The file parsing unit 310 is used to obtain the file to be uploaded and file information, the file information including at least the file size;

[0137] File fragmentation unit 320 is used to split the file to be uploaded into multiple initial file fragments based on the initial network speed.

[0138] The upload statistics unit 330 is used to upload the initial file fragments to the server sequentially and record the upload time and upload rate of the initial file fragments.

[0139] The fragment adjustment unit 340 is used to adjust the size of the next initial file fragment based on the upload time and upload rate of the already uploaded initial file fragments, so as to obtain the target file fragments.

[0140] The fragmented upload unit 350 is used to upload fragments of the target file to the server.

[0141] In some embodiments, the segmentation adjustment unit 340 includes:

[0142] The sliding window determination sub-unit is used to determine the initial width of the sliding window based on the user's historical network status data;

[0143] The sliding window processing subunit is used to process the uploaded initial file fragments based on a sliding window with an initial width, and to determine the average time and average rate of multiple initial file fragments within the sliding window.

[0144] The slice size determination subunit is used to determine the target slice size based on the average time consumption and the average rate.

[0145] The fragment size adjustment subunit is used to adjust the next initial file fragment to the target fragment size to obtain the target file fragment.

[0146] In some embodiments, the apparatus further includes:

[0147] The upload change statistics unit is used to determine the time variation coefficient based on the upload time of the K most recently uploaded initial file fragments;

[0148] The sliding window adjustment unit is used to adjust the initial width of the sliding window based on the time variation coefficient to obtain the target width;

[0149] The sliding window processing unit is used to process the uploaded initial file fragments based on a sliding window with a target width.

[0150] In some embodiments, the file fragmentation unit 320 includes:

[0151] The network class determination subunit is used to determine the network class to which the initial network speed belongs based on preset network class classification rules;

[0152] The initial fragmentation subunit is used to split the file to be uploaded based on the preset fragmentation size corresponding to the network level, resulting in multiple initial file fragments.

[0153] In some embodiments, the apparatus further includes:

[0154] The test request unit is used to send a test file retrieval request to the server and record the request initiation time;

[0155] The test response unit is used to receive the test file returned by the server and record the request end time and file size;

[0156] The initial network speed determination unit is used to determine the initial network speed based on the request initiation time, request end time, and file size.

[0157] In some embodiments, the apparatus further includes:

[0158] The request acquisition unit is used to acquire network requests and determine whether a network request is a critical resource request.

[0159] The fragment pause unit is used to pause the adjustment of the next initial file fragment size if the network request is a critical resource request;

[0160] The fragment recovery unit is used to restore the adjustment of the next initial file fragment size after detecting that the network request response has been completed.

[0161] In some embodiments, the apparatus further includes:

[0162] The upload statistics unit is used to determine whether the upload is complete based on the percentage of uploaded segments.

[0163] The fragment merge request unit is used to send an archive merge request to the server after each target file fragment has been uploaded, so that the server can merge the target file fragments to obtain the target merged file.

[0164] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0165] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0166] As can be seen from the above, the file chunking upload device in this embodiment adopts the method of "dynamic chunking upload + network adaptive" to determine the real-time network status based on the upload status of the initial file chunks during the file upload process, and dynamically adjusts the chunk size accordingly to improve the file upload success rate and user experience. It can be flexibly applied to different large file upload scenarios.

[0167] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0168] In some embodiments, the file chunking upload device can also be integrated into multiple electronic devices. For example, the file chunking upload device can be integrated into multiple servers, and multiple servers can implement the file chunking upload method of this application.

[0169] In this embodiment, as Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0170] The electronic device may include components such as a processor 410 with one or more processing cores, a memory 420 with one or more computer-readable storage media, a power supply 430, an input module 440, and a communication module 450. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0171] The processor 410 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 420, and by calling data stored in the memory 420, it performs various functions and processes data, thereby performing overall detection of the electronic device. In some embodiments, the processor 410 may include one or more processing cores; in some embodiments, the processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 410.

[0172] The memory 420 can be used to store software programs and modules. The processor 410 executes various functional applications and data processing by running the software programs and modules stored in the memory 420. The memory 420 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 420 may also include a memory controller to provide the processor 410 with access to the memory 420.

[0173] The electronic device also includes a power supply 430 that supplies power to the various components. In some embodiments, the power supply 430 can be logically connected to the processor 410 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 430 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0174] The electronic device may also include an input module 440, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0175] The electronic device may also include a communication module 450. In some embodiments, the communication module 450 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 450 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0176] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 410 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 420 according to the following instructions, and the processor 410 runs the applications stored in the memory 420 to realize various functions, as follows:

[0177] The process involves: obtaining the file to be uploaded and its information, including at least its size; splitting the file into multiple initial file fragments based on the current initial network speed; sequentially uploading these initial file fragments to the server and recording the upload time and speed for each fragment; adjusting the size of the next initial file fragment based on the upload time and speed of the previously uploaded fragments to obtain the target file fragment; and finally, uploading the target file fragment to the server. Specific implementation details for each of these operations can be found in the preceding embodiments and will not be repeated here.

[0178] As can be seen from the above, the embodiments of this application adopt the method of "dynamic segmented upload + network adaptive" to determine the real-time network status based on the upload status of the initial file segments during the file upload process, and dynamically adjust the segment size accordingly, so as to improve the file upload success rate and user experience, and can be flexibly applied to different large file upload scenarios.

[0179] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0180] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the file fragment upload methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0181] Obtain the file to be uploaded and its information, including at least the file size; based on the initial network speed, split the file to be uploaded into multiple initial file fragments; sequentially upload the initial file fragments to the server, and record the upload time and upload rate corresponding to each initial file fragment; adjust the size of the next initial file fragment based on the upload time and upload rate of the already uploaded initial file fragments to obtain the target file fragment; upload the target file fragment to the server.

[0182] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0183] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the aspects or various optional implementations of the aspects described in the above embodiments.

[0184] Since the instructions stored in the storage medium can execute the steps of any of the file fragment upload methods provided in the embodiments of this application, the beneficial effects that any of the file fragment upload methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0185] The foregoing has provided a detailed description of a file fragment uploading method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A file chunk uploading method, characterized in that, The method comprises the following steps: obtaining a file to be uploaded and file information, wherein the file information at least comprises a file size; splitting the file to be uploaded according to an initial network speed of a current network to obtain a plurality of initial file segments; uploading the initial file segments to a server in sequence and recording upload time consumption and upload speed of the initial file segments; adjusting a size of a next initial file segment according to the upload time consumption and the upload speed of the uploaded initial file segments to obtain a target file segment; uploading the target file segment to the server; wherein the step of adjusting the size of the next initial file segment according to the upload time consumption and the upload speed of the uploaded initial file segments to obtain the target file segment comprises the following steps: determining an initial width of a sliding window based on historical network state data of a user; determining a time variation coefficient based on upload time consumption of K latest initial file segments; adjusting the initial width of the sliding window based on the time variation coefficient to obtain a target width; processing the uploaded initial file segments based on the sliding window with the target width to determine time consumption average and speed average of a plurality of initial file segments in the sliding window; determining a target segment size based on the time consumption average and the speed average; adjusting the next initial file segment to the target segment size to obtain the target file segment.

2. The file chunk uploading method of claim 1, wherein, The step of splitting the file to be uploaded according to the initial network speed of the current network to obtain the plurality of initial file segments comprises the following steps: determining a network level to which the initial network speed belongs based on a preset network level division rule; splitting the file to be uploaded based on a preset segment size corresponding to the network level to obtain the plurality of initial file segments.

3. The file chunk uploading method of claim 1, wherein, The method further comprises the following steps: sending a test file acquisition request to the server and recording a request initiation time; receiving a test file returned by the server and recording a request end time and a file size; determining the initial network speed of the current network based on the request initiation time, the request end time and the file size.

4. The file chunk uploading method of claim 1, wherein, The method further comprises the following steps: acquiring a network request and determining whether the network request is a key resource request; if the network request is the key resource request, suspending adjustment of the size of the next initial file segment; resuming the adjustment of the size of the next initial file segment when it is detected that the network request is responded.

5. The file chunk uploading method of any one of claims 1-4, wherein, The method further comprises the following steps: determining whether the uploading is completed according to a percentage of uploaded segments; after each target file segment is uploaded, sending an archiving and merging request to the server to enable the server to merge the target file segments to obtain a target merged file.

6. A file chunk uploading apparatus characterized by comprising: The method comprises the following steps: a file analysis unit configured to obtain a file to be uploaded and file information, wherein the file information at least comprises a file size; a file segment unit configured to split the file to be uploaded according to an initial network speed of a current network to obtain a plurality of initial file segments; an upload recording unit configured to upload the initial file segments to a server in sequence and record upload time consumption and upload speed of the initial file segments; The slice adjusting unit is configured to adjust a size of a next initial file slice according to an upload time consumption and an upload rate of the uploaded initial file slice, to obtain a target file slice; The slice uploading unit is configured to upload the target file slice to a server; The slice adjusting unit comprises: The sliding window determining subunit is configured to determine an initial width of a sliding window based on historical network state data of a user; The upload variation statistical unit is configured to determine a time variation coefficient based on an upload time consumption of K initial file slices uploaded recently; The sliding window adjusting unit is configured to adjust the initial width of the sliding window based on the time variation coefficient, to obtain a target width; The sliding window processing subunit is configured to process the uploaded initial file slices based on the sliding window with the target width, to determine a time consumption mean and a rate mean of a plurality of initial file slices in the sliding window; The slice size determining subunit is configured to determine a target slice size based on the time consumption mean and the rate mean; The slice size adjusting subunit is configured to adjust a next initial file slice to the target slice size, to obtain a target file slice.

7. An electronic device, comprising: The computer readable storage medium stores a plurality of instructions, which are adapted to be loaded by a processor to execute the steps in the file slice uploading method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, which are adapted to be loaded by a processor to execute the steps in the file slice uploading method according to any one of claims 1-5.

Images