Large file or folder uploading method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of data processing, and relates to a large file or folder uploading method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a large file or folder needing to be uploaded; performing local caching on the large file or the folder; segmenting the cached large file or folder into a plurality of sub-files according to a preset file segmentation size threshold value; storing the plurality of sub-files in a local cache; starting multiple threads, and uploading the plurality of sub-files to a target end; and combining the plurality of sub-files into a large file or a folder, and releasing a local cache space. According to the method, the uploading speed and the success rate are improved, the security and the reliability of data are enhanced, the problem of uploading failure caused by factors such as network instability, page refreshing and browser abnormity when a large file or a large folder is uploaded in a browser can be effectively solved, an efficient, stable and breakpoint relaying large file and large folder uploading function is realized, and the uploading efficiency of the large file or the large folder is improved. And the user experience is obviously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a method, apparatus, computer device, and storage medium for uploading large files or folders. Background Art

[0002] In the era of the rapid development of Internet technology today, it has become extremely common for users to perform upload operations of large files or large folders in a browser. For example, designers need to upload a folder containing a large number of high-definition pictures and design source files to a cloud storage platform for team collaboration and resource sharing; enterprise users may need to upload large database backup files or folders containing numerous project materials to the company server. However, uploading large files faces many challenges. Due to the huge amount of data in the file itself, the upload process takes a long time. And the network environment is complex and changeable, with unstable factors such as network fluctuations and signal instability. During the upload process, once there is a network disconnection, the user accidentally refreshes the page, or the browser restarts due to a system failure, the upload task will be interrupted, resulting in upload failure, and the user has to start the entire upload process again, which brings great trouble and time waste to the user. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides a method for uploading a large file or folder, which adopts the following technical solutions, including:

[0004] Obtain a large file or folder to be uploaded;

[0005] Perform local caching on the large file or folder;

[0006] Divide the cached large file or folder into multiple sub-files according to a preset file splitting size threshold;

[0007] Store the multiple sub-files in local cache;

[0008] Start multiple threads to upload the multiple sub-files to the target end;

[0009] Merge the multiple sub-files into a large file or folder, and release the local cache space.

[0010] Preferably, the step of obtaining a large file or folder to be uploaded specifically includes:

[0011] Select the Vue.js or React.js front-end development framework, and combine with the HTML5 file system API to build a user operation interface for file selection and upload progress display;

[0012] Use Node.js combined with the Express framework to build a back-end server for receiving and processing uploaded files;

[0013] Obtain large files or folders to be uploaded through the front-end user operation interface.

[0014] Preferably, the step of locally caching the large file or folder specifically includes:

[0015] Copy the large file or folder to the local cache directory through the filesystem interface of the browser;

[0016] Record the file information of each cached file, where the file information includes the file name, file size, file type, and the large file information to which the split file belongs.

[0017] Preferably, the step of splitting the cached large file or folder into multiple sub-files according to a preset file split size threshold specifically includes:

[0018] Calculate the number of segments required to split the large file or folder according to the preset file split size threshold;

[0019] Split the large file or folder into multiple sub-files in the order of the number of segments, and set a unique identifier for each sub-file.

[0020] Preferably, the step of storing multiple sub-files in the local cache specifically includes:

[0021] Determine the local cache location and create a local cache directory;

[0022] Store multiple sub-files in the local cache directory;

[0023] Update the cache status in real time.

[0024] Preferably, the step of starting multiple threads to upload multiple sub-files to the target end specifically includes:

[0025] Create multiple Web Worker threads, and each Web Worker thread is responsible for uploading one or more sub-files;

[0026] Configure resource parameters at the target end to ensure that multiple file upload requests can be processed simultaneously;

[0027] Start multiple Web Worker threads, and let multiple Web Worker threads upload multiple sub-files to the target end.

[0028] Preferably, the step of merging multiple sub-files into a large file or folder and releasing the local cache space specifically includes:

[0029] Receive and store multiple sub-files;

[0030] Read the content of each sub-file in sequence according to the numbering order of the multiple sub-files, and write it into a new file;

[0031] Release the local cache space.

[0032] To solve the above technical problems, the present invention also provides a large file or folder uploading device, which adopts the following technical solutions, including:

[0033] An acquisition module, configured to acquire a large file or folder to be uploaded;

[0034] A cache module, configured to perform local caching on the large file or folder;

[0035] A splitting module, configured to split the cached large file or folder into multiple sub-files according to a preset file splitting size threshold;

[0036] A storage module, configured to store the multiple sub-files in local cache;

[0037] An uploading module, configured to start multi-threading and upload the multiple sub-files to a target end;

[0038] A merging module, configured to merge the multiple sub-files into a large file or folder and release the local cache space.

[0039] To solve the above technical problems, the present invention also provides a computer device, which adopts the following technical solutions, including a memory and a processor. Computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the steps of the above large file or folder uploading method are implemented.

[0040] To solve the above technical problems, the present invention also provides a computer-readable storage medium, which adopts the following technical solutions. Computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the steps of the above large file or folder uploading method are implemented.

[0041] Compared with the prior art, the present invention mainly has the following beneficial effects: To ensure the stability of the upload process, local caching is performed on large files or folders, which can effectively avoid data loss problems caused by network fluctuations; according to a preset file splitting size threshold, the cached large files or folders are split into multiple sub-files. This splitting strategy can not only reduce the risk of single upload failure but also significantly improve the upload efficiency because smaller files can usually be transmitted faster. The multiple split sub-files are also stored in the local cache for future upload; a multi-thread mechanism is started to upload these sub-files to the target end in parallel. The application of multi-thread technology can make full use of the network bandwidth and further shorten the upload time. Once all sub-files are successfully uploaded to the target end, a merge operation is performed to recombine them into a large file or folder; to save storage space, the local cache space is released in a timely manner to ensure the effective use of system resources, which not only improves the upload speed and success rate but also enhances the data security and reliability through the local caching and file splitting strategies; the multi-thread upload and merge operations also greatly simplify the user operation process; since the file cache is always retained locally, even if the user refreshes the page or even restarts the browser, breakpoint resumption can be achieved, effectively solving the problem of upload failure caused by factors such as network instability, page refresh, and browser anomalies during the upload of large files or large folders in the browser, realizing an efficient, stable, and breakpoint-resumable large file and large folder upload function, and significantly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the solutions in the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the following-described drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a flowchart of an embodiment of the method for uploading large files or folders of the present invention;

[0044] Figure 2 It is a schematic structural diagram of an embodiment of the device for uploading large files or folders of the present invention;

[0045] Figure 3 It is a schematic structural diagram of an embodiment of the computer device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs; the terms used in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit the present invention; the terms "comprising" and "having" and any variations thereof in the specification and claims of this invention and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this invention or the above drawings are used to distinguish different objects and not to describe a specific order.

[0047] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The phrase does not necessarily refer to the same embodiment at every occurrence in the specification, nor is it an independent or alternative embodiment mutually exclusive of other embodiments. Those skilled in the art will understand explicitly and implicitly that the embodiments described herein can be combined with other embodiments.

[0048] In order to enable those skilled in the art to better understand the solution of this invention, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings.

[0049] It should be noted that the method for uploading large files or folders provided by the embodiments of this invention is generally executed by a server / terminal device. Correspondingly, the device for uploading large files or folders is generally disposed in the server / terminal device.

[0050] It should be understood that the numbers of terminal devices, networks, and servers are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers.

[0051] Embodiment 1

[0052] Please refer to Figure 1 , which shows a flowchart of an embodiment of the method for uploading large files or folders of this invention. The method for uploading large files or folders includes the following steps:

[0053] Step S1, obtain the large file or folder to be uploaded.

[0054] In this embodiment, the electronic device (such as a server / terminal device) on which the large file or folder uploading method runs can receive a large file or folder uploading request through a wired connection or a wireless connection. It should be noted that the above wireless connection methods can include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future-developed wireless connection methods.

[0055] A large file refers to a single file that occupies a relatively large amount of disk space, which may exceed several hundred megabytes (MB) or even several gigabytes (GB) or dozens of GB, depending on the specific application scenario and system configuration. A large folder refers to a folder that contains a large number of files or sub-folders. Even if the individual files are not large, the overall occupied space is significant, and it may also contain a complex hierarchical structure, thus posing higher requirements for file system traversal, search, and backup operations. The specific definition of a large file or folder can be determined according to actual needs. For example, in this embodiment, a storage space exceeding 500 megabytes is defined as a large file or folder.

[0056] Specifically, in step S1, obtaining the large file or folder to be uploaded can further include the steps of:

[0057] S11, Select a front-end development framework such as Vue.js or React.js, and combine it with the HTML5 file system API to build a user operation interface for file selection and upload progress display.

[0058] Select a suitable front-end development framework, such as Vue.js or React.js, to build the user operation interface. These frameworks provide features such as componentization, data binding, and state management, making the development efficient and the code highly maintainable. Install the relevant dependencies through npm or yarn and initialize the project.

[0059] Create a file selection component. Use Vue.js or React.js to create a file selection component. Add a file input element to the component, set the multiple attribute to support multi-file selection, and set the webkitdirectory or directory attribute to support folder selection (note: different browsers have different support for folder selection). Listen for the change event, and when the user selects a file or folder, obtain the file list.

[0060] Through the HTML5 file input element ( <input type＝file> ) or the drag-and-drop interface, allow the user to select a file or folder.

[0061] Handle the file selection event. In the callback function of the file selection event, obtain the file list through event.target.files. Traverse the file list and process each file object, including reading information such as the file name and file size. Use conditional statements to check

[0062] whether window.webkitRequestFileSystem or window.requestFileSystem exists to determine whether the browser supports the File System API. If not supported, display a prompt message and provide alternative solutions (such as selecting files one by one through the file input element).

[0063] When the user selects a file or folder, obtain the list of file objects and display them to the user. At the same time, check the browser's support for the local file storage interface of the file system.

[0064] Display the file list. Maintain a file list in the state of the component. Use the rendering mechanism of Vue.js or React.js to render the file list onto the interface. Display information such as the name and size of each file.

[0065] Display the files or folders selected by the user in a list on the interface to facilitate the user to confirm the files to be uploaded.

[0066] Implement the upload progress display. Use the state management function of Vue.js or React.js to maintain an upload progress variable. In the callback function of the file upload, update the upload progress variable according to the number of bytes uploaded. Display the upload progress bar or progress percentage on the interface.

[0067] During the file upload process, the upload progress can be displayed in real time to improve the user experience.

[0068] S12. Build a backend server using Node.js combined with the Express framework to receive and process the uploaded files.

[0069] Build a Node.js server. Install Node.js and npm. Use npminit to initialize a new Node.js project. Install the Express framework and other necessary dependencies, such as multer (for handling file uploads). Create a server file (such as `server.js`) and configure the Express application.

[0070] Use Node.js and the Express framework to build a web server to receive the upload requests sent by the frontend.

[0071] Configure a file upload middleware, introduce the multer library, and create a multer instance. Configure storage options such as the storage location (disk path or cloud storage URL) and the file name generation rule. Set restrictions such as file size and quantity to prevent malicious uploads. Use the single, array, or fields method of the multer instance to handle different types of file upload requests.

[0072] Use the multer middleware to configure the file upload route and set the storage options and restrictions.

[0073] Create a file upload route. Define a POST route in the Express application, and set the path according to requirements (such as / upload). Use the configured multer middleware to handle the upload request. In the route handler function, obtain the uploaded file object and perform corresponding processing (such as saving the file information to the database, generating an access link, etc.). Return a response to the front end, including the upload result and possible error messages.

[0074] Define a file upload route to receive POST requests sent by the front end and handle the uploaded files.

[0075] Error handling and logging. Add error handling logic to the front end and back end to capture and handle possible exceptions (such as the file being too large, network errors, etc.). Use a logging library (such as winston) in the back end to record key information during the upload process (such as upload time, file name, user ID, etc.). Troubleshoot and handle upload problems based on the log information.

[0076] Handle possible errors during the file upload process and record logs for troubleshooting.

[0077] S13. Obtain the large file or folder to be uploaded through the front-end user operation interface.

[0078] The front-end part uses a front-end development framework to implement the interaction of the user interface and the display of the upload progress; the front-end interface provides an intuitive file selection function and supports multi-file selection; the back-end part uses the Node.js and Express frameworks to build a server for receiving and processing the uploaded files. Through the cooperation of the front end and the back end, obtain the large file or folder to be uploaded.

[0079] Step S2. Perform local caching on the large file or folder.

[0080] When specifically implemented, step S2 of performing local caching on the large file or folder may further include the steps:

[0081] S21. Copy the large file or folder to the local cache directory through the browser's filesystem interface.

[0082] The filesystem interface of the browser refers to a Web API that allows web applications to perform file operations in the user's local file system. This interface is mainly used to represent the file system, enabling web applications to access and operate on files and directories in the file system. It should be noted that for security reasons, web applications cannot directly access the entire file system of the user, but rather operate through the sandboxed file system provided by the browser.

[0083] Call the window.requestFileSystem() method to request a FileSystem object representing the sandboxed file system.

[0084] Once the FileSystem object is obtained, the root directory of the file system can be accessed through the root property of the object. Then, use the getFile() or getDirectory() method to locate the target file or folder.

[0085] It should be noted that for copying folders, it is necessary to recursively traverse all files and subfolders in the folder and copy them one by one to the local cache directory.

[0086] S22, record the file information of each cached file. The file information includes the file name, file size, file type, and the information of the large file to which the split file belongs.

[0087] Design a database table structure to store the file information of each cached file. The database table structure contains the following fields: file name (used to uniquely identify each file), file size (recording the file volume in bytes), file type (classified according to the file extension or MIME type), the name of the large file to which it belongs (for split files, recording the name of its original large file), and the split sequence number (used for sorting and reorganizing split files).

[0088] For each cached file, when the file is created or updated, automatically capture and insert the above information into the database. The file name should use a unique identifier generated by the system or retain the original file name to ensure traceability. The file size is directly obtained through the API of the file system. The file type is determined by parsing the file extension or using a file content detection library to ensure accuracy.

[0089] For split files, especially record the name of the large file to which they belong, and when necessary, maintain the order of file segments through the split sequence number. This helps in the subsequent file reorganization process to ensure that large files can be accurately restored.

[0090] In specific implementation, a regular data verification mechanism can also be set up. For example, by comparing hash values, the consistency and integrity of cached files can be verified to prevent data corruption or tampering.

[0091] Step S3: Split the cached large file or folder into multiple sub-files according to a preset file splitting size threshold.

[0092] In specific implementation, step S3: Splitting the cached large file or folder into multiple sub-files according to a preset file splitting size threshold can further include the steps:

[0093] S31: Calculate the number of segments into which the large file or folder needs to be split according to the preset file splitting size threshold.

[0094] The preset file splitting size threshold can be determined according to actual needs. If processing a single large file, its size can be directly obtained through the attribute information provided by the file system. If the target is a folder, recursively calculate the total size of all files in it.

[0095] After obtaining the total capacity of the large file or folder, divide it by the preset splitting size threshold, and the quotient obtained is the initially estimated number of segments. However, the remainder generated by the division needs to be considered in this process. If the remainder is not zero, it indicates that the last segment will be smaller than the other segments. At this time, it is necessary to decide according to the actual situation whether to take the remainder part as a separate segment or merge it into the previous segment (if a slight difference in segment size is allowed).

[0096] In addition, the limitations of the file system, such as the maximum size of a single file and the length of the file path, also need to be noted to ensure that the split segments can be correctly created and stored on the target storage medium.

[0097] S32: Split the large file or folder into multiple sub-files in the order of the number of segments, and set a unique identifier for each sub-file.

[0098] For large files, file splitting tools such as the split command (in Unix / Linux systems) or third-party software such as WinRAR, 7-Zip, etc. (in Windows systems) can be used. These tools support splitting by size or number and can automatically name the generated sub-files.

[0099] When performing the split operation, specify the number of cut segments and the size of each segment. Ensure that a unique identifier is set for each generated sub-file by embedding a serial number, timestamp, or hash value, etc. in the file name. For example, the sub-files can be named in the format of "original file name_segment serial number", where the "segment serial number" is a consecutive integer starting from 1.

[0100] After the splitting is completed, verify the integrity and readability of each sub-file. This can be done by attempting to open or recombine the sub-files. If a tool that supports checksum is used, the generated checksum file can also be utilized to verify the integrity of the sub-files.

[0101] Step S4, store multiple sub-files in the local cache.

[0102] Specifically, in implementation, step S4 of storing multiple sub-files in the local cache can further include the steps of:

[0103] S41, determine the local cache location and create a local cache directory.

[0104] Check the default cache path according to the operating system type. For example, in the Windows system, the cache location is in the "AppData" folder under the user directory; while in Linux or macOS, it is in the ".cache" directory under the user's home directory. Other local cache directories can also be selected according to actual needs.

[0105] Use a file manager or command-line tool to navigate to the target location and create a new cache directory according to the application requirements. Ensure that the directory naming is standardized and descriptive for subsequent management and maintenance. This process helps to standardize the data access and storage process and improve the overall operating efficiency of the system.

[0106] S42, store multiple sub-files in the local cache directory.

[0107] Utilize a programming language or script (such as the os module in Python) to implement an automated storage process, and store each sub-file in a preset sub-directory according to its attributes.

[0108] During the storage process, expired or invalid sub-files can be periodically cleared to free up storage space.

[0109] S43, update the cache status in real time.

[0110] During the process of storing sub-files, the front end needs to update the cache status in real time, including information such as the number of sub-files already stored, the total number of sub-files, and the upload progress. These information can be displayed to the user through the front-end interface so that the user can understand the progress and status of the upload.

[0111] Updating the cache status allows the user to understand the upload progress and status in real time, improving the user experience. At the same time, it also provides the necessary information support for subsequent file upload and merging.

[0112] Step S5, start multiple threads to upload multiple sub-files to the target end.

[0113] In specific implementation, step S5 of starting multiple threads to upload multiple sub-files to the target end can further include the following steps:

[0114] S51, Create multiple Web Worker threads, and each Web Worker thread is responsible for uploading one or more sub-files.

[0115] Web Worker threads are part of the HTML5 standard, which provides Web applications with the ability to run JavaScript code in background threads. These background threads execute independently of the main thread (i.e., the UI thread), so they do not block the rendering and interaction of the user interface and are very suitable for performing tasks that are computationally intensive, time-consuming, and do not require direct manipulation of the DOM.

[0116] To improve the user experience and application performance, the Web Worker threads can be used to move the file upload process to the background thread for execution.

[0117] Write Worker scripts: Create one or more independent JavaScript files, which will contain the upload logic that needs to be executed in the background.

[0118] S52, Configure resource parameters at the target end to ensure that multiple file upload requests can be processed simultaneously.

[0119] Evaluate the existing computing, storage, and network resources at the target end. For storage resources, ensure that there is enough storage space to accommodate the files to be uploaded soon, and consider using a distributed storage system to improve storage efficiency and access speed.

[0120] When configuring parameters, special attention should be paid to the setting of concurrent processing capabilities. This includes adjusting the thread pool size of the target end such as the server, the database connection pool size, etc., to ensure that the system can still remain stable in high-concurrency scenarios. At the same time, consider using load balancing technology to distribute upload requests to avoid overloading a single server.

[0121] S53, Start multiple Web Worker threads, and let multiple Web Worker threads upload multiple sub-files to the target end.

[0122] Create a Web Worker instance in the main thread: Use

[0123] the newWorker(URL.createObjectURL(newBlob([workerScript]))) constructor to create a Worker instance for each sub-file to be uploaded and specify the path of the Worker script.

[0124] Communication with Web Workers: The file data is sent to the Worker via the postMessage() method. After the Worker processes the data, the result is sent back to the main thread via the postMessage() method. The main thread can listen for the onmessage event of the Worker to receive these messages and perform corresponding operations accordingly, such as uploading file fragments.

[0125] Via the postMessage method, the main thread sends the corresponding sub-file data and its upload configuration to each Worker instance to ensure accurate data transfer. At the same time, the main thread should listen for message events from the Worker to track the upload progress or handle errors.

[0126] After receiving the data, each Web Worker instance independently performs the upload operation and sends a status update to the main thread via postMessage upon completion. The main thread aggregates this information and feeds back the overall upload progress or processes the final result to the user through the front end.

[0127] By making full use of the parallel processing ability of Web Workers, the efficiency and response speed of multi-file uploads are significantly improved. In this way, large files can be uploaded in chunks, and each chunk is processed in an independent Web Worker thread, thus greatly enhancing the upload efficiency and user experience. At the same time, since the Web Worker threads execute independently of the main thread, they do not block the rendering and interaction of the user interface, enabling the application to remain responsive even when uploading large files.

[0128] Step S6: Merge multiple sub-files into a large file or folder and release the local cache space.

[0129] Specifically, in step S6, merging multiple sub-files into a large file or folder and releasing the local cache space can further include the steps of:

[0130] S61: Receive and store multiple sub-files.

[0131] Prepare a storage device to ensure there is enough storage space to receive and store these sub-files. The storage device can be a local hard drive, a network drive, a cloud storage service, or any other storage device. Insufficient storage space may lead to operation failures or data loss.

[0132] Based on the source of the sub-files, select the receiving method. If downloading from the network, the browser's built-in download manager or a professional download tool can be used. If copying from an external storage device, simply connect the device to the computer and then copy the files to the specified location.

[0133] After receiving all the sub-files, their integrity can be verified by calculating the hash value or checksum of the files to ensure that no data corruption occurred during transmission.

[0134] S62, Read the content of each sub-file in the order of the sub-file numbers and write it into a new file.

[0135] Depending on the operating system and specific requirements, choose a scripting language or software tool. For example, on Windows systems, batch scripts or PowerShell scripts can be used; on Linux or Mac OS, Bash scripts or Python scripts can be used. In addition, specialized file merging tools such as EaseUS Todo PCTrans Pro can also be used.

[0136] Write a merging script. The following is an example script written in Python that can read the content of each sub-file in the order of the sub-file numbers and write them into a new file.

[0137] import os# Define the directory where the sub-files are located and the path of the new file.

[0138] subfile_dir = path / to / subfiles# Replace with the actual path of the sub-file directory.

[0139] newfile_path = path / to / newfile.txt# Replace with the actual path of the new file.

[0140] Get the list of sub-files and sort them in order of number:

[0141] subfiles = sorted([f for f in os.listdir(subfile_dir) if

[0142] f.endswith(.txt) and f.startswith(file)]) Assume the sub-file name starts with file followed by a number.

[0143] Open the new file for writing:

[0144] with open(newfile_path, 'w', encoding='utf-8') as newfile:

[0145] for subfile in subfiles:

[0146] subfile_path = os.path.join(subfile_dir, subfile)

[0147] # Open the subfile and read its content.

[0148] with open(subfile_path, 'r', encoding='utf-8') as sf:

[0149] content = sf.read()

[0150] # Write the subfile content to the new file.

[0151] newfile.write(content + '\n') # Add a newline character after each subfile content to separate.

[0152] print(f'All subfiles have been successfully merged into {newfile_path}')

[0153] Run the merge script. After writing the script, save it as an executable file (e.g.,.py file) and run it in the command line or terminal. Make sure the required Python environment has been installed before running the script.

[0154] Verify the merge result. After the merge is complete, open the new file and check if its content is correct. The merge result can be verified by comparing the content of the subfiles and the new file.

[0155] S62, Release the local cache space.

[0156] After merging the subfiles and creating the new file, the original subfiles may no longer be needed, so the local cache space they occupy can be released. This helps optimize system performance and save storage space.

[0157] Before deleting any files, back up important data. This can be achieved by copying the files to an external storage device, cloud storage service, or creating a backup image.

[0158] Delete the subfiles. Once it is confirmed that the new file contains all the necessary subfile content and important data has been backed up, the original subfiles can be deleted. This can be achieved through a file manager, command line, or script.

[0159] For example, on Windows systems, the command line tools del or rmdir can be used to delete files or folders. On Linux or Mac OS, the rm command can be used to delete files.

[0160] Clean the system cache. In addition to deleting sub-files, the system cache can be further cleaned to free up more space. This includes browser caches, system temporary files, application caches, etc.

[0161] Cleaning browser caches. Most browsers provide options to clean caches. In the browser settings or preferences menu, you can find the option to clean caches and check the cache items to be deleted.

[0162] Cleaning system temporary files. On Windows systems, you can use the Disk Cleanup tool to delete system temporary files. On Linux or Mac OS, you can use tools such as tmpwatch or bleachbit to clean temporary files.

[0163] Cleaning application caches. Many applications store cached data locally to speed up access. These cached data can be deleted through the application's settings menu or a dedicated cache cleaning tool.

[0164] Verify the freed space. After deleting sub-files and cleaning the system cache, check the disk usage in the file manager or use command-line tools (such as df or dir) to verify whether the space has been successfully freed.

[0165] By merging sub-files, the number of files can be reduced, thus reducing the overhead and time during the transmission process. This is especially applicable to the situation of transmitting large files over the network.

[0166] Freeing up local cache space can optimize system performance, reduce disk fragmentation, and improve file access speed. In addition, it can also provide more storage space for other important files or applications.

[0167] Merging sub-files and cleaning caches can simplify the file management process, reduce the risk of file chaos and loss. This helps improve work efficiency and reduce the likelihood of errors.

[0168] By backing up important data and deleting unnecessary sub-files, data security can be enhanced. This can prevent the leakage of sensitive information or malicious tampering of data.

[0169] Implementing this embodiment has the following beneficial effects: To ensure the stability of the upload process, large files or folders are locally cached. This step can effectively avoid data loss problems caused by network fluctuations. According to a preset file segmentation size threshold, the cached large files or folders are segmented into multiple sub-files. This segmentation strategy can not only reduce the risk of single upload failure but also significantly improve the upload efficiency because smaller files can usually be transmitted faster. The multiple segmented sub-files are also stored in the local cache for future upload. The multi-threaded mechanism is started to upload these sub-files to the target end in parallel. The application of multi-threaded technology can make full use of network bandwidth and further shorten the upload time. Once all sub-files are successfully uploaded to the target end, a merge operation is performed to recombine them into a large file or folder. To save storage space, the local cache space is released in a timely manner to ensure the effective use of system resources. This not only improves the upload speed and success rate but also enhances the data security and reliability through local caching and file segmentation strategies. The multi-threaded upload and merge operations also greatly simplify the user operation process. Since the file cache is always retained locally, even if the user refreshes the page or even restarts the browser, breakpoint resumption can be achieved, effectively solving the problem of upload failure caused by factors such as network instability, page refresh, and browser anomalies during the upload of large files or large folders in the browser, and realizing an efficient, stable, and breakpoint-resumable large file and large folder upload function, significantly enhancing the user experience.

[0170] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, and so on. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0171] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), etc., or a random access memory (RAM), etc.

[0172] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit and can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment but can be executed at different moments. Their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0173] Embodiment 2

[0174] Further referring to Figure 2 As an implementation of the method shown above Figure 1 The present invention provides an embodiment of a large file or folder uploading device. This device embodiment corresponds to the method embodiment shown in Figure 1 and can be specifically applied to various electronic devices.

[0175] As shown in Figure 2 the large file or folder uploading device 70 in this embodiment includes: an acquisition module 71, a cache module 72, a segmentation module 73, a storage module 74, an upload module 75, and a merging module 76. Among them:

[0176] The acquisition module 71 is used to acquire a large file or folder to be uploaded;

[0177] The cache module 72 is used to perform local caching on the large file or folder;

[0178] The segmentation module 73 is used to segment the cached large file or folder into multiple sub-files according to a preset file segmentation size threshold;

[0179] The storage module 74 is used to store multiple sub-files in local cache;

[0180] An upload module 75 for starting multiple threads to upload multiple sub-files to a target end;

[0181] A merging module 76 for merging multiple sub-files into a large file or folder and releasing local cache space.

[0182] Implementing this embodiment has the beneficial effects as follows: To ensure the stability of the upload process, local caching is performed on large files or folders. This step can effectively avoid data loss problems caused by network fluctuations; According to a preset file splitting size threshold, the cached large file or folder is split into multiple sub-files. This splitting strategy can not only reduce the risk of single upload failure but also significantly improve the upload efficiency because smaller files can usually be transmitted faster. The multiple split sub-files are also stored in the local cache for future upload; Start a multi-threaded mechanism to upload these sub-files to the target end in parallel. The application of multi-threaded technology can fully utilize the network bandwidth and further shorten the upload time. Once all sub-files are successfully uploaded to the target end, a merging operation is performed to recombine them into a large file or folder; To save storage space, the local cache space is released in a timely manner to ensure the effective utilization of system resources, which not only improves the upload speed and success rate but also enhances the data security and reliability through local caching and file splitting strategies; The multi-threaded upload and merging operations also greatly simplify the user operation process; Since the file cache is always retained locally, even if the user refreshes the page or even restarts the browser, breakpoint resume can be achieved, effectively solving the problem of upload failure caused by factors such as network instability, page refresh, and browser anomalies during the upload of large files or large folders in the browser, realizing an efficient, stable, and breakpoint-resumable large file and large folder upload function, and significantly improving the user experience.

[0183] Embodiment Three

[0184] To solve the above technical problems, an embodiment of the present invention also provides a computer device. Specifically, please refer to Figure 3 , Figure 3 which is the basic structural block diagram of the computer device in this embodiment.

[0185] The above computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are communicatively connected to each other via a system bus. It should be noted that only the computer device 8 with components such as the memory 81, the processor 82, and the network interface 83 is shown in the figure. However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of the present technology can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0186] The above computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The above computer device can perform human-computer interaction with the user through means such as a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0187] The above memory 81 includes at least one type of readable storage medium. The above readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the above memory 81 can be an internal storage unit of the above computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the above memory 81 can also be an external storage device of the above computer device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 8. Of course, the above memory 81 can also include both the internal storage unit of the above computer device 8 and its external storage device. In this embodiment, the above memory 81 is generally used to store the operating system and various application software installed on the above computer device 8, such as computer-readable instructions for the method of uploading large files or folders. In addition, the above memory 81 can also be used to temporarily store various data that have been output or will be output.

[0188] In some embodiments, the above-mentioned processor 82 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 82 is generally used to control the overall operation of the above-mentioned computer device 8. In this embodiment, the processor 82 is used to run the computer-readable instructions stored in the above-mentioned memory 81 or process data, such as running the computer-readable instructions of the above-mentioned large file or folder uploading method.

[0189] The above-mentioned network interface 83 may include a wireless network interface or a wired network interface, which is generally used to establish a communication connection between the above-mentioned computer device 8 and other electronic devices.

[0190] Implementing this embodiment has the following beneficial effects: To ensure the stability of the upload process, local caching is performed on large files or folders. This step can effectively avoid data loss problems caused by network fluctuations; according to the preset file splitting size threshold, the cached large files or folders are split into multiple sub-files. This splitting strategy can not only reduce the risk of single upload failure but also significantly improve the upload efficiency because smaller files can usually be transmitted faster. The multiple split sub-files are also stored in the local cache for future upload; start the multi-thread mechanism to upload these sub-files to the target end in parallel. The application of multi-thread technology can make full use of the network bandwidth and further shorten the upload time. Once all sub-files are successfully uploaded to the target end, a merge operation is performed to recombine them into a large file or folder; to save storage space, the local cache space is released in a timely manner to ensure the effective use of system resources, which not only improves the upload speed and success rate but also enhances the data security and reliability through local caching and file splitting strategies; multi-threaded upload and merge operations also greatly simplify the user operation process; since the file cache is always retained locally, even if the user refreshes the page or even restarts the browser, breakpoint resumption can be achieved, effectively solving the problem of upload failure caused by factors such as network instability, page refresh, and browser anomalies during the upload of large files or large folders in the browser, realizing an efficient, stable, and breakpoint-resumable large file and large folder upload function, and significantly improving the user experience.

[0191] Embodiment 4

[0192] The present invention also provides another implementation manner, that is, to provide a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor, so that the at least one processor executes the steps of the above-mentioned large file or folder uploading method.

[0193] Implementing this embodiment has the following beneficial effects: To ensure the stability of the upload process, large files or folders are locally cached. This step can effectively avoid data loss problems caused by network fluctuations. According to the preset file splitting size threshold, the cached large files or folders are split into multiple sub-files. This splitting strategy not only reduces the risk of single upload failure but also significantly improves the upload efficiency because smaller files can usually be transmitted faster. The multiple split sub-files are also stored in the local cache for future upload. The multi-thread mechanism is started to upload these sub-files to the target end in parallel. The application of multi-thread technology can make full use of the network bandwidth and further shorten the upload time. Once all sub-files are successfully uploaded to the target end, a merge operation is performed to recombine them into a large file or folder. To save storage space, the local cache space is released in a timely manner to ensure the effective use of system resources. This not only improves the upload speed and success rate but also enhances the data security and reliability through local caching and file splitting strategies. The multi-thread upload and merge operations also greatly simplify the user operation process. Since the file cache is always retained locally, even if the user refreshes the page or even restarts the browser, breakpoint resumption can be achieved, effectively solving the problem of upload failure caused by factors such as network instability, page refresh, and browser anomalies during the upload of large files or folders in the browser, and realizing an efficient, stable, and breakpoint-resumable large file and large folder upload function, significantly improving the user experience.

[0194] Through the description of the above implementation manners, those skilled in the art can clearly understand that the above embodiment method can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0195] Obviously, the embodiments described above are only a part of the embodiments of the present invention, rather than all of them. The preferred embodiments of the present invention are shown in the accompanying drawings, but they do not limit the patent scope of the present invention. The present invention can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present invention more thorough and comprehensive. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure that makes use of the content of the specification and drawings of the present invention and is directly or indirectly applied to other related technical fields shall be within the scope of the patent protection of the present invention by the same token.

Claims

1. A method for uploading large files or folders, characterized in that, It includes the following steps: Obtain a large file or folder to be uploaded; Perform local caching on the large file or folder; Divide the cached large file or folder into multiple sub-files according to a preset file splitting size threshold; Store multiple sub-files in local cache; Start multi-threading and upload multiple sub-files to the target end; Merge multiple sub-files into a large file or folder and release the local cache space.

2. The large file or folder uploading method according to claim 1, wherein The step of obtaining a large file or folder to be uploaded specifically includes: Select the Vue.js or React.js front-end development framework, and combine with the HTML5 File System API to build a user operation interface for file selection and upload progress display; Build a back-end server using Node.js combined with the Express framework for receiving and processing uploaded files; Obtain a large file or folder to be uploaded through the front-end user operation interface.

3. The large file or folder uploading method according to claim 1, wherein The step of performing local caching on the large file or folder specifically includes: Copy the large file or folder to the local cache directory through the browser's filesystem interface; Record the file information of each cached file, where the file information includes the file name, file size, file type, and the large file information to which the split file belongs.

4. The large file or folder uploading method according to claim 1, wherein The step of dividing the cached large file or folder into multiple sub-files according to a preset file splitting size threshold specifically includes: Calculate the number of segments into which the large file or folder needs to be split according to the preset file splitting size threshold; Divide the large file or folder into multiple sub-files in the order of the number of segments, and set a unique identifier for each sub-file.

5. The large file or folder uploading method according to claim 1, wherein The step of storing multiple sub-files in local cache specifically includes: Determine the local cache location and create a local cache directory; Store multiple sub-files in the local cache directory; Update the cache status in real time.

6. The large file or folder uploading method according to claim 1, wherein The step of starting multi-threading and uploading multiple sub-files to the target end specifically includes: Create multiple Web Worker threads, and each Web Worker thread is responsible for uploading one or more sub-files; Configure resource parameters at the target end to ensure that multiple file upload requests can be processed simultaneously; Start multiple Web Worker threads, and let multiple Web Worker threads upload multiple sub-files to the target end.

7. The method for uploading a large file or folder according to any one of claims 1 to 6, characterized in that The step of merging multiple sub-files into a large file or folder and releasing the local cache space specifically includes: Receive and store multiple sub-files; Read the content of each sub-file in sequence according to the number order of multiple sub-files, and write it into a new file; Release the local cache space.

8. An apparatus for uploading large files or folders, characterized in that, It includes: An acquisition module for obtaining a large file or folder to be uploaded; A caching module for performing local caching on the large file or folder; A splitting module for dividing the cached large file or folder into multiple sub-files according to a preset file splitting size threshold; A storage module for storing multiple sub-files in local cache; An upload module for starting multi-threading and uploading multiple sub-files to the target end; A merging module, which is used to merge multiple said sub-files into a large file or a folder, and release the local cache space.

9. A computer device, characterized in that, It includes a memory and a processor. Computer-readable instructions are stored in the memory. When the processor executes the computer-readable instructions, the steps of the large file or folder uploading method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that, Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by a processor, the steps of the large file or folder uploading method according to any one of claims 1 to 7 are implemented.

Images