Large file uploading processing method and device, equipment and medium

Abstract

The invention relates to a large file uploading processing method and device, equipment and a medium, and the method comprises the steps: receiving files uploaded by a client in batches, and writing the files received in a single time into a disk at the same time until the writing of the whole large file is completed, so as to obtain a temporary storage file; judging whether the temporary storage file has a file ending identifier or not; and if the temporary storage file has the file ending identifier, asynchronously uploading the temporary storage file to cloud storage to obtain a storage file. According to the method, the files are received for several times and are immediately written into the disk instead of storing the whole large file in the memory firstly, so that the usage amount of the memory of the server is remarkably reduced, the risk of memory overflow caused by insufficient memory can be effectively avoided, and after the file is uploaded by the client, the file is temporarily stored in the disk, so that the storage efficiency of the server is improved. Therefore, the waiting time of the client can be shortened, so that the resource utilization rate is improved, the hardware configuration is reduced, and the investment is saved.

Description

Technical Field

[0001] The present invention relates to the technical field of large file uploading, and in particular to a large file uploading processing method, device, equipment and medium. Background Art

[0002] With the development of the Internet, file upload has become one of the indispensable functions of many applications and services. Traditional file upload methods may encounter problems such as memory overflow when processing large files, and the upload speed is affected by network conditions and server disk performance.

[0003] The existing traditional file upload mainly includes the following methods: 1. Traditional HTTP file upload, which uses the standard HTTP protocol to upload files, and the files are usually sent to the server as part of the form; 2. Stream-based file upload, which uses streams (such as InputStream in the Servlet API) to read file data block by block, and usually uses technologies such as MultipartFile in the Servlet or Spring framework on the server to process file upload; 3. Block upload technology, which divides large files into smaller data blocks for uploading, and uses the verification mechanism on the client and server to ensure data integrity; 4. Asynchronous non-blocking I / O technology, which uses non-blocking I / O technology (such as Java NIO) to achieve high-performance file upload.

[0004] However, the above file upload method has the following problems: the use of synchronous calls between all system modules will lengthen the call chain and increase the request delay. The client will be waiting from uploading to response, and the business gateway will also wait, which will weaken the concurrent processing capability and fail to achieve the best resource utilization effect; the memory usage problem, when processing large files, the traditional file upload method is prone to server memory overflow; if the file service uses the traditional http protocol for file processing, when uploading large files or the file upload concurrency is large, it is easy to have memory overflow or blockage; when using the block upload method, both the client and the server need additional logic to handle block, merge, and error correction, which greatly increases the complexity; using non-blocking I / O technology (such as Java NIO) to achieve high-performance file upload requires a deep understanding of the underlying I / O model, which is difficult.

[0005] Therefore, it is particularly important to develop a technology that can efficiently handle large file uploads. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method, device, equipment and medium for uploading and processing large files.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] In a first aspect, this embodiment provides a method for uploading a large file, comprising the following steps:

[0009] Receive files uploaded by the client in batches, and write the files received in a single batch to the disk at the same time until the entire large file is written to obtain a temporary file;

[0010] Determine whether there is a file end mark in the temporary file;

[0011] If the temporary file has a file end mark, the temporary file is asynchronously uploaded to the cloud storage to obtain the storage file.

[0012] A further technical solution is: before the step of asynchronously uploading the temporary file to the cloud storage to obtain the storage file, it also includes: sending information to the client to inform that the temporary storage of the large file has been completed.

[0013] Its further technical solution is: after the step of determining whether there is a file end mark for the temporarily stored file, it also includes: if the temporarily stored file does not have a file end mark, returning to execute receiving files uploaded by the client in batches, and simultaneously writing the single received file to the disk until the entire large file is written to obtain a temporarily stored file.

[0014] Its further technical solution is: after the step of asynchronously uploading the temporary file to the cloud storage to obtain the storage file, it also includes: obtaining the upload result information sent by the cloud storage and asynchronously sending it to the client.

[0015] In a second aspect, this embodiment provides a large file upload processing device, including: a receiving and writing unit, a judging unit and an uploading unit;

[0016] The receiving and writing unit is used to receive files uploaded by the client in batches, and write the files received in a single batch to the disk at the same time, until the entire large file is written, so as to obtain a temporary file;

[0017] The judging unit is used to judge whether there is a file end mark in the temporarily stored file;

[0018] The uploading unit is used to asynchronously upload the temporary file to the cloud storage if there is a file end mark in the temporary file to obtain the storage file.

[0019] A further technical solution is as follows: the device further comprises: a sending unit, which is used to send information to the client to inform that the temporary storage of the large file has been completed.

[0020] Its further technical solution is: the device also includes: a returning unit, which is used to return to execute the receiving of files uploaded by the client in batches if there is no file end mark in the temporary file, and simultaneously write the single received file to the disk until the entire large file is written to obtain the temporary file.

[0021] A further technical solution is as follows: the device further comprises: an acquisition and sending unit, which is used to acquire the upload result information sent by the cloud storage and asynchronously send it to the client.

[0022] In a third aspect, this embodiment provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the large file upload processing method as described above when executing the computer program.

[0023] In a fourth aspect, this embodiment provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the large file upload processing method as described above can be implemented.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: by receiving files uploaded by the client in batches, and writing the single received file to the disk at the same time, until the entire large file is written, a temporary file is obtained, and it is determined whether the temporary file has a file end mark. If the temporary file has a file end mark, the temporary file is asynchronously uploaded to the cloud storage to obtain a storage file; that is, by receiving files in batches and writing them to the disk immediately, instead of storing the entire large file in the memory first, this method significantly reduces the usage of server memory, and can effectively avoid the risk of memory overflow caused by insufficient memory. After the client uploads the file, the file will be temporarily stored on the disk and then asynchronously uploaded to the cloud storage, which can reduce the client waiting time, thereby improving resource utilization, reducing hardware configuration and saving investment.

[0025] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 A flowchart of a large file upload processing method provided by an embodiment of the present invention;

[0028] Figure 2A schematic block diagram of a large file upload processing device provided by an embodiment of the present invention;

[0029] Figure 3 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0032] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0033] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] See also Figure 1 In the specific embodiment shown, the present invention discloses a method for uploading a large file, comprising the following steps:

[0035] S1, receiving files uploaded by the client in batches, and writing the files received in a single batch to the disk at the same time, until the entire large file is written, so as to obtain a temporary file;

[0036] Specifically, the file system and business system are decoupled by adjusting the service architecture. The file service does not need to worry about the specific business-side logic, and the business service does not need to understand the implementation logic of the file service. The file service provides general file storage capabilities to the outside world, and different business systems can reuse the file system with a low threshold.

[0037] More specifically, the client divides the large file into multiple smaller data blocks (or "shards"). The size of each shard can be set according to the actual situation, such as 1MB, 2MB, etc. The client uploads these shards to the server in sequence or concurrently. The server receives the shards uploaded by the client and immediately writes them to the temporary storage area of ​​the disk. The server assigns a unique identifier (such as file name + shard sequence number) to each shard for subsequent merging. The server can maintain a record file to track which shards have been uploaded and which have not been uploaded. During the upload process or after the upload is completed, the server can perform integrity verification on each shard, such as calculating a checksum (such as MD5, SHA-1, etc.) to ensure that the shard is not damaged. If a shard is found to be damaged, the server can ask the client to re-upload the shard. When all shards are successfully uploaded and verified, the server merges the shards into a complete file in sequence. The merged file is stored on the server's disk as a temporary file.

[0038] In addition, by splitting large files into multiple small pieces and writing them to disk one by one, the server does not need to load the entire file into memory, thereby reducing memory usage. Multi-piece upload allows the client to re-upload only the erroneous piece instead of the entire file when an error occurs during the upload process, thereby improving the stability of the upload. For clients that support concurrent uploads, multiple pieces can be uploaded at the same time, thereby making full use of network bandwidth and increasing upload speed. After splitting the file into multiple pieces, the server can more easily manage and process these pieces, such as compression, encryption, backup, etc. The existence of temporary files also makes subsequent file processing (such as asynchronous upload to cloud storage) more flexible and efficient. By reducing the requirements for server memory and processor performance, this technology helps to reduce hardware costs, making it affordable for more users to upload and store large files.

[0039] S2, determine whether there is a file end mark in the temporary file;

[0040] Specifically, EOF (End Of File) is the sign of the end of a file. In programming, it is usually defined as a special value, such as -1 (in C language), which indicates the end of the file content, not the physical end of the file. Presetting EOF at the end of a large file means adding this special value or character sequence at the end of the file content to clearly indicate the end of the file. During the file writing process, when all content has been successfully written to the server's disk, the program will add the EOF mark at the end of the file. This can be achieved by writing a specific character (such as a control character in ASCII code, but usually not a line feed or carriage return) or by writing a specific value (such as -1) to the file. In binary files, EOF is usually represented as -1 (that is, the minimum value of an integer), because each byte of a binary file can represent a value between 0 and 255, and -1 is a special value in the binary complement representation and will not be confused with any byte in the file content. In subsequent file processing, such as reading files, verifying file integrity, or merging files, the program will check whether there is an EOF mark at the end of the file. If the EOF marker is found at the end of the file, the program can be sure that the file has been written completely to disk and has not been interrupted by network errors, disk failure, or other reasons.

[0041] In addition, by presetting and checking the EOF marker, you can more reliably determine whether the file has been completely written to the disk, which helps avoid errors and data loss during file transfer or storage. When reading a file, the program can determine the end position of the file based on the EOF marker, thereby avoiding reading unnecessary data or causing read errors, which helps improve the performance and efficiency of file reading. The EOF marker can be used as part of file integrity verification. By checking whether there is an EOF marker at the end of the file, the program can verify whether the file has not been tampered with or damaged. Presetting and checking the EOF marker can simplify the logic of file processing. The program does not need to track the actual size or position of the file. It only needs to check the EOF marker to determine the end position of the file.

[0042] In one embodiment, before the step of asynchronously uploading the temporary file to the cloud storage to obtain the storage file, the step further includes: sending information to the client to inform that the temporary storage of the large file has been completed.

[0043] Specifically, a notification mechanism is designed on the server side, which is responsible for sending a notification to the client after the temporary storage of the file is completed. The notification mechanism can be implemented in a variety of ways, such as HTTP-based POST request, WebSocket real-time communication, or asynchronous notification methods such as email and SMS. The server side needs to maintain a status tracking system to record the upload progress and temporary storage status of each large file. When all the fragments of a large file are successfully uploaded and merged into a temporary storage file, the status tracking system will mark it as "temporary storage completed". Once the file is marked as "temporary storage completed", the notification mechanism will generate a notification message containing information such as file identification and temporary storage path, and then the notification message will be sent to the client through a preset communication channel (such as HTTP request, WebSocket, etc.). The client needs to implement a module for receiving and processing notification messages. When receiving the notification message sent by the server, the module will parse the message content and perform corresponding operations as needed, such as displaying notifications, updating interface status, etc.

[0044] In addition, by notifying the client in time that the file has been temporarily stored, the client can immediately know the status of the file, thus avoiding unnecessary waiting and anxiety, which helps to improve the overall user experience and enhance the user's trust and satisfaction with the system. The implementation of the notification mechanism enhances the interactivity between the server and the client. The client can understand the upload progress and status of the file in real time, and the server can also obtain the client's feedback and response in time. In addition, through the asynchronous notification method, the server can send notifications to the client without blocking the main thread, which helps to optimize the resource utilization of the server and improve the concurrent processing capability and response speed of the system. In addition, the design of the notification mechanism supports multiple communication methods, which allows the system to select the appropriate communication channel according to actual needs. For example, in scenarios where real-time communication is required, real-time communication technologies such as WebSocket can be selected; in scenarios where real-time requirements are not high, asynchronous notification methods such as HTTP requests or emails can be selected.

[0045] In one embodiment, after the step of determining whether there is a file end mark for the temporary file, it also includes: if the temporary file does not have a file end mark, returning to execute receiving files uploaded by the client in batches, and simultaneously writing the single received file to the disk until the entire large file is written to obtain a temporary file.

[0046] Specifically, after the file is written to the disk and a temporary file is generated, the system will first check whether there is an end-of-file mark (EOF) in the file. The checking process may involve reading the last byte or a group of bytes of the file and determining whether they match the preset end-of-file mark. If the check finds that the temporary file does not have an end-of-file mark, the system will consider that the file upload is incomplete or there is an error. At this time, the system will trigger a conditional judgment and return to execute the previous file upload process based on the judgment result. The system will re-enter the stage of receiving files uploaded by the client in batches. The client will continue to upload the remaining part of the file (if part of the file has been uploaded before), or restart uploading the entire file (if the previous upload was completely interrupted). The server will write the received file data to the disk again in batches until the entire file is fully uploaded and written to the disk. The above process may form a loop until the system successfully detects that there is an end-of-file mark in the temporary file. The termination condition of the loop is that the file is successfully uploaded and written to the disk, and the system can detect the end-of-file mark.

[0047] In addition, by checking the end-of-file mark and re-uploading the incomplete file, the system can ensure that each uploaded file is complete and undamaged, which helps to avoid upload failure or file damage caused by network problems, disk failures, etc. during the transmission process. For users, they do not need to worry about errors or interruptions that may occur during the file upload process. The system will automatically handle these problems and re-upload the file when necessary, thereby ensuring that users can smoothly upload and store the large files they need. This mechanism enables the system to cope with various abnormal situations, such as network instability and insufficient disk space. In addition, by re-uploading the incomplete file strategy, the system can maintain its stability and reliability and avoid system crashes or data loss caused by abnormal situations. In addition, by receiving and writing files in batches, the system can more effectively utilize server resources, which helps to reduce the server's memory and disk usage and improve the overall performance and resource utilization of the system.

[0048] S3: If the temporary file has a file end mark, the temporary file is asynchronously uploaded to the cloud storage to obtain the storage file.

[0049] Specifically, in the file upload process, the system will first check whether there is an end-of-file mark (EOF) in the temporary file. This step usually involves reading the end of the file to verify whether it conforms to the preset end-of-file mark format. Once it is confirmed that the temporary file has an end-of-file mark, the system will assume that the file has been completely uploaded to the local temporary storage area. Subsequently, the system will trigger the asynchronous upload mechanism to upload the temporary file to the cloud storage service. Asynchronous upload means that the system will not wait for the upload to complete before continuing to perform other tasks, thereby improving the overall processing efficiency. During the asynchronous upload process, the system needs to interact with the cloud storage service, including authentication, sending file upload requests, and monitoring the upload progress. The cloud storage service will receive and process the upload request, store the file in the specified cloud storage area, and return the upload result information. After the upload is completed, the system will obtain key information such as the URL or path of the stored file from the cloud storage service. This information will be used for subsequent file access, downloading, or sharing operations.

[0050] In addition, the asynchronous upload mechanism enables the system to process multiple file upload tasks at the same time, thereby improving upload efficiency, which is particularly important for systems that need to handle large amounts of file uploads. During the asynchronous upload process, the system can continue to perform other tasks without waiting for the upload to complete, which helps optimize server resource utilization and avoid idle or wasted resources. In addition, by detecting the end-of-file marker, the system can ensure that only complete and undamaged files are uploaded to the cloud storage, which helps avoid uploading incomplete or damaged files, thereby improving the reliability of the system. Users can perform other operations without waiting for the file upload to complete. At the same time, the system can provide upload progress feedback and upload result notifications to further enhance the user experience. The stored files are saved in the cloud storage service for subsequent file management and access. Users can access their files anytime and anywhere through the Internet to achieve file sharing and collaboration.

[0051] In one embodiment, after the step of asynchronously uploading the temporary file to the cloud storage to obtain the storage file, the method further includes: obtaining the upload result information sent by the cloud storage and asynchronously sending it to the client.

[0052] Specifically, after the server completes the preparation of temporary storage files, it will asynchronously upload them to the cloud storage service. Asynchronous upload means that the server will not wait for the upload to complete before continuing to perform other tasks, thereby improving the overall processing efficiency. After the cloud storage service completes the file upload, it will send a notification or response containing the upload result information to the server. This notification or response usually contains key information such as the storage location of the file, whether the upload is successful, and the error code (if any). After receiving the upload result information sent by the cloud storage, the server will process it. If the upload is successful, the server will record the storage location of the file and prepare to notify the client of this information. If the upload fails, the server may perform corresponding error processing according to the error code, such as retrying the upload, recording logs, etc. After processing the upload result information, the server will notify the client of the result of the file upload asynchronously. Asynchronous notification can be adopted in a variety of ways, such as WebSocket real-time communication, HTTP long polling, email notification, SMS notification, etc. The content of the notification usually includes key information such as whether the file is successfully uploaded and the storage location.

[0053] In addition, through asynchronous uploading and asynchronous notification, the server can handle file upload tasks more efficiently, which helps to reduce the response time of the system and improve the user experience. The asynchronous processing mode enables the server to handle multiple file upload tasks at the same time, which helps to enhance the scalability of the system and enable it to cope with larger concurrency and data volumes. In addition, the client can obtain the result information of the file upload in real time, which helps to improve user satisfaction and trust, making users more willing to use the system. In addition, through asynchronous processing, the server can perform other tasks while waiting for the response of the cloud storage service, which helps to achieve efficient use of resources and improve the overall performance of the system.

[0054] The present invention receives files uploaded by the client in batches, and writes the single received file to the disk at the same time until the entire large file is written to obtain a temporary file, and determines whether the temporary file has a file end mark. If the temporary file has a file end mark, the temporary file is asynchronously uploaded to the cloud storage to obtain a storage file; that is, by receiving files in batches and writing them to the disk immediately, rather than storing the entire large file in the memory first, this method significantly reduces the usage of server memory and can effectively avoid the risk of memory overflow caused by insufficient memory. After the client uploads the file, the file will be temporarily stored on the disk and then asynchronously uploaded to the cloud storage, which can reduce the client waiting time, thereby improving resource utilization, reducing hardware configuration and saving investment.

[0055] See also Figure 2 As shown, the present invention also discloses a large file upload processing device, comprising: a receiving and writing unit 10, a judging unit 20 and an uploading unit 30;

[0056] The receiving and writing unit 10 is used to receive files uploaded by the client in batches, and write the files received in a single batch to the disk at the same time, until the entire large file is written, so as to obtain a temporary file;

[0057] The judging unit 20 is used to judge whether there is a file end mark in the temporarily stored file;

[0058] The uploading unit 30 is used to asynchronously upload the temporary file to the cloud storage if there is a file end mark in the temporary file to obtain a storage file.

[0059] In one embodiment, the device further includes: a sending unit, configured to send information to the client to inform that temporary storage of the large file has been completed.

[0060] In one embodiment, the device also includes: a returning unit, which is used to return to the execution of receiving files uploaded by the client in batches if there is no file end mark in the temporary file, and simultaneously write the single received file to the disk until the entire large file is written to obtain a temporary file.

[0061] In one embodiment, the device further comprises: an acquisition and sending unit, configured to acquire the upload result information sent by the cloud storage and asynchronously send the information to the client.

[0062] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the above-mentioned large file upload processing device and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0063] The above-mentioned large file upload processing device can be implemented in the form of a computer program, and the computer program can be used in Figure 3 Runs on the computer device shown.

[0064] See also Figure 3 , Figure 3 is a schematic block diagram of a computer device provided in an embodiment of the present application; the computer device 500 may be a terminal or a server, wherein the terminal may be an electronic device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster composed of multiple servers.

[0065] See also Figure 3 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0066] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a large file upload processing method.

[0067] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0068] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a large file upload processing method.

[0069] The network interface 505 is used to communicate with other devices over the network. Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0070] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0071] Receive files uploaded by the client in batches, and write the single received files to the disk at the same time until the entire large file is written to obtain a temporary file; determine whether the temporary file has a file end mark; if the temporary file has a file end mark, asynchronously upload the temporary file to the cloud storage to obtain a storage file.

[0072] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0073] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0074] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the above-mentioned large file upload processing method can be implemented. The storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the above-mentioned method can be implemented. The program instructions include the following steps:

[0075] Receive files uploaded by the client in batches, and write the single received files to the disk at the same time until the entire large file is written to obtain a temporary file; determine whether the temporary file has a file end mark; if the temporary file has a file end mark, asynchronously upload the temporary file to the cloud storage to obtain a storage file.

[0076] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0077] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0078] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or skipped.

[0079] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0080] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0081] The above embodiments are preferred implementation schemes of the present invention. In addition, the present invention may also be implemented in other ways. Any obvious replacement without departing from the concept of the present technical solution is within the protection scope of the present invention.

Claims

1. A large file upload processing method, characterized in that: The following steps are involved: Receive files uploaded by the client in batches, and write the files received in a single batch to the disk at the same time until the entire large file is written to obtain a temporary file; Determine whether there is a file end mark in the temporary file; If the temporary file has a file end mark, the temporary file is asynchronously uploaded to the cloud storage to obtain the storage file.

2. The large file upload processing method according to claim 1, characterized in that: Before the step of asynchronously uploading the temporary file to the cloud storage to obtain the storage file, the method further includes: sending information to the client to inform that the temporary storage of the large file has been completed.

3. The method for uploading a large file according to claim 1, characterized in that: After the step of determining whether there is a file end mark for the temporary file, the method further includes: if there is no file end mark for the temporary file, returning to execute receiving files uploaded by the client in batches, and writing the single received files to the disk at the same time, until the entire large file is written, so as to obtain the temporary file.

4. The method for uploading a large file according to claim 1, characterized in that: After the step of asynchronously uploading the temporary file to the cloud storage to obtain the stored file, the method further includes: obtaining the upload result information sent by the cloud storage and asynchronously sending it to the client.

5. A large file upload processing device, characterized in that: include: Receiving writing unit, judging unit and uploading unit; The receiving and writing unit is used to receive files uploaded by the client in batches, and write the files received in a single batch to the disk at the same time, until the entire large file is written, so as to obtain a temporary file; The judging unit is used to judge whether there is a file end mark in the temporarily stored file; The uploading unit is used to asynchronously upload the temporary file to the cloud storage if there is a file end mark in the temporary file to obtain the storage file.

6. The large file upload processing device according to claim 5, characterized in that: The device also includes: a sending unit, which is used to send information to the client to inform that the temporary storage of the large file has been completed.

7. The large file upload processing device according to claim 5, characterized in that: The device also includes: a returning unit, which is used to return to receiving files uploaded by the client in batches if there is no file end mark in the temporary file, and write the single received file to the disk at the same time until the entire large file is written to obtain the temporary file.

8. The large file upload processing device according to claim 5, characterized in that: The device also includes: an acquisition and sending unit, which is used to acquire the upload result information sent by the cloud storage and asynchronously send it to the client.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the large file upload processing method according to any one of claims 1 to 4 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the large file upload processing method according to any one of claims 1 to 4 is implemented.

Images