A file uploading method, an electronic device, and a storage medium

Abstract

The application provides a file uploading method, an electronic device and a storage medium, relates to the technical field of computers, and solves the problem that different users may cover the same name file, and the OSS is not directly exposed to the service, so that the space is prevented from being abused and bad content is prevented from being spread through exposure of the write credentials of the OSS in any way; in the case that the write credentials of the OSS are leaked, the whole uploading operation can still record the operation person and relevant tracking information to all uploaded files, and the overall uploading and downloading performance is considered, and compared with the variant scheme mentioned in the prior art scheme, the self-service is developed to the same level as the mature middleware, so that the development difficulty and the development cost are effectively controlled.

Description

Technical Field

[0001] This invention relates to the field of computer storage, specifically to a file upload method, an electronic device, and a storage medium. Background Technology

[0002] User file uploads are a common function, and file storage is currently widely used for file storage. Object storage is commonly used in scenarios where user uploads are readable by anonymous users. However, current technologies using file storage have their own problems, as follows:

[0003] Option 1: As Figure 1 and Figure 2 As shown, this approach cannot control user reading and writing, directly exposing OSS credentials to Ehu Second-Level, which makes it easy for end users to break the rules and abuse them to spread illegal information. Furthermore, it is impossible to trace the source of the files, making it difficult to hold those who upload illegal information accountable.

[0004] Another point is that because OSS is exposed to the public network, once the OSS write credentials are leaked through other means (such as internal human leakage) and used to upload files, the application will not know the information of the person uploading the files; it may even be possible to use overwrite to replace the original files with illegal files, which has serious consequences; the workaround is that even if OSS is not directly exposed on the network, the problem still exists because the intermediate reverse proxy layer / SLB layer (Server Load Balancing) has no business processing capabilities and is only a proxy pass-through problem.

[0005] Option 2: Figure 3 and Figure 4 As shown, in this solution, OSS only exposes anonymous reads to the outside world. User uploads must go through a backend service proxy. End-user uploads can be managed and their behavior recorded by the backend service. However, there are still drawbacks because OSS is exposed to the public network, and the problem of leaking OSS write credentials mentioned in the previous method still exists. Therefore, a variant solution might be proposed that doesn't directly expose OSS; instead, it uses its own backend service / middleware such as nginx as a file read channel, shielding OSS read and write operations together with the backend service. Figure 4 In this way, OSS is no longer directly exposed to the public network, but because the file reading and writing services need to be distributed to different services / middleware, multiple layers of network processing are required. In addition, the self-developed backend service 1 serves as the writing channel, and all the byte streams of uploaded files need to pass through this service. This service has extremely high requirements for performance and concurrency, which increases development costs.

[0006] Option 3: As Figure 5 and Figure 6As shown, commercial OSS systems like Qiniu Cloud use a backend service to obtain temporary upload credentials for users. Users then use these temporary credentials to upload files that meet the credentials' requirements. The backend service manages and records the behavior of users who obtain upload credentials, and combines this with information such as the filename and time when the upload credentials were generated to track the actual uploader in the OSS. This approach has several drawbacks: firstly, it requires the use of a commercial OSS system that provides the corresponding functionality, which is costly; secondly, using a commercial OSS system requires the OSS system to be exposed to the public network, and the problems caused by OSS credential leakage in method 1 also exist in this method; and thirdly, the temporary credentials use the real filename for upload verification, which can lead to file overwriting or the latter being unable to be written when two users upload files with the same name. From a business perspective, it is unreasonable for a later user to be unable to upload because another user uses the same filename.

[0007] Therefore, a relatively safe way to use OSS is needed to solve the problems of lack of control, difficulty and cost of self-developed services in the existing solutions. Summary of the Invention

[0008] To address the aforementioned technical shortcomings, the purpose of this invention is to provide a file upload method that solves the problems of lack of control, difficulty and cost of self-developed services in existing solutions. At the same time, it introduces a new file name processing mechanism to solve the problem of different users uploading files with the same name that may be overwritten. Furthermore, OSS does not directly expose the service, thus preventing the space from being abused and spreading harmful content after the write credentials of OSS are exposed in any way.

[0009] A file upload method, characterized by comprising the following steps:

[0010] S1. Read the file information of the file to be uploaded and send a request to obtain a temporary certificate;

[0011] S2. Send a file upload request based on the obtained temporary credentials;

[0012] S3. Read the file information based on the temporary credential in the file upload request and compare it with the file information in the file upload request. If the comparison result is inconsistent, proceed to S4; if the comparison result is consistent, proceed to S5.

[0013] S4. Returns the file upload request;

[0014] S5. Generate an identification code and replace the filename in the file upload request with the identification code;

[0015] S6. Send the file upload request to the cloud storage service to complete the upload.

[0016] S1 is as follows: Based on the file upload request, the user's writing terminal selects the file and reads the file information, and sends a request to the backend service to obtain a temporary certificate with the file information in order to obtain the temporary certificate. The backend service authenticates the request to obtain the temporary certificate. If the authentication fails, the backend service returns an authentication failure message to the user's writing terminal. If the authentication succeeds, the backend service obtains the temporary certificate from the cloud storage service.

[0017] The backend service encodes the acquired temporary credentials, uses the encoding as the key, and stores the file information and user information from the temporary credentials acquisition request as the value in Database 1.

[0018] The backend service sends the user's identity information, file information, and temporary credentials to the file upload log service, and returns the temporary credentials to the user's writing terminal.

[0019] S3 specifically refers to:

[0020] The user terminal sends a file upload request to the upload server. The upload server and its upload module obtain temporary credentials and file information from the file upload request. The temporary credentials processing module encodes the temporary credentials. The authentication function uses the encoding of the temporary credentials as the key to read the file information from database one and compares the file information with the file information obtained from the file upload request.

[0021] If the comparison results are inconsistent, a message indicating that the temporary credential is incorrect will be returned to the user's writing terminal.

[0022] If the comparison results are consistent, an identification code is generated through the authentication function. The authentication function uses the identification code as the key and the original file name and file information as the value in a hash structure to store it in database two. The authentication function updates the entry written to database two by the backend service with the credential code as the key and adds the replaced file name.

[0023] The upload module replaces the filename in the file upload request with an identification code.

[0024] After uploading in S6, the following steps are also included:

[0025] S7. The file upload log service periodically reads entry information from database two using the credential code as the key, based on the received credential information.

[0026] S8. Determine if the entry already has a replaced filename information;

[0027] S9. If there is no replacement file name information, the following judgment is made: if there is no voucher entry, the timed processing ends; if there is a voucher entry and it has expired, the record is deleted; if there is a voucher entry and it has not expired, the next entry is read and the process jumps to S8.

[0028] If there is information about replacing filenames, record the file and its uploader information in the log.

[0029] It also includes file access methods, the specific steps of which are as follows:

[0030] S10, Receive file access request;

[0031] S11. Obtain the filename in the access request, and obtain the entry named key based on the file;

[0032] S12. If the retrieved entry does not exist, return an access failure message.

[0033] S13. If the obtained entry exists, obtain the original uploaded file name from the entry;

[0034] S14. Set the original uploaded file name as the downloaded file in the request header;

[0035] S15. Forward the access request configured to the cloud storage service.

[0036] S16. Obtain the file to be accessed.

[0037] S12 specifically refers to:

[0038] The object name conversion module retrieves the filename from the file access request information sent by the anonymous reading terminal and uses the filename as the key to retrieve the entry from database two.

[0039] An electronic device, comprising:

[0040] At least one processor, and

[0041] Memory, communicatively connected to at least one processor, is used to store executable instructions;

[0042] The processor can execute executable instructions stored in the memory to implement the file upload method.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a file upload method.

[0044] The beneficial effects of this invention are as follows: This invention solves the problem that files with the same name uploaded by different users may be overwritten, and OSS does not directly expose the service, thus preventing the space from being abused and spreading harmful content after the write credentials of OSS are exposed in any way; even if the write credentials of OSS are leaked, the entire upload operation can still record the operator and related tracking information for all uploaded files, while taking into account the overall upload and download performance. Compared with the variant solutions mentioned in the existing solutions that require developing proprietary services to the same level as mature middleware, the development difficulty and development cost are effectively controlled. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is the logic diagram for the existing Scheme 1.

[0047] Figure 2 This is the logic diagram of the modified version of the existing solution.

[0048] Figure 3 This is the logic diagram for the existing Scheme 2.

[0049] Figure 4 This is the logic diagram of the modified version of the existing scheme 2.

[0050] Figure 5 This is the logic diagram for the existing Scheme 1.

[0051] Figure 6 This is the logic diagram of the modified version of the existing scheme 2.

[0052] Figure 7 This is a logic diagram of the file upload method of the present invention.

[0053] Figure 8 This is a flowchart of the file upload method of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Example 1

[0056] like Figures 1 to 8 As shown, the present invention provides a file upload method, including the following steps:

[0057] S1. Read the file information of the file to be uploaded and send a request to obtain a temporary certificate;

[0058] Specifically, based on the file upload request, the user's writing terminal selects the file and reads the file information, and sends a request with the file information to the backend service to obtain a temporary certificate in order to obtain the temporary certificate. The backend service authenticates the request to obtain the temporary certificate. If the authentication fails, the backend service returns an authentication failure message to the user's writing terminal. If the authentication passes, the backend service obtains the temporary certificate from the cloud storage service.

[0059] The backend service encodes the obtained temporary credentials, uses the encoding as the key, and stores the file information and user information from the temporary credential acquisition request as the value in Database 1.

[0060] The backend service sends the user's identity information, file information, and temporary credentials to the file upload log service, and returns the temporary credentials to the user's writing terminal.

[0061] S2. Send a file upload request based on the obtained temporary credentials (the file upload request shall include at least the temporary credentials and file information);

[0062] S3. Based on the temporary certificate in the file upload request, read and obtain the file information corresponding to the temporary certificate code (the file information in the temporary certificate request) and compare it with the file information in the file upload request. If the comparison result is inconsistent, proceed to S4; if the comparison result is consistent, proceed to S5.

[0063] The request to obtain a temporary credential includes file information, and the file upload request also includes file information. By comparing the two sets of file information, it is possible to prevent someone from uploading other file content using a leaked temporary credential.

[0064] S4. Returns the file upload request;

[0065] S5. Generate an identification code and replace the filename in the file upload request with the identification code;

[0066] S3-S5 are as follows: The user terminal sends a file upload request to the upload server. The upload server and its upload module obtain temporary credentials and file information from the file upload request. The temporary credentials are encoded by the temporary credentials processing module. The authentication function reads the file information from database one using the encoding of the temporary credentials as the key and compares the file information with the file information obtained from the file upload request.

[0067] If the comparison results are inconsistent, a message indicating that the temporary credential is incorrect will be returned to the user's writing terminal.

[0068] If the comparison results are consistent, an identification code is generated through the authentication function. The authentication function uses the identification code as the key and the original file name and file information as the value in a hash structure to store it in database two. The authentication function updates the entry written to database two by the backend service with the credential code as the key and adds the replaced file name.

[0069] The upload module replaces the filename in the file upload request with an identification code;

[0070] S6. Send the file upload request to the cloud storage service to complete the upload;

[0071] S7. The file upload log service periodically reads entry information from database two using the credential code as the key, based on the received credential information.

[0072] S8. Determine if the entry already has a replaced filename information;

[0073] S9. If there is no replacement file name information, the following judgment is made: if there is no voucher entry, the timed processing ends; if there is a voucher entry and it has expired, the record is deleted; if there is a voucher entry and it has not expired, the next entry is read and the process jumps to S8.

[0074] If there is information about replacing filenames, then log the file and its uploader information.

[0075] The specific steps described above are as follows:

[0076] 1. The user writes "User is logged in" on the terminal;

[0077] 2. The user selects the file to upload in the terminal;

[0078] 3. The user writes to the terminal to read the file information to be uploaded (such as: original file name, file size, type, etc.);

[0079] 4. The user writes the terminal to send the file information to be uploaded to the backend service to obtain a temporary credential;

[0080] 5. Authentication of backend services' requests for obtaining temporary credentials;

[0081] 6. If the backend service authentication fails in step 5, proceed to step 7;

[0082] 7. The backend service returns an authentication failure message to the user's terminal.

[0083] 8. If authentication in step 5 is successful, proceed to step 9;

[0084] 9. The backend service obtains a temporary certificate from the cloud storage service (OSS) (on the OSS side, this temporary certificate is unrelated to the file);

[0085] 10. The backend service encodes the temporary credential information (using MD5 as an example) as the key, and stores the file information and user identity information carried in step 3 as the value in database one (i.e., Figure 7 (in Redis);

[0086] 11. The backend service sends user and file information, along with temporary credentials, to the file upload log service;

[0087] 12. The backend service returns the temporary credentials to the user's writing terminal;

[0088] 13. The user writes a terminal to upload a file to OSS using temporary credentials;

[0089] 14. The user writes a file upload request in the terminal and sends it to the upload server (nginx);

[0090] 15. The upload server (nginx) and upload module (nginx_upload_module) obtain temporary credential information and file information from the file upload request;

[0091] 16. Enter the pre-written temporary credential processing module for temporary credentials and document authentication;

[0092] The temporary voucher processing module of this invention uses Lua functions;

[0093] 17. The temporary voucher processing module encodes the temporary voucher information;

[0094] 18. The authentication function reads file information from Redis using the temporary credential's encoding as the key;

[0095] 19. The authentication function compares the cached file information read with the file information obtained by nginx_upload_module;

[0096] 20. If the comparison results in step 19 are inconsistent, proceed to step 21;

[0097] 21. Nginx no longer forwards requests to OSS, but directly returns information that the temporary credentials do not match the file to the user's terminal.

[0098] 22. If the comparison results in step 19 are consistent, proceed to step 23;

[0099] 23. The authentication function generates an identification code, namely UUID (or other unique string that serves the same purpose; UUID will be used as the reference below).

[0100] 24. The authentication function uses the UUID as the key and a hash structure to write the original filename and file information as the value into data two (Redis);

[0101] To avoid confusion, the manual uses the terms "Database 1" and "Database 2," but these do not refer to two completely different databases. Therefore, a separate document is attached to the manual. Figure 8 There is only one Redis instance in the database;

[0102] 25. The authentication function updates the entry written to database 2 by the backend service using the credential code as the key, and adds the replaced filename;

[0103] 26. Replace the filename with the above identifier (uuid) in nginx_upload_module;

[0104] 27. Forward the file upload request to OSS;

[0105] 28. Upload complete;

[0106] 29. Returns the URL for user-written terminal file access (the URL is the access URL of the UUID filename);

[0107] 30. The file upload log service periodically reads entry information from Redis using the received credential code as the key;

[0108] 31. Determine if the entry already contains the filename information for the replacement;

[0109] 32. If the filename information from step 31 has not yet been replaced, proceed to step 33.

[0110] 33. Determine if there are any unexpired and unrecorded voucher entries. (Step 34)

[0111] 34. If there are no more entries in step 33, then this timed processing ends;

[0112] 35. If there are still entries in step 33, delete the record if the entry has expired, or read the next entry and jump to step 31 if the entry has not expired.

[0113] 36. If the result of step 31 already contains the filename information after replacement, proceed to step 37;

[0114] 37. Record file information and the uploader's information in the log;

[0115] 38. Proceed to step 33;

[0116] This invention also provides an anonymous file reading method based on the above file upload method, the specific steps of which are as follows:

[0117] The steps for reading a file are as follows:

[0118] S10, Receive file access request;

[0119] S11. Obtain the filename in the access request, and obtain the entry named key based on the file;

[0120] Specifically, the object name conversion module obtains the file name from the file access request information sent by the anonymous reading terminal and retrieves the entry from database two using the file name as the key.

[0121] S12. If the retrieved entry does not exist, return an access failure message.

[0122] S13. If the obtained entry exists, obtain the original uploaded file name from the entry;

[0123] S14. Set the original uploaded file name as the downloaded file in the request header;

[0124] S15. Forward the access request configured to the cloud storage service.

[0125] S16. Obtain the file to be accessed;

[0126] 1. An anonymous terminal submits an access request (filename is named uuid);

[0127] 2. The access request reaches nginx;

[0128] 3. Download the renamed object name conversion module to obtain request information;

[0129] The object name conversion module of this invention also uses Lua functions;

[0130] 4. Download the renamed object name conversion module from database 2 and read the entry in the read area with the requested file name as the key;

[0131] 5. If the entry obtained in step 4 does not exist, proceed to step 6;

[0132] 6. ngxin directly returns that the file does not exist;

[0133] 7. If the entry obtained in step 4 exists, proceed to step 8;

[0134] 8. Download the renamed Lua function to obtain the original uploaded filename from the entry;

[0135] 9. Set the original uploaded filename as the downloaded filename in the request header;

[0136] 10. Forward the request to OSS;

[0137] 11. OSS returns a file stream;

[0138] 12. Nginx passes the returned file stream to the anonymous reading terminal;

[0139] In the steps above, the filename carried by the anonymous user is the globally unique random key value generated during the upload. By adding Lua functions to the reverse proxy, the relationship between the globally unique random key value stored during the upload and the original filename is read from the cache, and the filename is replaced with the original filename during the download, thus improving usability.

[0140] Example 2

[0141] The present invention also provides an electronic device capable of implementing the above-described embodiments, comprising:

[0142] At least one processor, and

[0143] Memory, communicatively connected to at least one processor, is used to store executable instructions;

[0144] The processor is capable of executing executable instructions stored in the memory to implement the methods mentioned in the above embodiments;

[0145] The communication connection can be one of the following methods: bus, wireless communication, etc.

[0146] Example 3

[0147] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the file upload method mentioned in Embodiment 1 above.

[0148] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A file upload method, characterized in that, Includes the following steps: S1. Read the file information of the file to be uploaded and send a request to obtain a temporary certificate; S2. Send a file upload request based on the obtained temporary credentials; S3. Read the file information based on the temporary credential in the file upload request and compare it with the file information in the file upload request. If the comparison result is inconsistent, proceed to S4; if the comparison result is consistent, proceed to S5. S4. Returns the file upload request; S5. Generate an identification code and replace the filename in the file upload request with the identification code; S6. Send the file upload request to the cloud storage service to complete the upload; S1 specifically refers to: based on the file upload request, the user writing terminal selects the file and reads the file information, and sends a request to the backend service to obtain a temporary certificate with the file information, so as to obtain the temporary certificate. The backend service authenticates the request to obtain the temporary certificate. If the authentication fails, the backend service returns the authentication failure information to the user writing terminal. If the authentication passes, the backend service obtains the temporary certificate from the cloud storage service. The backend service encodes the acquired temporary credentials, uses the encoding as the key, and stores the file information and user information from the temporary credentials acquisition request as the value in Database 1. The backend service sends the user's identity information, file information, and temporary credentials to the file upload log service, and returns the temporary credentials to the user's writing terminal. Specifically, S3 is: The user terminal sends a file upload request to the upload server. The upload server and its upload module obtain temporary credentials and file information from the file upload request. The temporary credentials processing module encodes the temporary credentials. The authentication function uses the encoding of the temporary credentials as the key to read the file information from database one and compares the file information with the file information obtained from the file upload request. If the comparison results are inconsistent, a message indicating that the temporary credential is incorrect will be returned to the user's writing terminal. If the comparison results are consistent, an identification code is generated through the authentication function. The authentication function uses the identification code as the key and the original file name and file information as the value in a hash structure to store it in database two. The authentication function updates the entry written to database two by the backend service with the credential code as the key and adds the replaced file name. The upload module replaces the filename in the file upload request with an identification code.

2. The file upload method as described in claim 1, characterized in that, After the upload is completed in step S6, the following steps are also included: S7. The file upload log service periodically reads entry information from database two using the credential code as the key, based on the received credential information. S8. Determine if the entry already contains the filename information of the replacement; S9. If there is no replacement file name information, the following judgment is made: if there is no voucher entry, the timed processing ends; if there is a voucher entry and it has expired, the record is deleted; if there is a voucher entry and it has not expired, the next entry is read and the process jumps to S8. If there is information about replacing filenames, record the file and its uploader information in the log.

3. The file upload method as described in claim 2, characterized in that, It also includes file access methods, the specific steps of which are as follows: S10, Receive file access request; S11. Obtain the filename in the access request, and obtain the entry named key based on the file; S12. If the retrieved entry does not exist, return an access failure message. S13. If the obtained entry exists, obtain the original uploaded file name from the entry; S14. Set the original uploaded file name as the downloaded file in the request header; S15. Forward the access request configured to the cloud storage service. S16. Obtain the file to be accessed.

4. The file upload method as described in claim 3, characterized in that, Specifically, S11 is: The object name conversion module retrieves the file name from the file access request information sent by the anonymous reading terminal and uses the file name as the key to retrieve the entry from database two.

5. A file upload device, characterized in that, include: At least one processor, and Memory, communicatively connected to at least one processor, is used to store executable instructions; The processor is capable of executing executable instructions stored in the memory to implement the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images