Method and apparatus for processing file data

Abstract

This invention provides a method for processing and using file data, comprising: acquiring user-uploaded file data through a front-end file management component according to business characteristics; storing the acquired file data in a file server's database; storing the file data in a CDN node and the file server's cache according to access needs; performing scaling and permission configuration on the file data in the file server according to business needs; and, in response to access to the file data, acquiring the file data according to the following priority: CDN node, file server cache, and file server database. The purpose of this invention is to provide a method and apparatus for processing and using file data that can be personalized for acquisition, transmission, and processing based on the company's business characteristics.

Description

Technical Field

[0001] This invention belongs to the field of data transmission processing technology, and in particular relates to a method and apparatus for processing file data. Background Technology

[0002] Currently, there are various file storage systems on the market that can perform operations such as file storage, updating, and querying. However, they lack the ability to combine business needs with logical and physical data processing to improve ease of use. Furthermore, they cannot provide personalized acquisition, transmission, processing, or access optimization based on the specific characteristics of a company's business. Summary of the Invention

[0003] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a method and apparatus for processing and using file data that can be personalized for acquisition, transmission and processing according to the characteristics of a company's business.

[0004] This application provides a method for processing document data, including:

[0005] Based on the business characteristics, obtain the file data uploaded by users through the front-end file management component;

[0006] The acquired file data is stored in the database of the file server;

[0007] Based on access requirements, the file data is stored in the cache of the CDN node and the file server;

[0008] The file data in the file server is shrunk or modified and permissions are configured according to business needs;

[0009] In response to access to the file data, the file data is retrieved in the following priority order: CDN node, the file server's cache, and the file server's database.

[0010] Furthermore, the process of obtaining the file data according to the following priority includes: first obtaining the file data from the CDN node; if the CDN node does not have the file data, then obtaining it through an interface; the interface first determines whether there is a file address in the cache of the file server; if there is, it is obtained from the cache first; if not, it is obtained from the database of the file server, written to the cache, and a valid time is set for the cache.

[0011] Furthermore, when a user accesses the file data, a compressed file is displayed.

[0012] A device for file data processing is also provided, comprising:

[0013] The data acquisition module is used to acquire file data uploaded by users through the front-end file management component according to business characteristics; store the acquired file data in the database of the file server; and store the file data in the CDN node and the cache of the file server according to access needs.

[0014] The data processing module performs file reduction / modification and permission configuration on the file data in the file server according to business needs;

[0015] The data access module, in response to access to the file data, retrieves the file data according to the following priority: CDN node, the cache of the file server, and the database of the file server.

[0016] The improvements in this application bring the following advantages: The file data processing method of this application can obtain the file format and size data required by different business configurations and transmit it to servers in different environments such as development and testing, reducing the configuration workload of operations. It processes the data into the required compressed format and file size, facilitating display to front-end users in different scenarios. Logical association processing is performed on the data to control access permissions for different users. The data is then transmitted to cache and CDN, improving the speed of user file access. Attached Figure Description

[0017] Figure 1 This is an optional system architecture used for implementing file data processing in the embodiments of this application.

[0018] Figure 2 A flowchart illustrating the file data processing method provided in an embodiment of this application is shown. Detailed Implementation

[0019] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0020] Figure 1 An optional system architecture for implementing file data processing provided in the embodiments of this application, such as Figure 1 The system architecture may include: client 100, CDN 200, and file server 300.

[0021] Users can upload files using a client; CDN stands for Content Delivery Network; and a file server can be a server or a cluster of servers that stores and manages file data.

[0022] This application aims to achieve the following: Users upload file data through a client; the file server obtains the uploaded file data through a front-end file management component and stores it in its database; based on access needs, the file data is stored in the CDN node and the file server's cache for later access; then, the file data in the file server is physically processed according to business needs, i.e., reducing file size or compressing it into different data formats for easier storage and front-end user use; then, logical processing is performed, i.e., configuring different access permissions for different users; finally, when users, administrators, and users access relevant file data, the file data is retrieved from the CDN node, the file server's cache, and the file server's database in a progressively decreasing priority order; first, the file data is retrieved from the CDN node; if the file data is not present on the CDN node, it is retrieved through an interface; the interface first checks if the file address exists in the file server's cache; if it does, it is retrieved from the cache first; otherwise, it is retrieved from the file server's database, written to the cache, and an expiration time is set for the cache.

[0023] Next, from the perspective of file server 300, the file data processing and usage method of this application will be introduced, such as... Figure 2 As shown, Figure 2 A flowchart of a file data processing method provided in an embodiment of this application is shown. The method may include the following steps:

[0024] S100, based on business characteristics, obtains user-uploaded file data through a front-end file management component.

[0025] S200 stores the acquired file data in the file server's database.

[0026] In the testing environment, the acquired file data can be transferred to the development environment server for use in the development environment, reducing repetitive configuration work for developers.

[0027] The S300 stores file data in the cache of CDN nodes and file servers as needed for access.

[0028] Data acquisition methods can be configured according to business needs. For data transmission, data can be transferred from the uploaded file system to the read file system, improving data access efficiency; it can also be transferred to different servers, reducing repetitive configuration work for operations; and it can be transferred to caching systems and CDNs to improve user access speed. For data processing, physical size and logical relationships can be handled according to business needs, facilitating display and access control.

[0029] S400 performs file reduction / modification and permission configuration on file server data according to business needs.

[0030] The processed data can also be used by product operators in the management backend to select products, activity templates, recharge centers, navigation, and other scenarios, and access can be controlled based on the preceding logic.

[0031] Based on the characteristics of various business operations, different file formats and sizes can be obtained according to the configuration, processed into the format and size required by the business, and logical data permission handling can be performed to facilitate personalized display and use. Furthermore, physical data can be transferred to different file servers, and logical data can be transferred to different environment databases. A single backend configuration can be used in multiple environments, reducing repetitive configuration work for operations.

[0032] The S500, in response to access to file data, retrieves file data according to the following priority: CDN nodes, file server cache, and file server database.

[0033] When a user accesses the file data, a compressed version of the file is displayed. For example, thumbnails are shown to the user to reduce the amount of data processed and transmitted, thus improving efficiency.

[0034] The method provided in this application supports the basic operation functions of file systems on the market, and can also obtain specific files required by the business. It can transmit user-uploaded image files to a dedicated server and compress them, and can process uploaded logical file data in conjunction with the company's user permission system, control access to files by different organizations and users, and allow users to access files faster by transmitting file data to a cache or CDN.

[0035] Based on different business configurations, it can obtain the file format and size data required by the business and transmit it to servers in different environments such as development and testing, reducing the configuration workload of operations. It processes the data into the required compressed format and file size for easy display to front-end users in different scenarios. It performs logical association processing on the data to control access for different users. Finally, it transmits the data to cache and CDN to improve the speed of file access for users.

[0036] For physical file storage, add, update, and delete operations can be assigned to a single machine. When a file is added or updated, the system automatically synchronizes it to other servers for querying. Alternatively, file synchronization can be avoided, and add, update, and query operations can share a single file server. Deploying multiple file services improves system access performance, and caching enhances access speed. However, when access volume is high, limiting the frequency of file add and update operations may result in a slightly worse user experience.

[0037] An apparatus for document data processing includes:

[0038] The data acquisition module is used to acquire user-uploaded file data through the front-end file management component according to business characteristics; store the acquired file data in the database of the file server; and store the file data in the cache of CDN nodes and the file server as needed.

[0039] The data processing module performs file reduction / modification and permission configuration on the file server according to business needs;

[0040] The data access module, in response to access to file data, retrieves file data according to the following priority: CDN nodes, file server cache, and file server database.

[0041] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for use in processing file data, characterized by, include: Based on the business characteristics, obtain the file data uploaded by users through the front-end file management component; The acquired file data is stored in the database of the file server; Based on access requirements, the file data is stored in the cache of the CDN node and the file server; The file data in the file server is shrunk or modified and permissions are configured according to business needs; In response to access to the file data, the file data is retrieved in the following priority order: CDN node, the file server's cache, and the file server's database; The step of scaling down and configuring permissions for the file data in the file server according to business requirements includes: Based on the characteristics of each business, files of different formats and sizes are obtained according to the configuration, processed into the format and size required by the business, and logical data permission processing is performed. The file data can be transferred to different file servers and the logical data can be transferred to databases in different environments, so that a single backend configuration can be used in multiple environments.

2. The method according to claim 1, characterized in that, The method of obtaining the file data according to the following priority includes: first obtaining the file data from the CDN node; if the CDN node does not have the file data, then obtaining it through an interface; the interface first determines whether there is a file address in the cache of the file server; if there is, it is obtained from the cache first; if not, it is obtained from the database of the file server, written to the cache, and a valid time is set for the cache.

3. The method of claim 1, wherein, When a user accesses the file data, the compressed file is displayed.

4. An apparatus for use in processing file data, characterized by include: The data acquisition module is used to obtain user-uploaded file data through the front-end file management component, based on business characteristics; The acquired file data is stored in the database of the file server; Based on access requirements, the file data is stored in the cache of the CDN node and the file server; The data processing module performs file reduction / modification and permission configuration on the file data in the file server according to business needs; The data access module, in response to access to the file data, retrieves the file data according to the following priority: CDN node, the cache of the file server, and the database of the file server; The data processing module is specifically used for: Based on the characteristics of each business, files of different formats and sizes are obtained according to the configuration, processed into the format and size required by the business, and logical data permission processing is performed. The file data can be transferred to different file servers and the logical data can be transferred to databases in different environments, so that a single backend configuration can be used in multiple environments.

Images