Cloud disk service acceleration scheduling system

Abstract

This invention relates to a cloud disk service acceleration scheduling system. The system comprises: when a user requests access to a cloud disk, the user-side service scheduling collects user behavior data and stores it in a user-end database, and sends the user behavior data and user request to a content delivery network (CDN); the CDN determines whether its node servers store the requested content based on the user behavior data; if so, it returns a result; otherwise, it sends the user request to an object storage server; the object storage server processes the requested content and stores it in the CDN for use in cloud disk service acceleration scheduling. This invention solves the technical problems of slow access and upload / download speeds on cloud disk servers and the inability to optimize cloud disk storage layout according to customer preferences.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a cloud disk service acceleration scheduling system. Background Technology

[0002] In existing technologies, to improve product service performance, network service providers not only continuously increase the number of network servers in various regions but also utilize CDN technology for content distribution and global load balancing technology to redirect user access to the nearest streaming media server. This not only improves the user experience but also enhances website availability. With the continuous development of cloud storage services, the daily uploads, downloads, and massive HTTP requests (including PC, mobile, and mini-program access) on cloud storage servers place enormous traffic pressure on cloud storage servers and object storage node servers. For servers in different regions, excessive access pressure on a particular node server can lead to low node efficiency and, in severe cases, even service outages. For cloud storage systems, upload and download speeds directly impact user experience. Therefore, how to effectively and efficiently schedule cloud storage servers and object storage servers to meet the ever-increasing diverse access demands of users and ensure upload and download speeds is a key issue that current cloud storage systems need to address.

[0003] Therefore, we hope to find a method and system for accelerating cloud disk service scheduling that can solve the problems existing in the current technology. Summary of the Invention

[0004] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method and system for accelerating cloud disk service scheduling, which solves the technical problems of slow access and upload / download speeds of cloud disk servers and the inability to optimize cloud disk storage layout according to customer preferences.

[0005] To achieve the above objectives, the present invention employs a method for accelerating cloud disk service scheduling, wherein the scheduling method is as follows:

[0006] When a user requests access to the cloud drive, the user-side service scheduler collects user behavior data and stores it in the user-side database, and then sends the user behavior data and user request to the content delivery network.

[0007] The content delivery network determines whether the content delivery network node server stores the user's requested content based on user behavior data. If the content delivery network node server stores the user's requested content, it returns the result; otherwise, the content delivery network sends the user's request to the object storage server.

[0008] The object storage server processes user requests and stores the requested content on the content distribution network for use by cloud disk services to accelerate scheduling.

[0009] A system for accelerating the scheduling of cloud disk services, the system comprising:

[0010] The user service scheduling module is used to store user behavior analysis data and send user behavior analysis data and user requests to the content delivery network service scheduling module.

[0011] The content delivery network service scheduling module determines the result returned by the content delivery network server based on user behavior analysis data, or sends the user request to the object storage server.

[0012] The object storage server scheduling module executes user requests according to the load balancing strategy and stores the content requested by the user to the content delivery network scheduling module.

[0013] Optionally, the user behavior analysis data includes the user's IP address, mobile phone number location, usual location, whether the user is a privileged user, and data on user behavior characteristics analyzed by the user-end platform.

[0014] Optionally, the content delivery network service scheduling module includes: content delivery network link detection and node selection, and content delivery network fault handling.

[0015] Optionally, the content delivery network link detection selects nodes:

[0016] Send probe messages to the content distribution network nodes of the specified object to obtain the link transmission time;

[0017] All content delivery network nodes that receive a response are stored in the content delivery network service scheduling module according to the link transmission time. The content delivery network node with the shortest link transmission time from the user terminal to the content delivery network edge node is selected, and the user request content is sent to the content delivery network edge node along the selected content delivery network node.

[0018] The system periodically sends messages to the content delivery network nodes to obtain the current link transmission time. When it receives a request from the user, it directly returns the information to the content delivery network node based on the cached content in the content delivery network service scheduling module.

[0019] Optionally, the content delivery network fault handling is as follows: when a content delivery network node fails and the content delivery network scheduling module is unable to obtain the corresponding link transmission time, the current content delivery network node is set as a fault node, and the content delivery network scheduling module periodically detects the connection status of the relevant links.

[0020] Optionally, the load balancing strategy includes: proximity principle, remaining storage space principle, current connection number principle, and rights user principle.

[0021] An electronic device includes: a memory and a server; the server is used to execute a computer program stored in the memory to implement a system for accelerating the scheduling of cloud disk services as described above.

[0022] The beneficial effects of this invention are:

[0023] This invention proposes a method and system for accelerating cloud disk service scheduling. It is an intelligent scheduling system specifically designed for cloud disk services, which utilizes basic load balancing and content delivery network services to provide an accelerated service process for cloud disk servers. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the system for accelerating scheduling based on cloud disk services according to the present invention.

[0025] Figure 2 This is a flowchart of the method for accelerating scheduling based on cloud disk service according to the present invention.

[0026] Figure 3 A flowchart for scheduling content delivery network services. Detailed Implementation

[0027] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0029] A Content Delivery Network (CDN) is a network of interconnected computers connected via the internet. It utilizes servers located closest to each user to deliver music, images, videos, applications, and other files to users faster and more reliably. CDNs provide users with high-performance, scalable, and low-cost online content. The main scheduling modules on the CDN side of the scheduling system include: CDN link detection, CDN node selection, and CDN fault handling.

[0030] Content Delivery Networks (CDNs) bypass bottlenecks and links on the Internet that can affect data transmission speed and stability, thus enabling faster and more stable content delivery. A CDN system can redirect user requests to the nearest service node in real time based on comprehensive information such as network traffic, the connectivity and load of each node, as well as distance and response time to the user. The goal is to allow users to access the content they need from the nearest location, alleviate Internet congestion, and improve the response speed of websites accessed by users.

[0031] A content delivery network (CDN) employs a large number of caching servers (CDN edge nodes) deployed in areas or networks where user access is relatively concentrated. When a user visits a website, global load balancing technology directs the user's request to the nearest caching server, which then responds to the user's request.

[0032] Using a content delivery network service alone has its limitations: for specific server needs such as cloud disk servers, general content delivery network acceleration solutions cannot fully meet daily scheduling requirements and cannot collect user data for subsequent analysis to provide more intelligent services; secondly, for servers with certain expansion needs, a simple content delivery network service cannot fully meet the requirements.

[0033] Load balancing distributes workloads across multiple processing units, such as web servers, FTP servers, enterprise-critical application servers, and other mission-critical servers, to collectively complete tasks. Built upon existing network infrastructure, load balancing provides a cost-effective and transparent method to expand the bandwidth of network devices and servers, increase throughput, enhance network data processing capabilities, and improve network flexibility and availability.

[0034] If a user wants to access content on a website, the specific steps involved in CDN acceleration are as follows:

[0035] ① When a user clicks on content in the app, the app seeks an IP address from the local DNS (Domain Name System) based on the URL. ② The local DNS system delegates the domain name resolution authority to the CDN's dedicated DNS server. ③ The CDN's dedicated DNS server returns the IP address of the CDN's global load balancer to the user. ④ The user initiates a content URL access request to the CDN's load balancer. ⑤ The CDN load balancer selects a cache server in the user's region based on the user's IP address and the requested content URL. ⑥ The load balancer tells the user the IP address of this cache server, allowing the user to send a request to the selected cache server. ⑦ The user sends a request to the cache server, which responds by delivering the requested content to the user's device. ⑧ If the cache server does not have the desired content, it requests the content from the website's origin server. ⑨ The origin server returns the content to the cache server, which then sends it to the user and determines whether to cache the content on the cache server based on the user's custom caching strategy.

[0036] A content delivery network (CDN) employs a large number of caching servers (CDN edge nodes) deployed in areas or networks where user access is relatively concentrated. When a user visits a website, global load balancing technology directs the user's request to the nearest caching server, which then responds to the user's request.

[0037] Example 1

[0038] like Figure 1 and 2 As shown, a system for accelerating the scheduling of cloud disk services includes:

[0039] User-side service scheduling module 201 is used to store user behavior analysis data and send user behavior analysis data and user requests to the content delivery network service scheduling module.

[0040] The content delivery network-side service scheduling module 202 determines the result returned by the content delivery network server based on user behavior analysis data, or sends the user request to the object storage server.

[0041] The server-side scheduling module 203 executes user requests according to the load balancing strategy and stores the content requested by the user in the content distribution network scheduling module.

[0042] The user-side service scheduling module 201 includes a user behavior analysis system, which aims to uncover the secrets hidden in user behavior. The platform provides various analysis methods and scenarios, such as event analysis, retention analysis, conversion analysis, user segmentation, and user retention. It can record some access data in the database when a user visits, laying the data foundation for generating user profiles. It can also optimize the storage layout of the user's cloud drive files, sending the user's IP address, mobile phone number location, and whether they are a VIP (privileged user) to the CDN service for subsequent scheduling. Therefore, a dedicated CDN service for the cloud drive server needs to be deployed on the user side. For HTTP requests to the cloud drive server, the CDN needs to intelligently cache static website resources and files to improve the access speed of the cloud drive website server on the user side. For files on the cloud drive, intelligent caching is performed based on local bandwidth costs and idle time to improve download speed. For specific requests, intelligent scheduling to the specific cloud drive server is required based on custom routing rules.

[0043] User behavior analysis systems primarily involve data collection and analysis, and information transmission.

[0044] Data Collection and Analysis: This scheduling system integrates data collection functionality across web, PC, mini-program, and app platforms. The collected data primarily includes user identification information such as their mobile phone number, the user's current IP address and location while using the cloud storage service, their usual address, the type of user's current operation, and the size of the files involved. This information is stored in a database. Subsequently, the cloud storage scheduling system will incorporate AI capabilities, employing deep learning methods to analyze users' frequent operations and implement appropriate scheduling for different users.

[0045] Information transmission: The information here is mainly used for subsequent CDN and object storage server scheduling. It transmits information such as the user's mobile phone number, IP address, and place of residence in the user's request message to the scheduling system for scheduling and transmission optimization according to different strategies.

[0046] User-side routing selection: On the user side, custom routing rules are used, such as hash calculations based on IP address or mobile phone location information, to ensure that users can always retrieve the file information they uploaded to the server. When the user's home location changes, the system redirects the user to the server at the previously home location based on database information recorded by the user behavior analysis system. For the cloud storage system, to provide better service to VIP (VIP users), the user behavior analysis system also checks whether the current requesting user is a VIP user. Servers for VIP users will be faster and more reliable. Custom routing rules also include CDN link RTT calculation and data center link RTT calculation, intelligently detecting the best CDN link and scheduling the next data center link to provide faster and more reliable cloud storage services. User-side scheduling ultimately selects the best CDN based on the custom routing rules and sends the request packet to that server, followed by the intelligent scheduling system's CDN-side service scheduling module.

[0047] Content Delivery Network Side Service Scheduling Module 202: CDN Link Detection, CDN Node Selection, CDN Fault Handling.

[0048] The process of the content delivery network side service scheduling module 202 is as follows:

[0049] The link probing and node selection rules at the CDN are as follows: (1) Send a probe message to the specified target CDN node to obtain the link transmission time (RTT) and select a suitable CDN node according to the link length; (2) Store all the CDN nodes that have received a response in the database according to the RTT and return the CDN node server with the shortest current RTT, and forward the request to the CDN server; (3) Send messages to the CDN node at regular intervals to obtain the latest connection status of the current link. When a client request is received again, the CDN node can be directly returned according to the cached content.

[0050] CDN Fault Handling: When a CDN node fails and the scheduling module is unable to obtain the corresponding RTT, the current CDN node is set as an unreachable node (faulty node). The scheduling system will still periodically detect the connection status of the relevant links, and when the CDN node recovers, it can continue to forward requests to the node.

[0051] In special cases, such as when the CDN node does not cache the requested resource or file, the CDN node needs to directly forward the request packet to the corresponding object storage server for server-side service scheduling. The CDN node will then retrieve the resource requested by the user, allowing the user to benefit from CDN acceleration services the next time they request the same resource. If the CDN edge node does not have the resource, the CDN needs to continue sending the request to the server-side service scheduling module for server-side scheduling.

[0052] Server-side scheduling module 203: Object storage server performance detection, object server load balancing

[0053] Object storage server performance probing:

[0054] While CDN nodes can provide a large number of acceleration services, for files or resources not stored on CDN nodes, CDN needs to send requests for these resources to object storage servers. Therefore, the server-side scheduling module in the scheduling system is also indispensable, and its main workflow is as follows:

[0055] (1) If a CDN node does not store the current file or resource in its internal server after receiving the request message, it means that the resource is stored in the object server. In this case, the CDN cannot provide acceleration service and needs to send the request to the object server.

[0056] (2) Determine the link length from the current node to all OSS cloud disk node servers, and the number of requests currently being processed by the target node server (current number of tasks). If the request is related to cloud disk files, it is also necessary to obtain the remaining physical storage capacity of the target node server and the supported storage methods.

[0057] (3) The scheduling system performs performance testing on the object storage server and returns the object storage server with the shortest link RTT. For example, requests sent by users in South China will generally be sent to the object storage server cluster in South China.

[0058] Server performance detection: the current read / write frequency of the target node server; for object storage servers, it is also necessary to obtain the amount of free data storage space of the target node server.

[0059] Load balancing means distributing workloads across multiple processing units, such as web servers, FTP servers, enterprise-critical application servers, and other mission-critical servers, to collectively complete tasks. Built upon existing network infrastructure, load balancing provides a cost-effective and transparent method to expand the bandwidth of network devices and servers, increase throughput, enhance network data processing capabilities, and improve network flexibility and availability.

[0060] Load balancing strategy:

[0061] (1) Proximity principle: The distance from the IP address to the server.

[0062] (2) Remaining capacity principle: The remaining disk capacity of the OSS node server

[0063] (3) Current Connection Count Principle: The number of current connection requests on the server.

[0064] (4) User Rights Principle: The user's rights level

[0065] In order to cope with emergencies, the scheduling system needs to further forward user request packets according to the load balancing strategy to ensure that users can receive normal server responses.

[0066] Example 2

[0067] like Figure 2 As shown, the process of the content delivery network service scheduling module includes: obtaining the message of the user's request, parsing the requested resource name in the current message, determining whether the current content delivery network edge node has a cache of the resource, if the resource is cached, directly returning the result from the current content delivery network edge node, if the current content delivery network edge node does not have a cache, forwarding the user's request content to the object storage server for load balancing strategy scheduling, and returning the result.

[0068] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0069] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A cloud disk service acceleration scheduling system, characterized in that, The scheduling system includes: When a user requests access to the cloud drive, the user-side service scheduler collects user behavior data and stores it in the user-side database, and then sends the user behavior data and user request to the content delivery network. The content delivery network determines whether the content delivery network node server stores the user's requested content based on user behavior data. If the content delivery network node server stores the user's requested content, it returns the result; otherwise, the content delivery network sends the user's request to the object storage server. The object storage server processes user requests and stores the requested content in the content delivery network for use by the cloud disk service to accelerate scheduling. The user-side service scheduling module is used to store user behavior analysis data and send user behavior analysis data and user requests to the content delivery network service scheduling module. The user-side service scheduling module includes a user behavior analysis system. The platform provides event analysis, retention analysis, conversion analysis, user segmentation, and user retention analysis methods and scenarios. When a user accesses a service, the system records access data in a database to provide a data foundation for generating user profiles. It optimizes the storage layout of user cloud disk files and sends the user's IP address, mobile phone number location, location, and whether the user is a privileged user to the CDN service for scheduling. For HTTP requests to the cloud disk server, the CDN needs to intelligently cache static website resources and files to improve the access speed of the cloud disk website server on the user side. For files on the cloud disk, it intelligently caches them based on local bandwidth costs and idle status to improve the download speed of cloud disk files. For specific requests, it needs to intelligently schedule them to specific cloud disk servers according to custom routing selection rules. A user behavior analysis system includes: data collection, data analysis, and information transmission; The data collection information mainly includes user information that can identify the user by their mobile phone number, the IP address and location of the user currently using the cloud storage service, their permanent residence, the type of the user's current operation, and the size of the files involved. This information is saved to the database. Data analysis is used by the cloud storage scheduling system to analyze the user's common operations using deep learning methods, and to perform reasonable scheduling for different users. Information transmission is used for subsequent CDN and object storage server scheduling. It transmits the user's mobile phone number, IP address, and residence information in the request message to the scheduling system for scheduling and transmission optimization according to different strategies. User-side routing selection: On the user side, according to the custom routing selection rules, the best CDN is selected and the request packet is sent to the server. Then, the intelligent scheduling system CDN-side service scheduling module performs the custom routing selection rules, including CDN link RTT calculation and data center link RTT calculation. It intelligently detects the current best CDN link and the data center link after scheduling, providing faster and more reliable cloud disk services. The Content Delivery Network (CDN) service scheduling module determines the result to be returned by the CDN server based on user behavior analysis data, or sends the user request to the object storage server; the object storage server scheduling module executes the user request according to the load balancing strategy and stores the content accessed by the user request to the CDN scheduling module. The object storage server scheduling module performs the following performance probes on the object storage server: (1) If a CDN node does not store the current file or resource in its internal server after receiving the request message, it means that the resource is stored in the object server. At this time, the CDN cannot provide acceleration service and needs to send the request to the object server. (2) Find the link length from the current node to all cloud disk OSS node servers, and the number of requests currently being processed by the target node server. If the request is related to cloud disk files, it is also necessary to obtain the remaining physical storage capacity of the target node server and the supported storage methods. (3) The scheduling system performs performance testing on the object storage server and returns the object storage server with the shortest link RTT; Server performance detection: the current read / write frequency of the target node server; for object storage servers, it is also necessary to obtain the amount of free data storage space of the target node server.

2. The cloud disk service acceleration scheduling system as described in claim 1, characterized in that: The content delivery network selects content delivery network node servers according to preset routing rules.

3. The cloud disk service acceleration scheduling system as described in claim 1, characterized in that: The object storage server selects a suitable object storage server to process user requests based on a load balancing scheduling strategy.

4. The cloud disk service acceleration scheduling system as described in claim 3, characterized in that: The content delivery network service scheduling module includes: content delivery network link detection and node selection, and content delivery network fault handling.

5. The cloud disk service acceleration scheduling system as described in claim 4, characterized in that: The content delivery network link detection selects nodes: Send probe messages to the content distribution network nodes of the specified object to obtain the link transmission time; All content delivery network nodes that receive a response are stored in the content delivery network service scheduling module according to the link transmission time. The content delivery network node with the shortest link transmission time from the user terminal to the content delivery network edge node is selected, and the user request content is sent to the content delivery network edge node along the selected content delivery network node. The system periodically sends messages to the content delivery network nodes to obtain the current link transmission time. When it receives a request from the user, it directly returns the information to the content delivery network node based on the cached content in the content delivery network service scheduling module.

6. The cloud disk service acceleration scheduling system as described in claim 5, characterized in that: Content Delivery Network (CDN) Fault Handling: When a CDN node fails and the CDN scheduling module is unable to obtain the corresponding link transmission time, the current CDN node is set as a faulty node, and the CDN scheduling module periodically detects the connection status of the relevant links.

7. The cloud disk service acceleration scheduling system as described in claim 6, characterized in that: The load balancing strategy includes: proximity principle, remaining storage space principle, current connection number principle, and rights user principle.

8. An electronic device, characterized in that, include: Storage and servers; The server is used to execute the computer program stored in the memory to implement the system for accelerating the scheduling of cloud disk services as described in any one of claims 1 to 7.

Images