Method for accessing data storage service, computing device and computer storage medium

Abstract

Embodiments of the present application provide a data storage service access method, a central cloud node, an edge cloud node, a computer readable storage medium and a computing device. The data storage service access method comprises: obtaining a plurality of probe data sent by an edge cloud node, the plurality of probe data being generated by respectively probing service states of a plurality of data storage services deployed in the edge cloud node; generating service scheduling parameters corresponding to the plurality of data storage services according to the plurality of probe data; and sending the plurality of service scheduling parameters to the edge cloud node, so that the edge cloud node determines a target data storage service for responding to a data processing request from the plurality of data storage services based on the plurality of service scheduling parameters. The technical scheme provided by the embodiments of the present application reduces the cloud return request rate of the edge cloud node, and improves the security and performance of data processing of the edge cloud node.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of cloud computing technology, and in particular to a data storage service access method, computing device and computer storage medium. Background Technology

[0002] Edge computing refers to migrating storage and network resources from traditional central cloud nodes to edge cloud nodes, moving them closer to the user side, reducing the physical distance between users and resources, achieving lower data interaction latency, saving network traffic, and thus providing users with low-latency, high-stability network interaction solutions.

[0003] Edge cloud nodes typically deploy multiple data storage services for data storage. These services receive data processing requests and data to be stored from users, and then route the data to storage devices. Users are unaware of the number of data storage services present in an edge cloud node or the specific data content stored in each service.

[0004] In related technologies, when a user wants to obtain data stored in an edge cloud node, they can usually send a data processing request to the edge cloud node. The edge cloud node then needs to forward the received data processing request to the central cloud node so that the central cloud node can determine the data storage service corresponding to the request and return the determined target data storage service address to the edge cloud node. Thus, the edge cloud node can schedule the object retrieval request to the target data storage service, so that the data can be read from the target data storage service and returned to the user.

[0005] Because edge cloud nodes need to communicate over the public network when forwarding object retrieval requests to central cloud nodes, users' access to data storage services in high-concurrency scenarios is subject to significant technical challenges due to fluctuations in network quality. Summary of the Invention

[0006] This invention provides a method, apparatus, computing device, and computer storage medium for accessing data storage services.

[0007] In a first aspect, embodiments of the present invention provide a method for accessing a data storage service, applied to a central cloud node, the method comprising:

[0008] Multiple probe data sent by edge cloud nodes are acquired, and the multiple probe data are generated by probing the service status of multiple data storage services deployed in the edge cloud nodes respectively;

[0009] Based on the multiple probe data, service scheduling parameters corresponding to the multiple data storage services are generated respectively;

[0010] Multiple service scheduling parameters are sent to the edge cloud node so that the edge cloud node can determine a target data storage service from multiple data storage services to respond to the data processing request based on the multiple service scheduling parameters.

[0011] Secondly, this invention provides a method for accessing a data storage service, applied to an edge cloud node, the method comprising:

[0012] Receive data processing requests;

[0013] Retrieve service scheduling parameters for multiple pre-stored data storage services;

[0014] Determine whether there is a target service scheduling parameter among multiple service scheduling parameters that matches the data processing request;

[0015] If the target service scheduling parameters exist, the target data storage service corresponding to the target service scheduling parameters is used to respond to the data processing request.

[0016] Thirdly, this invention provides a method for accessing a data storage service, applied to an edge cloud node, the method comprising:

[0017] The service status of multiple data storage services deployed on the edge cloud nodes is detected, and multiple detection data are generated.

[0018] Multiple probe data are sent to the central cloud node so that the central cloud node can generate multiple service scheduling parameters based on the multiple probe data;

[0019] The service scheduling parameters sent by the central cloud node are received so as to determine the target data storage service for responding to the data processing request from among the multiple data storage services based on the multiple service scheduling parameters.

[0020] Fourthly, embodiments of the present invention provide a central cloud node, comprising:

[0021] The first acquisition module is used to acquire multiple probe data sent by the edge cloud node. The multiple probe data are generated by probing the service status of multiple data storage services deployed in the edge cloud node respectively.

[0022] The parameter generation module is used to generate service scheduling parameters corresponding to the multiple data storage services based on the multiple probe data.

[0023] The parameter sending module is used to send multiple service scheduling parameters to the edge cloud node, so that the edge cloud node can determine the target data storage service for responding to the data processing request from multiple data storage services based on the multiple service scheduling parameters.

[0024] Fifthly, embodiments of the present invention provide an edge cloud node, comprising:

[0025] The request receiving module is used to receive data processing requests;

[0026] The parameter acquisition module is used to acquire service scheduling parameters of multiple pre-stored data storage services;

[0027] The judgment module is used to determine whether there is a target service scheduling parameter among multiple service scheduling parameters that matches the data processing request;

[0028] The request-response module is used to respond to the data processing request using the target data storage service corresponding to the target service scheduling parameters, if the target service scheduling parameters exist.

[0029] Sixthly, another embodiment of the present invention provides an edge cloud node, comprising:

[0030] The detection module is used to detect the service status of multiple data storage services deployed on the edge cloud node and generate multiple detection data.

[0031] The data sending module is used to send multiple probe data to the central cloud node, so that the central cloud node can generate multiple service scheduling parameters based on the multiple probe data;

[0032] The parameter receiving module is used to receive the service scheduling parameters sent by the central cloud node, so as to determine the target data storage service for responding to the data processing request from multiple data storage services based on the multiple service scheduling parameters.

[0033] In a seventh aspect, embodiments of the present invention provide a computing device, including a processing component and a storage component;

[0034] The storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the data storage service access method provided in this embodiment of the invention.

[0035] Eighthly, embodiments of the present invention provide a computer storage medium storing a computer program, wherein when the computer executes the computer program, it implements the data storage service access method provided in the embodiments of the present invention.

[0036] This invention provides a method for accessing data storage services. The method involves acquiring multiple probe data sent by an edge cloud node, generated by probing the service status of multiple data storage services deployed on the edge cloud node; generating service scheduling parameters corresponding to each of the multiple data storage services based on the probe data; and sending these service scheduling parameters to the edge cloud node. This allows the edge cloud node to determine a target data storage service from the multiple data storage services to respond to a data processing request. Upon receiving a data processing request, the edge cloud node can schedule the request to a matching target data storage service based on locally stored service scheduling parameters, without needing to communicate with the central cloud node. This avoids the impact of network quality on the communication process between the edge cloud node and the central cloud node, as is common in related technologies, reduces the cloud return request rate of the edge cloud node, and improves the security and performance of data processing at the edge cloud node.

[0037] These or other aspects of the invention will become more apparent from the following description of the embodiments. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of a network system provided in one embodiment of the present invention;

[0040] Figure 2 This illustration schematically demonstrates a method for accessing a data storage service provided by an embodiment of the present invention;

[0041] Figure 3 This illustration schematically shows a method for accessing a data storage service provided by another embodiment of the present invention;

[0042] Figure 4 This illustration schematically shows a method for accessing a data storage service provided by another embodiment of the present invention;

[0043] Figure 5 This diagram schematically illustrates the architecture of a network system that utilizes the data storage service access method provided in this embodiment of the invention.

[0044] Figure 6 The diagram illustrates a block diagram of a central cloud node according to an embodiment of the present invention.

[0045] Figure 7 The diagram illustrates a block diagram of an edge cloud node according to an embodiment of the present invention.

[0046] Figure 8 The diagram illustrates a block diagram of an edge cloud node according to another embodiment of the present invention;

[0047] Figure 9 The diagram illustrates a block diagram of a computing device provided in one embodiment of the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0049] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Figure 1 This is a schematic diagram of the structure of a network system provided in one embodiment of the present invention. Figure 1 As shown, the network system 100 includes a central cloud node 101 and at least one edge cloud node 102.

[0052] The network system 100 in this embodiment is a cloud computing platform built on edge infrastructure based on cloud computing technology and edge computing capabilities. It is a cloud platform with computing, networking, storage and security capabilities at the edge.

[0053] In the network system of this embodiment, the central cloud node 101 can refer to a software module, application, service, or physical device with management and control functions. There can be one or more central cloud nodes 101. Multiple central cloud nodes 101 can be deployed on different physical machines, or in different containers or virtual machines (VMs). Of course, these containers or VMs can be deployed on the same physical machine or on multiple different physical machines.

[0054] Optionally, different physical devices can be connected wirelessly or via wired connection. For example, different physical devices can be connected via Ethernet cable or fiber optic cable. Alternatively, different physical devices can be connected via mobile network communication, and the mobile network standard can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), 5G, WiMax, etc. Optionally, different physical devices can also communicate via Bluetooth WiFi, infrared, Virtual Private Cloud (VPC), or Remote Direct Memory Access (RDMA).

[0055] Corresponding to the central cloud node 101, the network system 100 in this embodiment can be considered an edge cloud network system. Edge cloud is a relative concept, referring to a cloud computing platform relatively close to the terminal. The central cloud node 101 may include a data center with large-scale resources and concentrated locations, while the edge cloud node 102 covers a wider network range and therefore has the characteristic of being closer to the terminal. A single edge cloud node 102 has a smaller resource scale, but there are many edge cloud nodes 102, and multiple edge cloud nodes 102 constitute the edge cloud in this embodiment. The terminal in this embodiment refers to the demand side of cloud computing services, such as a terminal or user terminal in the Internet of Things (IoT). The edge cloud network is a network built based on the infrastructure between the central cloud or traditional cloud computing system and the terminal. The network system 100 includes at least one edge cloud node 102, and each edge cloud node 102 includes a series of edge infrastructures, including but not limited to: storage service clusters, wireless data centers or clusters, operator communication networks, core network equipment, base stations, edge gateways, home gateways, computing devices and / or storage devices, and corresponding network environments, etc. It should be noted that the location, capabilities, and included infrastructure of different edge cloud nodes 102 may or may not be the same.

[0056] In an edge cloud scenario, the central cloud node 101 and the edge cloud node 102 can be located in different networks. For example, the central cloud node 101 may be located in a wide area network (WAN), while the edge cloud node 102 may be located in a local area network (LAN). The LAN can be a Virtual Private Cloud (VPC) network.

[0057] In related technologies, when a terminal wants to obtain data stored in an edge cloud node, it can usually send a data processing request to the edge cloud node. The edge cloud node then needs to forward the received data processing request to the central cloud node so that the central cloud node can determine the data storage service corresponding to the request and return the determined target data storage service address to the edge cloud node. Thus, the edge cloud node can schedule the data processing request to the target data storage service, so that the data can be read from the target data storage service and returned to the terminal.

[0058] Because edge cloud nodes need to communicate over the public network when forwarding data processing requests to central cloud nodes, users' access to data storage services in high-concurrency scenarios is subject to significant technical challenges due to fluctuations in network quality.

[0059] To address the technical problems existing in related technologies, embodiments of the present invention provide a method for accessing data storage services. This method involves acquiring multiple probe data sent by edge cloud nodes, generated by probing the service status of multiple data storage services deployed on the edge cloud nodes; generating service scheduling parameters corresponding to each of the multiple data storage services based on the probe data; and sending these service scheduling parameters to the edge cloud nodes. This allows the edge cloud nodes to determine the target data storage service for responding to data processing requests based on these parameters. Upon receiving a data processing request, the edge cloud node can schedule the request to the matching target data storage service for processing based on the locally stored service scheduling parameters, without needing to communicate with the central cloud node. This avoids the impact of network quality on the communication process between the edge cloud node and the central cloud node, reduces the cloud return request rate of the edge cloud node, and improves the security, efficiency, and performance of data processing at the edge cloud node.

[0060] Figure 2 This illustration illustrates an embodiment of a data storage service access method provided by the present invention. The data storage service access method is applied to a central cloud node and may include the following steps:

[0061] 201. Obtain multiple probe data sent by the edge cloud node. These multiple probe data are generated by probing the service status of multiple data storage services deployed in the edge cloud node.

[0062] 202. Generate service scheduling parameters corresponding to multiple data storage services based on multiple probe data.

[0063] 203. Send multiple service scheduling parameters to the edge cloud node so that the edge cloud node can determine the target data storage service to respond to the data processing request from multiple data storage services based on the multiple service scheduling parameters.

[0064] According to embodiments of the present invention, a data storage system can be used to provide data storage services, wherein the data storage service may include metadata storage services and / or file data storage services.

[0065] According to an embodiment of the present invention, an edge cloud node can record the service status of multiple data storage services within a preset time interval, thereby generating probe data. Specifically, the edge cloud node can use a first time point as the starting time point to begin probing the service status of multiple data storage services, and at a second time point separated from the first time point by a preset time interval, summarize the probe results to generate probe data and send it to the central cloud node. Subsequently, the second time point can be used as the starting time point for the next round of probe data collection, but this is not limited to this; a third time point after the second time point can also be used as the starting time point for the next round of probe data collection. The specific value of the preset time interval can be flexibly set by those skilled in the art according to actual application needs, for example, it can be 10 minutes, 15 minutes, 30 minutes, etc. This embodiment of the present invention does not specifically limit the value of the preset time period.

[0066] According to embodiments of the present invention, the service status of the data storage service may include, for example, the data write status and the data read status. Correspondingly, the probe data may include latency information and / or success rate information during data writing or reading.

[0067] According to an embodiment of the present invention, after acquiring probe data from multiple data storage services, the central cloud node can perform statistical analysis on the probe data to generate service scheduling parameters corresponding to the multiple data storage services. These service scheduling parameters can be used to characterize the performance status of the corresponding data storage service during data read / write operations.

[0068] According to embodiments of the present invention, the probe data can be recorded data generated by directly recording the service status of the data storage service. This recorded data is typically discrete and abstract. Because edge cloud nodes lack core storage and computing components, they cannot store scheduling strategies and lack the corresponding probe data parsing capabilities. Therefore, edge cloud nodes cannot directly and intuitively determine the performance of the corresponding data storage service through the probe data. However, because central cloud nodes have resource-scale and centrally located data centers with stronger computing power, after acquiring the probe data, they can perform statistical analysis to generate standardized service scheduling parameters. The performance level of the corresponding data storage service can be directly determined through the values ​​of these service scheduling parameters.

[0069] According to an embodiment of the present invention, after generating service scheduling parameters, the central cloud node can send the service scheduling parameters to the edge cloud node. Thus, the edge cloud node can use the service scheduling parameters, which reflect the service status of the data storage service in the previous time interval, to schedule the received data processing request to the matching target data storage service for processing.

[0070] According to an embodiment of the present invention, by sending the service scheduling parameters calculated by the central cloud node to the edge cloud node, the edge cloud node can schedule the data processing request directly through the locally stored service scheduling parameters without communicating with the central cloud node through the public network to obtain the service scheduling strategy after receiving the data processing request. This reduces the back-to-cloud request rate of the edge cloud node and improves the security, efficiency and performance of the data processing of the edge cloud node.

[0071] According to embodiments of the present invention, edge cloud nodes may be unable to schedule data processing requests locally due to unforeseen circumstances. For example, network fluctuations may cause the edge cloud node's probe data to fail to be sent to the central cloud node, or the service scheduling parameters generated by the central cloud node may fail to be sent to the edge cloud node. As a result, the service scheduling parameters stored in the edge cloud node are expired service scheduling parameters received a long time ago. Expired service scheduling parameters received a long time ago usually cannot reflect the data processing performance of the corresponding data storage service at the current time. Therefore, the edge cloud node cannot schedule data processing requests locally.

[0072] According to an embodiment of the present invention, the method for accessing the data storage service further includes:

[0073] Receive data processing requests, which are sent when the edge cloud node does not have a target data storage service that can respond to the data processing request.

[0074] The first data storage service that matches the data processing request is determined from multiple data storage services based on the description information of multiple data storage services deployed in pre-stored edge cloud nodes.

[0075] Service scheduling information is generated based on the first data storage service.

[0076] The service scheduling information is sent to the edge cloud node so that the edge cloud node can use the first data storage service to process the corresponding data request.

[0077] According to an embodiment of the present invention, when an edge cloud node is unable to schedule data processing requests locally due to unforeseen circumstances, the edge cloud node can forward the received data processing requests to the central cloud node so that the central cloud node can process the data processing requests.

[0078] According to embodiments of the present invention, the description information of the data storage service may include, for example, the type information of the data storage service, the characteristic information of the data storage service, and the historical service status information of the data storage service.

[0079] According to embodiments of the present invention, historical service status information of data storage services can be obtained by statistically analyzing probe data sent by edge cloud nodes within a preset historical period. For example, information such as the failure rate, latency variation curve, and data read / write success rate variation curve of each data storage service within a preset historical period can be obtained based on the statistical analysis of the probe data.

[0080] According to embodiments of the present invention, different types of data storage services often have different characteristics, and thus their processing performance will vary for different data processing requests. For example, hash index type data storage services are generally suitable for data processing requests involving small amounts of data. They are efficient in querying and modifying data and are suitable for random queries but not for range queries.

[0081] According to an embodiment of the present invention, upon receiving a data processing request, the request can first be parsed to determine the type of data processing requested. Then, based on the parsed data processing type, at least one candidate data storage service whose type information matches the data processing type can be determined from a plurality of data storage services. Finally, a target data storage service can be determined from the at least one candidate data storage service based on its historical service status information. Specifically, the candidate data storage service with a low failure rate, small latency variation, and high success rate within a preset historical period can be determined as the target data storage service.

[0082] According to an embodiment of the present invention, after determining the target data storage service, service scheduling information can be generated based on the identification information or address of the target data storage service, and the service scheduling information can be sent to the edge cloud node.

[0083] According to an embodiment of the present invention, the generation of service scheduling parameters corresponding to multiple data storage services based on multiple probe data can be specifically implemented as follows:

[0084] Determine the differences between multiple detection data and standard working data respectively;

[0085] The weighting factors corresponding to multiple data storage services are determined based on the difference.

[0086] The weighting factor is determined as the service scheduling parameter.

[0087] According to an embodiment of the present invention, standard working data can be pre-set for the data storage service. The standard working data may include data obtained by collecting the service status of the data storage service under normal working conditions.

[0088] According to an embodiment of the present invention, by determining the difference between the probe data and the standard working data, it is possible to determine whether the current service status of the data storage service corresponding to the probe data is normal. For example, if the difference is positive, it indicates that the current service status of the data storage service corresponding to the probe data is good; if the difference is 0, it indicates that the current service status of the data storage service corresponding to the probe data is normal; if the difference is negative, the current service status of the data storage service corresponding to the probe data is poor.

[0089] According to an embodiment of the present invention, the values ​​of weighting factors corresponding to multiple difference intervals can be preset. After determining the differences between multiple probe data and standard working data, the differences can be matched with multiple difference intervals to determine the corresponding weighting factor values. For example, the difference between a probe data and standard working data is A3, and the weighting factor corresponding to the difference interval A0-A2 is λ1, and the weighting factor corresponding to the difference interval A3-A4 is λ2. Since the difference between the probe data and the standard working data is A3, this difference A3 falls within the difference interval A3-A4, and therefore, the weighting factor corresponding to the probe data can be determined to be λ2.

[0090] According to an embodiment of the present invention, the weighting factor can characterize the priority of each data storage service when scheduling data storage services. For example, when scheduling data processing requests, data processing requests can be preferentially scheduled to the data storage service with the highest weighting factor value.

[0091] Based on and Figure 2 The invention concept shown is the same as or similar to the data storage service access method. Figure 3 This illustration illustrates another embodiment of the present invention, a method for accessing a data storage service applied to an edge cloud node, which may include the following steps:

[0092] 301, Receive data processing request;

[0093] 302, retrieve service scheduling parameters for multiple pre-stored data storage services;

[0094] 303, determine if there is a target service scheduling parameter among multiple service scheduling parameters that matches the data processing request;

[0095] 304. If target service scheduling parameters exist, the target data storage service corresponding to the target service scheduling parameters will be used to respond to the data processing request.

[0096] According to an embodiment of the present invention, the service scheduling parameters can be obtained and stored in advance from the central cloud node.

[0097] According to an embodiment of the present invention, the service scheduling parameters may include, for example, a weight factor calculated by the central cloud node for each data storage service, and the weight factor may characterize the priority of each data storage service when scheduling data storage services.

[0098] According to an embodiment of the present invention, determining whether there is a target service scheduling parameter among multiple service scheduling parameters that matches the data processing request can be specifically implemented as follows:

[0099] The algorithm determines whether at least one weight factor among the weight factors corresponding to multiple data storage services is greater than a preset weight factor threshold. If so, a target weight factor is determined from the at least one weight factor that is greater than the preset weight factor threshold, and the data storage service corresponding to the target weight factor is identified as the target data storage service. Specifically, for example, the weight factor with the largest value among the at least one weight factor that is greater than the preset weight factor threshold can be identified as the target weight factor.

[0100] According to embodiments of the present invention, edge cloud nodes may be unable to schedule data processing requests locally due to unforeseen circumstances. For example, network fluctuations may cause the edge cloud node's probe data to fail to be sent to the central cloud node, or the service scheduling parameters generated by the central cloud node may fail to be sent to the edge cloud node. As a result, the service scheduling parameters stored in the edge cloud node are expired service scheduling parameters received a long time ago. Expired service scheduling parameters received a long time ago usually cannot reflect the data processing performance of the corresponding data storage service at the current time. Therefore, the edge cloud node cannot schedule data processing requests locally.

[0101] According to an embodiment of the present invention, the method for accessing the data storage service further includes:

[0102] If no target service scheduling parameter matches the data processing request in the service scheduling parameters, the data processing request will be sent to the central cloud node.

[0103] Receive service scheduling information sent by the central cloud node;

[0104] The data processing request is scheduled to the first data storage service corresponding to the service scheduling information.

[0105] According to an embodiment of the present invention, when an edge cloud node is unable to schedule data processing requests locally due to unforeseen circumstances, the edge cloud node can forward the received data processing requests to the central cloud node so that the central cloud node can process the data processing requests.

[0106] The central cloud node can determine the first data storage service that matches the data processing request from multiple data storage services based on the description information of multiple data storage services deployed in the pre-stored edge cloud nodes, and generate service scheduling information based on the first data storage service.

[0107] According to embodiments of the present invention, the description information of the data storage service may include, for example, the type information of the data storage service, the characteristic information of the data storage service, and the historical service status information of the data storage service.

[0108] According to embodiments of the present invention, historical service status information of data storage services can be obtained by statistically analyzing the service status of multiple data storage services deployed in edge cloud nodes within a preset historical period. For example, information such as the failure rate, latency variation curve, and data read / write success rate variation curve of each data storage service within the preset historical period can be analyzed and determined.

[0109] According to embodiments of the present invention, different types of data storage services often have different characteristics, and thus their processing performance will vary for different data processing requests. For example, hash index type data storage services are generally suitable for data processing requests involving small amounts of data. They are efficient in querying and modifying data and are suitable for random queries but not for range queries.

[0110] According to an embodiment of the present invention, after receiving a data processing request, the central cloud node can first parse the data processing request to determine the type of data processing requested. Then, based on the parsed data processing type, it can determine at least one candidate data storage service whose type information matches the data processing type from a plurality of data storage services. However, it can also determine a target data storage service from at least one candidate data storage service based on the historical service status information of the at least one candidate data storage service. Specifically, the candidate data storage service with a low failure rate, small latency variation, and high success rate within a preset historical period can be determined as the target data storage service.

[0111] According to an embodiment of the present invention, after determining the target data storage service, the central cloud node can generate service scheduling information based on the identification information or address of the target storage service and send the service scheduling information to the edge cloud node.

[0112] According to an embodiment of the present invention, the method for accessing the data storage service further includes:

[0113] Receive service scheduling parameters for multiple data storage services sent by the central cloud node;

[0114] The service scheduling parameters are stored in the cache storage area, the cache middleware storage area, and the disk storage area, respectively.

[0115] Retrieving service scheduling parameters for multiple pre-stored data storage services includes:

[0116] The system sequentially accesses the cache storage area, cache middleware storage area, and disk storage area until it successfully obtains the service scheduling parameters.

[0117] According to an embodiment of the present invention, after receiving the service scheduling parameters sent by the central cloud node, a copy of the service scheduling parameters can be stored in the cache storage area, the cache middleware storage area, and the disk storage area of ​​the edge cloud node.

[0118] According to embodiments of the present invention, the cache storage area has the characteristics of transient storage, loss upon power failure, and high-speed reading. Storing service scheduling parameters in the cache storage area can improve the reading speed of service scheduling parameters during request scheduling. However, due to the transient storage characteristic of the cache storage area, if only service scheduling parameters are saved in the cache storage area, there is a risk of loss of service scheduling parameters if the central cloud node experiences an unexpected power outage.

[0119] According to an embodiment of the present invention, the caching middleware serves as an intermediate medium connecting to the underlying database, allowing access to the underlying database. Caching middleware typically employs a persistent data storage mechanism; therefore, by synchronously storing service scheduling parameters in the caching middleware's storage area, data security issues inherent in caching can be avoided.

[0120] According to an embodiment of the present invention, in order to further ensure the data security of service scheduling parameters, the service scheduling parameters can also be stored in a disk storage area for direct persistent storage.

[0121] According to an embodiment of the present invention, since the cache storage area has the fastest read speed, when data storage service scheduling is required, the cache storage area can be read first. If the service scheduling parameters can be obtained directly from the cache storage area, the obtained service scheduling parameters can be used directly for scheduling. If the service scheduling parameters cannot be obtained directly from the cache storage area, the cache middleware storage area and the disk storage area can be accessed sequentially until the service scheduling parameters are obtained.

[0122] According to an embodiment of the present invention, by storing a copy of the service scheduling parameters in the cache storage area, the cache middleware storage area, and the disk storage area of ​​the edge cloud node, the data security of the service scheduling parameters can be guaranteed while ensuring the acquisition speed of the service scheduling parameters.

[0123] According to an embodiment of the present invention, the method for accessing the data storage service further includes:

[0124] Obtain file cleanup instructions from the central cloud node. The file cleanup instructions carry description information of the files to be cleaned up.

[0125] In response to a file cleanup command, access multiple data storage services to identify the files to be cleaned that correspond to the description information;

[0126] Perform file cleanup operations on the files to be cleaned.

[0127] According to embodiments of the present invention, when a user stores a data file in a data storage service on an edge cloud node, the user may not need to persistently store the data file, but only need to store it for a certain period of time, such as one week or one month. Therefore, when storing a data file, the user can configure the storage time for the data file, such as how long it needs to be stored from the current time, or the expected cleanup time for the data file.

[0128] According to an embodiment of the present invention, after the data file storage is completed, the edge cloud node can record the storage time of each data file and report it to the central cloud node so that the central cloud node can monitor the data files stored in the edge cloud node.

[0129] According to an embodiment of the present invention, when the central cloud node monitors and determines that the data file stored in the data storage service of the edge cloud node has reached the cleaning requirement, it can send a file cleaning instruction to the edge cloud node based on the description information of the data file, thereby enabling the edge cloud node to control the storage service storing the data file to be cleaned to delete the data file to be cleaned.

[0130] According to an embodiment of the present invention, the method for accessing the data storage service further includes:

[0131] Determine the number of data processing requests and the threshold number of requests that the edge cloud node can handle;

[0132] If the number of requests exceeds the request volume threshold, the request volume threshold is used to filter the requests so as to generate data processing requests with a request volume not exceeding the request volume threshold.

[0133] According to an embodiment of the present invention, in order to control the concurrent request pressure of edge cloud nodes, the requests received by the edge cloud nodes can be rate-limited by using a pre-set request volume threshold.

[0134] According to an embodiment of the present invention, the request volume threshold may include the sum of the maximum number of requests that each data storage service deployed in the edge cloud node can respond to.

[0135] According to an embodiment of the present invention, when the number of data processing requests exceeds a request threshold, the number of data processing requests can be filtered by randomly discarding them, so that the number of filtered data processing requests is less than or equal to the request threshold.

[0136] Figure 4 This illustration illustrates a data storage service access method provided by another embodiment of the present invention. This data storage service access method can be applied to edge cloud nodes and includes the following steps:

[0137] 401, to probe the service status of multiple data storage services deployed on edge cloud nodes and generate multiple probe data;

[0138] 402, send multiple probe data to the central cloud node so that the central cloud node can generate multiple service scheduling parameters based on the multiple probe data;

[0139] 403, Receive service scheduling parameters sent by the central cloud node, so as to determine the target data storage service for responding to the data processing request from multiple data storage services based on multiple service scheduling parameters.

[0140] According to an embodiment of the present invention, an edge cloud node can record the service status of multiple data storage services within a preset time interval, thereby generating probe data. Specifically, the edge cloud node can use a first time point as the starting time point to begin probing the service status of multiple data storage services, and at a second time point separated from the first time point by a preset time interval, summarize the probe results to generate probe data and send it to the central cloud node. Subsequently, the second time point can be used as the starting time point for the next round of probe data collection, but this is not limited to this; a third time point after the second time point can also be used as the starting time point for the next round of probe data collection. The specific value of the preset time interval can be flexibly set by those skilled in the art according to actual application needs, for example, it can be 10 minutes, 15 minutes, 30 minutes, etc. This embodiment of the present invention does not specifically limit the value of the preset time period.

[0141] According to embodiments of the present invention, the service status of the data storage service may include, for example, the data write status and the data read status. Correspondingly, the probe data may include latency information and / or success rate information during data writing or reading.

[0142] According to an embodiment of the present invention, after obtaining probe data from multiple data storage services, the central cloud node can perform statistical analysis on the probe data to generate service scheduling parameters corresponding to the multiple data storage services. These service scheduling parameters can be used to characterize the performance status of the corresponding data storage service during data read / write operations.

[0143] According to embodiments of the present invention, the probe data can be recorded data generated by directly recording the service status of the data storage service. This recorded data is usually discrete and relatively abstract. Since edge cloud nodes lack core storage and computing components, they cannot store scheduling strategies or parse the corresponding probe data. Therefore, edge cloud nodes cannot directly and intuitively determine the performance of the corresponding data storage service through the probe data. However, since central cloud nodes have resource-scale and centrally located data centers with stronger computing power, after obtaining the probe data, they can perform statistical analysis on the probe data to generate standardized service scheduling parameters. The value of the service scheduling parameters can directly reflect the performance level of the corresponding data storage service.

[0144] According to an embodiment of the present invention, after generating service scheduling parameters, the central cloud node can send the service scheduling parameters to the edge cloud node. Thus, the edge cloud node can use the service scheduling parameters, which reflect the service status of the data storage service in the previous time interval, to schedule the received data processing request to the matching target data storage service for processing.

[0145] According to an embodiment of the present invention, by sending the service scheduling parameters calculated by the central cloud node to the edge cloud node, the edge cloud node can schedule the data processing request directly through the locally stored service scheduling parameters without communicating with the central cloud node through the public network to obtain the service scheduling strategy after receiving the data processing request. This reduces the back-to-cloud request rate of the edge cloud node and improves the security, efficiency and performance of the data processing of the edge cloud node.

[0146] Figure 5 The diagram illustrates the architecture of a network system that utilizes the data storage service access method provided in the embodiments of the present invention.

[0147] like Figure 5 As shown, 501 can represent a central cloud node, and 502 can represent an edge cloud node. Multiple data storage services 5021 are deployed in the edge cloud node 502.

[0148] An exploration module can be deployed in edge cloud node 502. This module can record the service status of multiple data storage services 5021 within a preset time interval, thereby generating exploration data. After generating the exploration data, the module can send it to the data reporting module, which then reports the data to the log center of central cloud node 501. The log center then performs statistical analysis on the exploration data and generates service scheduling parameters.

[0149] After the log center generates service scheduling parameters, it can send the service scheduling parameters to the core management module. The core management module records the list of edge cloud nodes managed by the central cloud node, the attribute information and status information of each edge cloud node, as well as the list of data storage services, data storage service type information, and feature information contained in each edge cloud node.

[0150] After receiving the service scheduling parameters, the core management and control module can send the service scheduling parameters to the management and control refresh module of the edge cloud node 502, and the management and control refresh module will store the service scheduling parameters in the cache storage area, cache middleware storage area and disk storage area of ​​the edge cloud node 502 respectively.

[0151] Edge cloud nodes can receive data processing requests from users through a unified access gateway. After receiving a data processing request, edge cloud node 502 can first determine the request volume and then rate-limit the data processing request if the request volume exceeds the request volume threshold.

[0152] If the number of data processing requests does not exceed the request threshold or the rate limiting is successful, the sender of the data processing request can be authenticated. Specifically, the data processing request can carry the identifier information of the request initiator, so that the identifier information can be matched and searched using a pre-stored user table that meets the authentication requirements. If there is a user in the user table that corresponds to the identifier information, the authentication is considered successful; otherwise, the authentication is considered unsuccessful.

[0153] After the data processing request is authenticated, the cache storage area, cache middleware storage area and disk storage area can be accessed in sequence until the service scheduling parameters of multiple pre-stored data storage services 5021 are obtained. It is then determined whether there is a target service scheduling parameter that matches the data processing request among the multiple service scheduling parameters. If there is a target service scheduling parameter that matches the data processing request among the multiple service scheduling parameters, the target data storage service corresponding to the target service scheduling parameter is used to respond to the data processing request.

[0154] Edge cloud node 502 can also receive file cleanup instructions sent by the central cloud node. The file cleanup instructions carry description information of the files to be cleaned up. After receiving the file cleanup instructions, it can access multiple data storage services 5021, identify the files to be cleaned up corresponding to the description information, and delete the found files to be cleaned up.

[0155] Figure 6 The diagram illustrates a block diagram of a central cloud node provided in an embodiment of the present invention. The central cloud node 600 includes a first acquisition module 601, a parameter generation module 602, and a parameter sending module 603.

[0156] The first acquisition module 601 is used to acquire multiple probe data sent by the edge cloud node. The multiple probe data are generated by probing the service status of multiple data storage services deployed in the edge cloud node.

[0157] The parameter generation module 602 is used to generate service scheduling parameters corresponding to multiple data storage services based on multiple probe data.

[0158] The parameter sending module 603 is used to send multiple service scheduling parameters to the edge cloud node, so that the edge cloud node can determine the target data storage service to respond to the data processing request from multiple data storage services based on the multiple service scheduling parameters.

[0159] According to an embodiment of the present invention, the central cloud node 600 further includes:

[0160] The request receiving module is used to receive data processing requests, which are sent when the edge cloud node does not have a target data storage service that responds to the data processing request.

[0161] The first service determination module is used to determine the first data storage service that matches the data processing request from multiple data storage services based on the description information of multiple data storage services deployed in the pre-stored edge cloud nodes.

[0162] The scheduling information generation module is used to generate service scheduling information based on the first data storage service.

[0163] The scheduling information sending module is used to send service scheduling information to edge cloud nodes so that the edge cloud nodes can use the first data storage service to process the corresponding data requests.

[0164] According to an embodiment of the present invention, the parameter generation module 602 includes:

[0165] The difference determination unit is used to determine the differences between multiple detection data and standard working data respectively;

[0166] The weighting factor determination unit is used to determine the weighting factors corresponding to multiple data storage services based on the difference.

[0167] The parameter determination unit is used to determine the weighting factors as service scheduling parameters.

[0168] Figure 6 The central cloud node can execute Figure 2 The implementation principle and technical effects of the data storage service access method in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the data processing apparatus in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0169] Figure 7 The diagram illustrates a block diagram of an edge cloud node provided in an embodiment of the present invention. The edge cloud node 700 includes a request receiving module 701, a parameter acquisition module 702, a judgment module 703, and a request response module 704.

[0170] The request receiving module 701 is used to receive data processing requests;

[0171] The parameter acquisition module 702 is used to acquire service scheduling parameters of multiple pre-stored data storage services;

[0172] The judgment module 703 is used to determine whether there is a target service scheduling parameter that matches the data processing request among multiple service scheduling parameters;

[0173] The request-response module 704 is used to respond to the data processing request using the target data storage service corresponding to the target service scheduling parameter when there is a target service scheduling parameter that matches the data processing request among multiple service scheduling parameters.

[0174] According to an embodiment of the present invention, the edge cloud node 700 further includes:

[0175] The request sending module is used to send data processing requests to the central cloud node when the target service scheduling parameters are not available.

[0176] The scheduling information receiving module is used to receive service scheduling information sent by the central cloud node;

[0177] The request scheduling module is used to schedule data processing requests to the first data storage service corresponding to the service scheduling information.

[0178] According to an embodiment of the present invention, the edge cloud node 700 further includes:

[0179] The parameter receiving module is used to receive service scheduling parameters for multiple data storage services sent by the central cloud node;

[0180] The storage module is used to store service scheduling parameters in the cache storage area, the cache middleware storage area, and the disk storage area, respectively.

[0181] According to an embodiment of the present invention, the parameter acquisition module 702 includes:

[0182] The access unit is used to access the cache storage area, cache middleware storage area and disk storage area in sequence until the service scheduling parameters are successfully obtained.

[0183] According to an embodiment of the present invention, the edge cloud node 700 further includes:

[0184] The instruction acquisition module is used to acquire file cleanup instructions from the central cloud node. The file cleanup instructions carry description information of the files to be cleaned up.

[0185] The service access module is used to respond to file cleanup commands, access multiple data storage services, and determine the files to be cleaned that correspond to the description information.

[0186] The file cleanup module is used to perform file cleanup operations on files to be cleaned.

[0187] According to an embodiment of the present invention, the edge cloud node 700 further includes:

[0188] The request volume determination module is used to determine the request volume of data processing requests and the request volume threshold that the edge cloud node can handle.

[0189] The filtering module is used to filter the request volume using the request volume threshold when the request volume exceeds the threshold, so as to generate data processing requests with a request volume not exceeding the threshold.

[0190] Figure 7 Edge cloud nodes can perform Figure 3 The implementation principle and technical effects of the data storage service access method in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the data processing apparatus in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0191] Figure 8 The diagram illustrates a block diagram of an edge cloud node according to another embodiment of the present invention. The edge cloud node 800 includes a detection module 801, a data transmission module 802, and a parameter receiving module 803.

[0192] The detection module 801 is used to detect the service status of multiple data storage services deployed on edge cloud nodes and generate multiple detection data.

[0193] The data sending module 802 is used to send multiple probe data to the central cloud node, so that the central cloud node can generate multiple service scheduling parameters based on the multiple probe data.

[0194] The parameter receiving module 803 is used to receive service scheduling parameters sent by the central cloud node, so as to determine the target data storage service for responding to the data processing request from multiple data storage services based on multiple service scheduling parameters.

[0195] Figure 8 Edge cloud nodes can perform Figure 4The implementation principle and technical effects of the data storage service access method in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the data processing apparatus in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0196] In one possible design, the central cloud node and / or edge cloud node provided in the embodiments of the present invention can be implemented as computing devices, such as... Figure 9 As shown, the computing device may include a storage component 901 and a processing component 902;

[0197] Storage component 901 stores one or more computer instructions, wherein one or more computer instructions are invoked and executed by processing component 902 to implement the data storage service access method provided in the embodiments of the present invention.

[0198] Of course, computing devices may also include other components, such as input / output interfaces and communication components. Input / output interfaces provide an interface between processing components and peripheral interface modules, which can be output devices, input devices, etc. Communication components are configured to facilitate wired or wireless communication between the computing device and other devices.

[0199] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0200] When the computing device is a physical device, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device.

[0201] In practical applications, this computing device can be specifically deployed as a node in a message queue system, acting as a producer, consumer, relay server, or naming server in the message queue system.

[0202] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, enables the access method for the data storage service provided in this invention.

[0203] This invention also provides a computer program product, including a computer program that, when executed by a computer, can implement the data storage service access method provided in this invention.

[0204] The processing component in the corresponding embodiments described above may include one or more processors to execute computer instructions to complete all or part of the steps in the method described above. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the method described above.

[0205] Storage components are configured to store various types of data to support operation within the device. Storage components can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0206] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0207] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0208] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for accessing a data storage service, characterized in that, Applied to a central cloud node, the method includes: Multiple probe data sent by the edge cloud node are acquired. The multiple probe data are generated by probing the service status of multiple data storage services deployed in the edge cloud node. Based on the multiple probe data, service scheduling parameters corresponding to the multiple data storage services are generated respectively; The multiple service scheduling parameters are sent to the edge cloud node so that the edge cloud node can determine the target data storage service for responding to the data processing request from the multiple data storage services based on the multiple service scheduling parameters.

2. The method according to claim 1, characterized in that, The method further includes: Receive a data processing request, which is sent when the edge cloud node does not have a target data storage service that responds to the data processing request; Based on the pre-stored description information of the plurality of data storage services deployed in the edge cloud node, and the plurality of service scheduling parameters, a first data storage service that matches the data processing request is determined from the plurality of data storage services. Service scheduling information is generated based on the first data storage service. The service scheduling information is sent to the edge cloud node so that the edge cloud node can respond to the data processing request using the first data storage service.

3. The method according to claim 1, characterized in that, The step of generating service scheduling parameters corresponding to the multiple data storage services based on the multiple probe data includes: Determine the differences between the plurality of detection data and the standard working data respectively; Based on the difference, a weighting factor corresponding to the plurality of data storage services is determined; The weighting factor is determined as the service scheduling parameter.

4. A method for accessing a data storage service, characterized in that, Applied to edge cloud nodes, the method includes: Receive data processing requests; The service scheduling parameters of multiple pre-stored data storage services are obtained. The service scheduling parameters are generated by the central cloud node and sent to the edge cloud node. The central cloud node obtains multiple probe data sent by the edge cloud node and generates service scheduling parameters corresponding to the multiple data storage services according to the multiple probe data. The multiple probe data are generated by probing the service status of the multiple data storage services deployed in the edge cloud node. Determine whether there is a target service scheduling parameter among multiple service scheduling parameters that matches the data processing request; If the target service scheduling parameters exist, the target data storage service corresponding to the target service scheduling parameters is used to respond to the data processing request.

5. The method according to claim 4, characterized in that, Also includes: If the target service scheduling parameters are not available, the data processing request will be sent to the central cloud node; Receive service scheduling information sent by the central cloud node; The data processing request is scheduled to the first data storage service corresponding to the service scheduling information.

6. The method according to claim 4, characterized in that, Also includes: The service scheduling parameters of the multiple data storage services sent by the central cloud node are received. The service scheduling parameters of the multiple data storage services are stored in the cache storage area, the cache middleware storage area, and the disk storage area, respectively. The service scheduling parameters for obtaining multiple pre-stored data storage services include: The cache storage area, the cache middleware storage area, and the disk storage area are accessed sequentially until the service scheduling parameters are successfully obtained.

7. The method according to claim 4, characterized in that, Also includes: Obtain a file cleanup instruction from the central cloud node, the file cleanup instruction carrying description information of the files to be cleaned; In response to the file cleanup command, the plurality of data storage services are accessed to determine the file to be cleaned corresponding to the description information; Perform a file cleanup operation on the files to be cleaned.

8. The method according to claim 4, characterized in that, The method further includes: Determine the request volume of the data processing request, and the threshold of the request volume that the edge cloud node can handle; If the number of requests exceeds the request volume threshold, the request volume is filtered using the request volume threshold to generate a data processing request with a request volume not exceeding the request volume threshold.

9. A method for accessing a data storage service, characterized in that, Applied to edge cloud nodes, the method includes: The service status of multiple data storage services deployed on the edge cloud nodes is detected, generating multiple detection data. Multiple probe data are sent to a central cloud node, so that the central cloud node generates service scheduling parameters corresponding to the multiple data storage services based on the multiple probe data. The service scheduling parameters sent by the central cloud node are received so as to determine the target data storage service for responding to the data processing request from among the multiple data storage services based on the multiple service scheduling parameters.

10. A computing device, characterized in that, This includes processing components and storage components; The storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the data storage service access method as described in any one of claims 1 to 3, or to implement the data storage service access method as described in any one of claims 4 to 8, or to implement the data storage service access method as described in claim 9.

11. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements the data storage service access method as described in any one of claims 1 to 3, or the data storage service access method as described in any one of claims 4 to 8, or the data storage service access method as described in claim 9.

Images