Edge Cloud-Based Data Processing Method and Device

Abstract

The embodiments of this specification provide a data processing method and apparatus based on an edge cloud. The method includes receiving a data writing request sent by a user; determining a target storage domain according to a unit identifier, and determining at least two initial edge nodes corresponding to the target storage domain; determining a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information; determining a target storage unit corresponding to the unit identifier in the target edge node, and writing the data to be written into the target storage unit. This method conducts overall management and scheduling of the storage resources of multiple edge nodes. When processing the data writing request sent by a user, it conducts global scheduling path planning according to the node attribute information of multiple edge nodes in the storage domain corresponding to the data writing request and the user attribute information, so as to achieve a globally optimal scheduling strategy and improve the user experience.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and particularly to a data processing method based on an edge cloud. Background Art

[0002] An edge cloud is a cloud computing platform built on edge infrastructure based on cloud computing technology and edge computing capabilities. By placing tasks such as network forwarding, storage, computing, and data analysis on edge nodes, it reduces response latency, alleviates cloud pressure, and reduces bandwidth costs.

[0003] In the prior art, when performing object storage based on an edge cloud, it is used according to the node dimension, and each edge node has an independent service ability for object storage. However, users need to solve many complex problems such as node resource management, resource read / write scheduling, and single-node availability operation and maintenance by themselves, with a high usage threshold and an unfriendly experience. Summary of the Invention

[0004] In view of this, the embodiments of this specification provide a data processing method based on an edge cloud. One or more embodiments of this specification also relate to a data processing device based on an edge cloud, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects existing in the prior art.

[0005] According to the first aspect of the embodiments of this specification, a data processing method based on an edge cloud is provided, including:

[0006] Receiving a data writing request sent by a user, where the data writing request carries data to be written, user attribute information, and a unit identifier of a target storage unit corresponding to the data to be written;

[0007] Determining a target storage domain according to the unit identifier, and determining at least two initial edge nodes corresponding to the target storage domain;

[0008] Determining a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information;

[0009] Determining a target storage unit corresponding to the unit identifier in the target edge node, and writing the data to be written into the target storage unit.

[0010] According to the second aspect of the embodiments of this specification, a data processing device based on an edge cloud is provided, including:

[0011] A request receiving module, configured to receive a data writing request sent by a user, where the data writing request carries data to be written, user attribute information, and a unit identifier of a target storage unit corresponding to the data to be written;

[0012] An initial node determination module, configured to determine a target storage domain according to the unit identifier, and determine at least two initial edge nodes corresponding to the target storage domain;

[0013] A target node determination module, configured to determine a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information;

[0014] A data storage module, configured to determine a target storage unit corresponding to the unit identifier among the target edge nodes, and write the data to be written into the target storage unit.

[0015] According to a third aspect of the embodiments of the present specification, there is provided a computing device, including:

[0016] A memory and a processor;

[0017] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned edge cloud-based data processing method are implemented.

[0018] According to a fourth aspect of the embodiments of the present specification, there is provided a computer-readable storage medium, which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the above-mentioned edge cloud-based data processing method are implemented.

[0019] According to a fifth aspect of the embodiments of the present specification, there is provided a computer program, where when the computer program is executed on a computer, the computer is made to execute the steps of the above-mentioned edge cloud-based data processing method.

[0020] An embodiment of the present specification implements an edge cloud-based data processing method and apparatus. The method includes receiving a data writing request sent by a user, where the data writing request carries data to be written, user attribute information, and a unit identifier of a target storage unit corresponding to the data to be written; determining a target storage domain according to the unit identifier, and determining at least two initial edge nodes corresponding to the target storage domain; determining a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information; determining a target storage unit corresponding to the unit identifier among the target edge nodes, and writing the data to be written into the target storage unit.

[0021] Specifically, the edge cloud-based data processing method performs overall management and scheduling on the storage resources of multiple edge nodes. When processing a data write request sent by a user, it performs global scheduling path planning based on the node attribute information of multiple edge nodes in the storage domain corresponding to the data write request and the user attribute information, selects a relatively optimal edge node as the target edge node to process the data write request, achieves global optimality of the scheduling policy, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a specific processing schematic diagram of an edge cloud-based data processing method provided by an embodiment of this specification;

[0023] Figure 2 is a flowchart of an edge cloud-based data processing method provided by an embodiment of this specification;

[0024] Figure 3 is a schematic diagram of a multi-engine cluster in an edge node in an edge cloud-based data processing method provided by an embodiment of this specification;

[0025] Figure 4 is a schematic diagram of the overall scheduling hierarchical relationship of an edge cloud-based data processing method provided by an embodiment of this specification;

[0026] Figure 5 is a flowchart of the processing process of an edge cloud-based data processing method provided by an embodiment of this specification;

[0027] Figure 6 is a schematic diagram of the structure of an edge cloud-based data processing device provided by an embodiment of this specification;

[0028] Figure 7 is a block diagram of the structure of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Many specific details are set forth in the following description in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementations disclosed below.

[0030] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0032] First, the noun terms related to one or more embodiments of this specification are explained.

[0033] Edge node: An edge node can be composed of multiple edge devices (such as a certain physical device, edge gateway, home gateway, etc.). An edge node can have the capabilities of network forwarding, storage, computing, data analysis, etc., and each edge node can be understood as an edge cloud.

[0034] Collaborative storage: Based on the resources and storage capabilities of multiple distributed edge nodes, through multi-node collaborative management and scheduling, a distributed storage that is location-insensitive, has a consistent experience, large capacity, high elasticity, and high reliability is constructed.

[0035] Object storage: A data storage based on objects, where data is stored as different units for management and operation; each data unit (object) has metadata description and is not saved in a folder in the form of a file.

[0036] Storage space: Bucket, a container for storing objects (Object). All objects must belong to a certain storage space; the storage space has various configuration attributes, including region, access permission, storage type, etc.; Bucket is globally unique and cannot be modified.

[0037] QPS: Queries-per-second, the query rate per second, which is a measure of how much traffic a specific query server processes within a specified time.

[0038] EC: Erasure Coding, also known as erasure code, is a redundancy protection mechanism that achieves data redundancy protection by calculating parity chunks.

[0039] Triple replication: The storage system adopts a triple replication mechanism to ensure data reliability. That is, for a certain piece of data, by default, the data is divided into data blocks of 1MB in size, and each data block is replicated into 3 copies. Then, according to a certain distributed storage algorithm, these copies are stored on different nodes in the cluster.

[0040] LDNS: Local Dns, that is, local DNS, the domain name system.

[0041] In this specification, a data processing method based on edge cloud is provided. This specification also relates to a data processing device based on edge cloud, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.

[0042] See Figure 1 , Figure 1 which shows a specific processing schematic diagram of a data processing method based on edge cloud provided according to an embodiment of this specification.

[0043] Figure 1 It includes user 102, node center control system 104, and edge node cluster 106; among them, the edge node cluster 106 includes multiple edge nodes.

[0044] Specifically in implementation, the node center control system 104 receives a data read / write request sent by user 102, and determines a target edge node from the edge node cluster 106 according to the data read / write request for data reading.

[0045] Taking the data write request as an example, the node center control system 104 receives a data write request sent by user 102. The data write request carries the data to be written, user attribute information, and the storage space (i.e., Bucket) corresponding to the data to be written. Among them, Bucket usually includes a globally unique Bucket name, Bucket lifecycle (for example, cameras are usually saved for seven days, and those exceeding seven days are automatically eliminated), and the storage domain to which the Bucket belongs, which is generally specified by the background during initialization. The external sales form may be national, regional, provincial, etc., and it will be mapped to the corresponding storage domain internally.

[0046] Determine the corresponding target storage domain according to the storage space corresponding to the data to be written, and obtain multiple edge nodes corresponding to the target storage domain; filter the multiple edge nodes, for example, first filter the edge nodes with abnormal service status (such as network cut off, pre-offline, etc.), and secondly filter the edge nodes of the same operator (for example, if the user's client is China Unicom, then filter out the edge nodes of China Mobile, China Telecom, etc.), the same large region (for example, if the large region where the user is located is the South China region, then filter out the edge nodes of other large regions), the dedicated line interconnection (for example, if the user can only use the dedicated line, then filter out the edge nodes of the public network, etc.); then, according to sorting rules such as geographical location sorting, water level sorting, cost sorting, user service form sorting, file form sorting, etc., sort the edge nodes remaining after filtering, and select the better edge nodes to write the data to be written into the physical Bucket corresponding to the Bucket.

[0047] The data processing method based on edge cloud provided by the embodiments of this specification uniformly manages a series of edge nodes through a node center control system for global resource planning; through node filtering, node sorting, etc., select better edge nodes for data processing, realize optimal scheduling of the whole network resources, and improve the user experience.

[0048] See Figure 2 , Figure 2 shows a flowchart of a data processing method based on edge cloud provided by an embodiment of this specification, which specifically includes the following steps.

[0049] Step 202: Receive a data writing request sent by the user.

[0050] Wherein, the data writing request carries the data to be written, user attribute information, and the unit identifier of the target storage unit corresponding to the data to be written.

[0051] Specifically, the target storage unit can be understood as the above-mentioned storage space Bucket; the unit identifier of the target storage unit can be understood as a globally unique Bucket name; and each Bucket has its corresponding storage domain.

[0052] The data to be written can be understood as data of any format, any size, and any type, such as picture data, video data, or text data, etc.; the user attribute information includes but is not limited to the user's name, level, and geographical location, etc.

[0053] Step 204: Determine the target storage domain according to the unit identifier, and determine at least two initial edge nodes corresponding to the target storage domain.

[0054] Specifically, after receiving the data writing request sent by the user, in response to the data writing request, determine the target storage domain corresponding to the Bucket according to the unit identifier carried in the data writing request, and obtain at least two initial edge nodes corresponding to the target storage domain (which can be understood as the edge nodes in the above embodiments).

[0055] In practical applications, each target storage domain corresponds to multiple initial edge nodes, and each initial edge node can belong to multiple target storage domains.

[0056] Among them, the edge nodes in the embodiments of this specification can be any edge heterogeneous nodes or central nodes, that is, any edge nodes that achieve cloud-edge collaboration and cloud-edge integration are acceptable, and users do not need to care about the specific storage location and node status.

[0057] Step 206: Determine a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information.

[0058] Specifically, after determining at least two initial edge nodes, a target edge node can be determined from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information.

[0059] In practical applications, in order to improve the data writing rate and enhance the user experience, the node center control system will select a better edge node (i.e., the target edge node) from at least two initial edge nodes for data processing; therefore, it is necessary to screen and obtain the target edge node according to the node attribute information of the at least two initial edge nodes and the user attribute information. The specific implementation method is as follows:

[0060] The determining a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information includes:

[0061] Determine candidate edge nodes from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes, where the node attribute information includes node current status information, node current attribute information, and node current running information;

[0062] Determine a target edge node from the candidate edge nodes according to the node current attribute information, node current running information of the candidate edge nodes, and the user attribute information.

[0063] Among them, the current node status information includes, but is not limited to, online, offline, pre-offline, pre-online, abnormal detection status, etc.; the current node attribute information includes, but is not limited to, geographical location, region, operator, network, self-built / cloud connection, bearing scale, cost, etc.; the current operation information of the node (i.e., the node water level) includes, but is not limited to, real-time bandwidth, dedicated line bandwidth, storage space, QPS, lock quantity, etc.

[0064] In specific implementation, after determining at least two initial edge nodes, first determine candidate edge nodes from the at least two initial edge nodes according to the current node status information, current node attribute information, and current operation information of each initial edge node in the at least two initial edge nodes; then select target edge nodes from the candidate edge nodes according to the current node attribute information, current operation information of the candidate nodes, and user attribute information.

[0065] The data processing method based on edge cloud provided by the embodiments of this specification realizes the acquisition of better target edge nodes through two implementation steps of determining candidate edge nodes and target edge nodes, so as to improve the writing efficiency of subsequent data writing through the target edge nodes.

[0066] In specific application, candidate edge nodes are determined by filtering and screening at least two initial edge nodes. After filtering out some unavailable nodes to obtain candidate edge nodes, and then processing and selecting target edge nodes from a small number of candidate edge nodes, the node determination calculation amount can be greatly reduced. The specific implementation method is as follows:

[0067] Determining candidate edge nodes from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes includes:

[0068] Filter out candidate edge nodes from the at least two initial edge nodes according to the current node status information, current node attribute information, and current operation information of the at least two initial edge nodes.

[0069] That is, determine candidate edge nodes from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes; it can be understood as filtering out candidate edge nodes from the at least two initial edge nodes according to the current node status information, current node attribute information, and current operation information of the at least two initial edge nodes.

[0070] For example, if there are 100 at least two initial edge nodes, 50 candidate edge nodes are filtered out from the 100 initial edge nodes according to the current node status information, current node attribute information, and current operation information of the at least two initial edge nodes.

[0071] For the specific screening process, the first screening can be performed based on the current node status information of at least two initial edge nodes, the second screening can be performed based on the current node attribute information, and the third screening can be performed based on the current node running information, etc. In the embodiments of this specification, there is no fixed order for screening the current node status information, the current node attribute information, and the current node running information. Just screen all of the current node status information, the current node attribute information, and the current node running information once. The specific implementation method is as follows:

[0072] Screening candidate edge nodes from the at least two initial edge nodes according to the current node status information, the current node attribute information, and the current node running information of the at least two initial edge nodes includes:

[0073] Filtering out the initial edge nodes that do not match the user attribute information from the at least two initial edge nodes according to the current node attribute information;

[0074] Screening out the initial edge nodes that meet the first screening condition from the filtered initial edge nodes according to the current node status information;

[0075] Screening out the candidate edge nodes that meet the second screening condition from the initial edge nodes that meet the first screening condition according to the current node running information.

[0076] Among them, the first screening condition and the second screening condition can be set according to actual applications, and the embodiments of this specification do not make any limitations in this regard. For example, the first screening condition can be to screen out the initial edge nodes with non-abnormal node status, and the second screening condition can be to screen out the initial edge nodes with the node water level meeting the data writing request, etc.

[0077] During specific implementation, filtering out the initial edge nodes that do not match the user attribute information from the at least two initial edge nodes according to the current node attribute information can be understood as filtering out the initial edge nodes in different regions and different operators from the user's client from at least two edge nodes according to the region and operator in the current node attribute information. For example, if the user's client belongs to the South China region and the Unicom network, then filter out the initial edge nodes that do not belong to the South China region and do not belong to the Unicom network from at least two initial edge nodes. At this time, if the user is a dedicated line user, then filter out the initial edge nodes that do not belong to dedicated line interconnection.

[0078] Based on the current status information of the node, initial edge nodes that meet the first screening condition are selected from the filtered initial edge nodes. It can be understood that according to the online, offline, pre-offline, pre-online, abnormal detection status, etc. in the current status information of the node, initial edge nodes with normal node status such as online and pre-online are selected from the initial edge nodes obtained after the above filtering, that is, initial edge nodes with offline, pre-offline, and abnormal detection status are filtered out.

[0079] Based on the current running information of the node, candidate edge nodes that meet the second screening condition are selected from the initial edge nodes that meet the first screening condition. It can be understood that according to the real-time bandwidth, dedicated line bandwidth, storage space, QPS, lock quantity, etc. in the current running information of the node, initial edge nodes whose node water level meets the processing of data write requests are selected from the initial edge nodes obtained after the above screening.

[0080] In the data processing method based on edge cloud provided in the embodiments of this specification, after determining at least two initial edge nodes, in order to save the computational effort of subsequent target edge nodes, starting from multiple dimensions such as filtering operators, regions, dedicated line interconnection, abnormal service status, and abnormal node water level, candidate edge nodes are filtered from at least two initial edge nodes to ensure the efficiency and accuracy of selecting target edge nodes from subsequent candidate edge nodes, and to avoid wasting computational resources by processing unavailable edge nodes.

[0081] In addition, after each filtering of the initial edge nodes, the target storage domain is scaled up or down according to the number of nodes of the filtered initial edge nodes, so as to ensure that there are sufficient candidate edge nodes in the subsequent process and a better target edge node can be determined. The specific implementation method is as follows:

[0082] When the number of nodes of the filtered initial edge nodes, the initial edge nodes that meet the first screening condition, or the candidate edge nodes that meet the second screening condition is less than the first quantity threshold, the edge nodes and storage units are expanded for the target storage domain.

[0083] Among them, the first quantity threshold can be set according to actual applications, and the embodiments of this specification do not make any limitations in this regard. For example, the first quantity threshold is 1.

[0084] In practical applications, when the number of nodes of the initial edge nodes filtered in any dimension is less than 1, it is necessary to expand the edge nodes and storage units for the target storage domain to ensure that there are candidate edge nodes for selecting target edge nodes in the subsequent process, and to avoid the situation where there are no edge nodes to process data write requests, thereby improving the user experience.

[0085] After filtering and screening to obtain candidate edge nodes, a target edge node can be selected from the candidate edge nodes to process the data write request. The specific implementation method is as follows:

[0086] Determining a target edge node from the candidate edge nodes according to the current node attribute information, current node running information, and the user attribute information of the candidate edge nodes includes:

[0087] When it is determined that the number of candidate edge nodes is greater than or equal to a second quantity threshold, a target edge node is determined from the candidate edge nodes according to the current node attribute information, current node running information, and the user attribute information of the candidate edge nodes.

[0088] Wherein, the second quantity threshold can be set according to actual applications, and the embodiments of this specification do not make any limitations thereto. For example, the second quantity threshold is 2.

[0089] In practical applications, the acquisition of the target edge node is determined according to the sorting result after sorting the candidate edge nodes in various dimensions. Therefore, only when the number of candidate edge nodes is relatively large can the sorting operation be implemented. When there is only one candidate edge node, there is no sorting operation.

[0090] Taking the second quantity threshold as 2 as an example, when it is determined that the number of candidate edge nodes is greater than or equal to the second quantity threshold 2, a target edge node will be determined from the candidate edge nodes according to the current node attribute information, current node running information, and user attribute information of the candidate edge nodes. When the number of candidate edge nodes is less than the second quantity threshold 2, it means that there is only one candidate edge node, and then this candidate edge node needs to be used as the target edge node to process the subsequent data write request.

[0091] In specific implementation, when determining a target edge node from the candidate edge nodes according to the current node attribute information, current node running information, and user attribute information of the candidate edge nodes, different node sorting strategies are configured for each user according to the differences of users, so as to select a better target edge node for each user according to this node sorting strategy to process the data write request and improve the user experience. The specific implementation method is as follows:

[0092] Determining a target edge node from the candidate edge nodes according to the current node attribute information, current node running information, and the user attribute information of the candidate edge nodes includes:

[0093] Determine a node sorting strategy according to the current node attribute information, the current node running information of the candidate edge nodes, and the user attribute information;

[0094] Sort the candidate edge nodes according to the node sorting strategy, and determine the target edge node from the sorted candidate edge nodes.

[0095] In practical applications, when the user attribute information is different, the node sorting strategies determined according to the current node attribute information and the current node running information of the candidate edge nodes are also different; for example, if it is determined according to the user attribute information that the user is an ordinary user, then the node sorting strategy determined according to the current node attribute information and the current node running information of the candidate edge nodes can be: sorting the nodes according to cost, service quality, and proximity; if it is determined according to the user attribute information that the user is a user with a higher level and has a higher requirement for service quality, then the node sorting strategy determined according to the current node attribute information and the current node running information of the candidate edge nodes is: sorting the nodes according to service quality, cost, and proximity, etc.

[0096] The above are only two ways to determine the node sorting strategy. In practical applications, there are also other node sorting strategies, such as sorting the nodes according to cost, service quality, file size, etc. For example, first sort the candidate edge nodes according to cost, then select the top 20 candidate nodes after sorting and sort them according to service quality, and finally select the top 10 candidate nodes after sorting by service quality and sort them according to file size, etc.

[0097] Of course, in practical applications, when the number of candidate edge nodes is relatively large, in order to save computing resources, a part of the edge nodes can also be screened from the candidate edge nodes, and the node sorting and the determination of the target edge node are performed according to the node sorting strategy.

[0098] Step 208: Determine the target storage unit corresponding to the unit identifier among the target edge nodes, and write the data to be written into the target storage unit.

[0099] Specifically, after determining the target edge node, the target storage unit in the target edge node, that is, the physical Bucket, is determined according to the unit identifier, and then the data to be written is written into the target storage unit.

[0100] That is, in the data processing method based on edge cloud provided in the embodiments of this specification, a data processing request will go through two stages: node filtering and node sorting, to obtain a better target edge node, and subsequent data write request processing can be performed according to the better target edge node, improving the user experience.

[0101] Applied to the selection of edge nodes, the data processing method based on edge cloud provided in the embodiments of this specification globally manages and schedules the storage resources of multiple edge nodes. When processing a data write request sent by a user, a global scheduling path is planned according to the node attribute information of multiple edge nodes in the storage domain corresponding to the data write request and the user attribute information, and a relatively optimal edge node is selected as the target edge node to process the data write request, so as to achieve global optimality of the scheduling strategy and improve the user experience.

[0102] In actual applications, after obtaining a relatively optimal edge node, the data write request will access the access gateway of the relatively optimal edge node, and at the same time, it will also go through the node scheduling process (usually N physical clusters, that is, engine clusters, are deployed within a single node to ensure high service availability and adapt to different scenarios). Node-level scheduling usually considers multiple factors such as the weight ratio between engines, QPS water level, and engine status to select a relatively optimal engine cluster for data processing and improve data processing efficiency. The specific implementation method is as follows:

[0103] Determining the target storage unit corresponding to the unit identifier in the target edge node and writing the data to be written into the target storage unit includes:

[0104] Determining the target storage unit corresponding to the unit identifier in the target edge node and at least two engine clusters in the target edge node;

[0105] Determining a target engine cluster according to the current running parameters of each engine cluster in the at least two engine clusters;

[0106] Writing the data to be written into the target storage unit according to the target engine cluster.

[0107] Among them, the current running parameters of the engine include but are not limited to weight configuration, QPS peak shaving, engine load / disaster tolerance, storage space, file type, life cycle, etc.

[0108] Specifically, after obtaining the target edge node, determine the target storage unit corresponding to the unit identifier in the target edge node and at least two engine clusters in the target edge node; then, according to the current running parameters of each engine cluster in the at least two engine clusters, select a relatively optimal target engine cluster; finally, write the data to be written into the target storage unit according to the relatively optimal target engine cluster.

[0109] Of course, in order to ensure high service availability, in the case where the target edge node is unavailable or the target engine cluster is unavailable, other relatively optimal target edge nodes or target engine clusters can be selected according to the current running information of other candidate edge nodes or the current running parameters of other engine clusters.

[0110] See Figure 3 , Figure 3 which shows a schematic diagram of a multi-engine cluster in an edge node in a data processing method based on an edge cloud provided by an embodiment of this specification.

[0111] Figure 3 The edge node in [[ ]] includes three engine clusters (LDNS), namely Engine Cluster 1, Engine Cluster 2, and Engine Cluster 3. After determining the optimal edge node, the data write request accesses the edge access gateway of this edge node. Meanwhile, through the in-node scheduling process, multi-engine scheduling is performed among Engine Cluster 1, Engine Cluster 2, and Engine Cluster 3 to select the optimal engine cluster.

[0112] The processing of the data write request by the target edge node includes processing in two dimensions. One dimension is the processing of selecting the optimal target engine cluster, and the other dimension is the processing of selecting a suitable data storage method for data writing. Specifically, the processing of selecting the optimal target engine cluster can be determined according to the above embodiment, and the processing of selecting a suitable data storage method for data writing can be understood as determining the writing method according to the attribute information of the data to be written (such as data size, data type, etc.). Then, when storing the data to be written according to the target engine cluster, the data to be written can be stored in the target storage unit according to the corresponding writing method. The specific implementation method is as follows:

[0113] Writing the data to be written into the target storage unit according to the target engine cluster includes:

[0114] Writing the data to be written into the target storage unit through the writing method corresponding to the data to be written according to the target engine cluster.

[0115] Among them, the writing method includes but is not limited to dynamic EC writing or triple-copy writing, etc.

[0116] Specifically, first, determine the writing method corresponding to the data to be written according to the attribute information of the data to be written (such as file size), Bucket attribute, or read-write ratio, etc.; then, write the data to be written into the target storage unit according to the target engine cluster in the writing method corresponding to the data to be written (such as dynamic EC or triple-copy) to ensure the stability and security of the written data.

[0117] In practical applications, the node center control system includes a real-time water level statistics module. When specifically applied, the current operating information of each initial edge node is obtained according to the real-time water level statistics module, so as to ensure that the subsequent node center control system can perform reasonable node scheduling based on the current operating information of each initial edge node obtained by the real-time water level statistics module. The specific implementation method is as follows:

[0118] The data processing method based on edge cloud further includes:

[0119] Receiving the current operating information of the at least two initial edge nodes sent by the real-time water level statistics module, where the current operating information of the node includes the current bandwidth of the node, the query rate per second of the node, and the current storage space of the node.

[0120] In addition, the embodiments of this specification can also process the data reading request sent by the user. By carrying the unit identifier of the target storage unit and the file name in the data reading request, the data to be read can be quickly and accurately read. The specific implementation method is as follows:

[0121] After writing the data to be written into the target storage unit, it further includes:

[0122] Receiving the data reading request sent by the user, where the unit identifier of the target storage unit of the data to be read and the file name are carried in the data reading request;

[0123] Determining the target storage unit according to the unit identifier, and reading the data to be read from the target storage unit according to the file name.

[0124] Among them, the user sending the data reading request can be the user in the above embodiments, or other users who have stored data. The unit identifier of the target storage unit can refer to the introduction in the above embodiments and will not be elaborated here.

[0125] In the data processing method based on edge cloud provided by the embodiments of this specification, the scheduler (node center control system) will perform global scheduling path planning according to the real-time water level, achieve global optimization of the scheduling strategy, such as proximity, etc., and improve the efficiency of data processing (data writing or data reading, etc.); at the same time, if a single node (edge node) has a problem, the scheduler will filter the abnormal node in a timely manner by counting the status of the nodes with the whole network real-time statistics module, and use the normal nodes for data processing to ensure the high availability of the service; at the same time, through the deployment scheme of the multi-engine cluster within the node, the high availability of the service within a single node is ensured.

[0126] See Figure 4 , Figure 4The figure shows a schematic diagram of the overall scheduling hierarchy of a data processing method provided by an embodiment of this specification based on edge cloud.

[0127] Figure 4 It includes a resource pool 402, a storage domain 404, node scheduling 406, and in-node scheduling 408. Among them, the resource pool 402 and the storage domain 404 are introductions of resource dimensions; the node scheduling 406 and the in-node scheduling 408 are specific node scheduling strategies.

[0128] Among them, the resource pool 402 includes node attributes, node status, and node water levels. The node attributes include geographical location (the geographical location of the node, such as Province A), the region to which it belongs (the geographical location of the node, such as the South China region), operator (such as China Unicom, China Mobile, China Telecom), network (the maximum and minimum carrying capacities of the uplink and downlink bandwidths), self-built / cloud connection (self-built computer room or a computer room rented from other operators), carrying scale (the maximum carrying capacity), cost, etc.; the node status includes online, pre-online, offline, pre-offline, and detection status abnormal (such as network disconnection); the node water levels include real-time bandwidth / special line bandwidth, storage (storage space), QPS (queries per second), lock quantity (resources reserved for special customers), etc.

[0129] The storage domain 404 includes pre-partitioning of the storage domain, storage domain management, and dynamic scaling of the storage domain. The pre-partitioning of the storage domain includes business dimension, region dimension, user dimension (storage domain specifically marked for certain users), custom, etc.; the storage domain management includes the linkage between the storage domain and the node status (a storage domain corresponds to multiple nodes, and each node can correspond to multiple storage domains), and the priority management of the storage domain (the priority of each node in different storage domains is different); the dynamic scaling of the storage domain includes cross-node physical Bucket scaling (when the storage domain cannot meet the data processing requirements, some nodes can be expanded).

[0130] Node scheduling 406 includes node model scheduling, service model scheduling, storage domain linkage, policy execution and correction; and node model scheduling includes location, traffic, storage space, cost (that is, truly taking these factors into account according to location, traffic, storage space, cost, etc. to perform node scheduling); service model scheduling includes service model (node model and service model are in a parallel relationship. For example, without any special configuration, the node scheduling scheme of the general node model may be used. For example, if a user is close to a certain node, all requests are directed to this node. For the service model, node scheduling is customized according to service requirements), user model (node scheduling is performed according to the user's attribute information. For example, if a user has written data to a certain node before, requests can continue to be directed to this node), file model (for example, picture files are directed to nodes with lower quality, and video files are directed to nodes with higher quality, etc.); storage domain linkage includes combining storage domain, real-time water level / safety water level of nodes, disaster recovery, etc. (that is, storage scaling); policy execution and correction includes service quality statistics, real-time correction (implementing adjustment of node scheduling policies. For example, if the response time of the selected target edge node is very long, real-time adjustment and correction can be performed).

[0131] Intra-node scheduling 408 includes multi-engine scheduling, physical Bucket scheduling (physical bucket scheduling), dynamic EC (erasure code) / triple replication. And multi-engine scheduling includes weight configuration, QPS peak shaving (query per second peak shaving), engine load / disaster recovery, storage space, file type, life cycle, etc. (that is, selecting a better target engine cluster according to these parameters); physical Bucket scheduling includes dynamic scaling / balancing of the number of files (adding or deleting physical Buckets to ensure scheduling balance); dynamic EC / triple replication includes Bucket attributes (bucket attributes), file size, read / write ratio, etc. (that is, selecting an appropriate data processing method according to these parameters).

[0132] The data processing method based on edge cloud provided by the embodiments of this specification divides nodes through a resource pool and a storage domain, and then realizes optimal scheduling of network-wide resources through two major scheduling strategies of node scheduling and intra-node scheduling, improving the user experience.

[0133] The following combines the attached Figure 5 , taking the scheduling application of the data processing method based on edge cloud provided by this specification in edge nodes as an example, further illustrates the data processing method based on edge cloud. Among them, Figure 5 shows the processing procedure flowchart of a data processing method based on edge cloud provided by an embodiment of this specification, specifically including the following steps.

[0134] Step 502: The node central control system receives a data writing request sent by a user. The data writing request carries the data to be written, user attribute information, and the bucket identifier of the physical Bucket corresponding to the data to be written.

[0135] Step 504: The node central control system determines the target storage domain according to the bucket identifier and determines at least two initial edge nodes corresponding to the target storage domain.

[0136] Step 506: The node central control system filters and sorts the at least two initial edge nodes according to the node attribute information and user attribute information of the at least two initial edge nodes to obtain the target edge node.

[0137] Specifically, for the specific determination method of the target edge node, refer to the introduction of the above embodiments. For example, Figure 5 as shown in the detailed internal process of the scheduler on the left, first perform node filtering according to location scheduling (region, operator, dedicated line, etc.), water level scheduling (upstream bandwidth, downstream bandwidth, QPS queries per second, storage space, etc.), etc., and then perform edge node sorting according to user scheduling (service attribute, user priority, gray control, cost control) or file model (file size, file type, save time) to obtain the target edge node.

[0138] That is, in the node filtering stage, for location scheduling: first, according to the user's requirements, filter out the regions required by the user (for example, if the user specifies Province A when creating resources, then this request filters out the edge nodes outside Province A); secondly, according to the operator, filter out the edge nodes that do not match the operator of the user's client this time (for example, if the user's client is China Mobile, and there are 3 China Mobile edge nodes, 3 China Unicom edge nodes, and 3 China Telecom edge nodes in Province A, then the China Telecom and China Unicom edge nodes will be filtered out); finally, filter out the dedicated line edge nodes of some other users.

[0139] For water level scheduling: filter out the edge nodes whose current water level exceeds the preset water level threshold (such as 95%), and exclude the edge nodes that do not meet the conditions by filtering the storage space water level, the upstream and downstream bandwidth water levels, and the QPS queries per second water level.

[0140] In the node sorting stage, an independent policy can be configured for the user, or the default policy can be adopted. Among them, the default policy can be: water level first, that is, preferentially select edge nodes with low water levels. If the water levels of all edge nodes are relatively low at this time, for example, the water levels of the three mobile edge nodes and the three edge nodes of the three lines (that is, the edge nodes supported by mobile, Unicom, and Telecom) are all 50%; then a second sorting is performed: cost sorting. At this time, because the cost of the three lines is too high, mobile nodes will be preferentially selected; next, the remaining policy sorting will be performed, such as geographical location first. At this time, if one of the three eligible mobile edge nodes is significantly closer to the user, then the edge node with a closer distance will be preferentially selected. Assume that all three edge nodes are relatively close, then the next round of sorting will be performed, such as sorting according to the scheduling concentration (that is, if the requests of this user have all been scheduled to a certain edge node before, then it is inclined to schedule to this edge node next). And so on, until a better edge node is selected. If there are still multiple eligible nodes after all sorting policies are completed, then random selection can be made.

[0141] Still refer to Figure 5 As shown in the detailed internal process of the scheduler on the left, after determining the target edge node, the engine parameters (weight, peak shaving and valley filling of queries per second (similar to the water level, each engine supports a limited QPS. If the QPS is too high at this time point, peak shaving is required to lower the QPS), space, central processing unit, read and write) in the engine model are adopted inside the target edge node to determine the target engine cluster, and the data writing method is determined according to the parameters (space, number of files, type (triple copy / erasure code), etc.) in the bucket model.

[0142] Finally, the node center control system can write the data to be written into the physical Bucket according to the target engine cluster through the writing method corresponding to the data to be written, where the physical Bucket is the physical Bucket corresponding to the bucket identifier.

[0143] In specific implementation, the whole-network real-time water level statistics module in the node center control system will detect the water levels of each edge node in real time for the scheduler to plan a better scheduling path.

[0144] The data processing method based on edge cloud provided in the embodiments of this specification uniformly manages a series of edge nodes, conducts global resource planning, and at the same time combines the global real-time water level statistics module to perform better scheduling of the whole-network resources, and at the same time realizes the high availability and disaster tolerance of the overall service. And a deployment scheme of multiple physical clusters is adopted inside the node, and global optimization between multiple clusters is realized through in-node scheduling. At the same time, it has good disaster tolerance for situations such as single-cluster anomalies.

[0145] Corresponding to the above method embodiments, this specification also provides embodiments of a data processing apparatus based on an edge cloud. Figure 6 The following shows a schematic structural diagram of a data processing apparatus based on an edge cloud provided by an embodiment of this specification. As Figure 6 shown, the apparatus includes:

[0146] A request receiving module 602, configured to receive a data writing request sent by a user, where the data writing request carries data to be written, user attribute information, and a unit identifier of a target storage unit corresponding to the data to be written;

[0147] An initial node determining module 604, configured to determine a target storage domain according to the unit identifier, and determine at least two initial edge nodes corresponding to the target storage domain;

[0148] A target node determining module 606, configured to determine a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information;

[0149] A data storage module 608, configured to determine a target storage unit corresponding to the unit identifier among the target edge nodes, and write the data to be written into the target storage unit.

[0150] Optionally, the target node determining module 606 is further configured to:

[0151] Determine candidate edge nodes from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes, where the node attribute information includes node current state information, node current attribute information, and node current operation information;

[0152] Determine a target edge node from the candidate edge nodes according to the node current attribute information, node current operation information of the candidate edge nodes, and the user attribute information.

[0153] Optionally, the target node determining module 606 is further configured to:

[0154] Screen out candidate edge nodes from the at least two initial edge nodes according to the node current state information, node current attribute information, and node current operation information of the at least two initial edge nodes.

[0155] Optionally, the target node determining module 606 is further configured to:

[0156] Filter out the initial edge nodes that do not match the user attribute information from the at least two initial edge nodes according to the current attribute information of the node;

[0157] Screen out the initial edge nodes that meet the first screening condition from the filtered initial edge nodes according to the current state information of the node;

[0158] Screen out candidate edge nodes that meet the second screening condition from the initial edge nodes that meet the first screening condition according to the current running information of the node.

[0159] Optionally, the device further includes:

[0160] An expansion module, configured to:

[0161] In the case where the number of nodes of the filtered initial edge nodes, the initial edge nodes that meet the first screening condition, or the candidate edge nodes that meet the second screening condition is less than the first quantity threshold, perform edge node and storage unit expansion for the target storage domain.

[0162] Optionally, the target node determination module 606 is further configured to:

[0163] In the case where the number of nodes of the candidate edge nodes is greater than or equal to the second quantity threshold, determine the target edge node from the candidate edge nodes according to the current attribute information of the nodes of the candidate edge nodes, the current running information of the nodes, and the user attribute information.

[0164] Optionally, the target node determination module 606 is further configured to:

[0165] Determine a node sorting strategy according to the current attribute information of the nodes of the candidate edge nodes, the current running information of the nodes, and the user attribute information;

[0166] Sort the candidate edge nodes according to the node sorting strategy, and determine the target edge node from the sorted candidate edge nodes.

[0167] Optionally, the data storage module 608 is further configured to:

[0168] Determine the target storage unit corresponding to the unit identifier among the target edge nodes, and at least two engine clusters in the target edge nodes;

[0169] Determine the target engine cluster according to the current running parameters of each engine cluster in the at least two engine clusters;

[0170] Write the data to be written into the target storage unit according to the target engine cluster.

[0171] Optionally, the data storage module 608 is further configured to:

[0172] Write the data to be written into the target storage unit according to the target engine cluster by a write method corresponding to the data to be written.

[0173] Optionally, the device further includes:

[0174] A water level statistics module, configured to:

[0175] Receive the current running information of the at least two initial edge nodes sent by the real-time water level statistics module, where the current running information of the node includes the current bandwidth of the node, the query rate per second of the node, and the current storage space of the node.

[0176] Optionally, the device further includes:

[0177] A data reading module, configured to:

[0178] Receive a data reading request sent by a user, where the data reading request carries the unit identifier of the target storage unit of the data to be read and the file name;

[0179] Determine the target storage unit according to the unit identifier, and read the data to be read from the target storage unit according to the file name.

[0180] The data processing method based on edge cloud provided in the embodiments of this specification overall manages and schedules the storage resources of multiple edge nodes. When processing a data write request sent by a user, a global scheduling path is planned according to the node attribute information of multiple edge nodes and the user attribute information in the storage domain corresponding to the data write request, and a better edge node is selected as the target edge node to process the data write request, realizing global optimization of the scheduling strategy and improving the user experience.

[0181] The above is a schematic solution of a data processing device based on edge cloud in this embodiment. It should be noted that the technical solution of the data processing device based on edge cloud and the technical solution of the above-mentioned data processing method based on edge cloud belong to the same concept. For the details not described in the technical solution of the data processing device based on edge cloud, reference can be made to the description of the technical solution of the above-mentioned data processing method based on edge cloud.

[0182] Figure 7The structural block diagram of a computing device 700 provided according to an embodiment of this specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0183] The computing device 700 further includes an access device 740, and the access device 740 enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interfaces (e.g., Network Interface Card (NIC)), such as IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, Worldwide Interoperability for Microwave Access (Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, and so on.

[0184] In an embodiment of this specification, the above components of the computing device 700 and Figure 7 other components not shown may also be connected to each other, for example, via a bus. It should be understood that Figure 7 the shown structural block diagram of the computing device is only for illustrative purposes and is not a limitation on the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0185] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 700 can also be a mobile or stationary server.

[0186] Among them, the processor 720 is used to execute the following computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above data processing method based on the edge cloud are implemented.

[0187] The above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above data processing method based on the edge cloud belong to the same concept. For the details not described in the technical solution of the computing device, reference can be made to the description of the technical solution of the above data processing method based on the edge cloud.

[0188] One embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-described edge-cloud-based data processing method.

[0189] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above-described edge-cloud-based data processing method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above-described edge-cloud-based data processing method.

[0190] One embodiment of this specification also provides a computer program, which, when executed on a computer, causes the computer to execute the steps of the above-described edge-cloud-based data processing method.

[0191] The above is a schematic solution of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the above-described edge-cloud-based data processing method belong to the same concept. For the details not described in detail in the technical solution of the computer program, reference can be made to the description of the technical solution of the above-described edge-cloud-based data processing method.

[0192] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0193] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0194] It should be noted that, for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this specification are not limited by the described action sequence, because according to the embodiments of this specification, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential for the embodiments of this specification.

[0195] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The alternative embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method based on edge cloud, applied to a node center control system, including: Receiving a data writing request sent by a user, where the data writing request carries data to be written, user attribute information, and a unit identifier of a target storage unit corresponding to the data to be written; Determining a target storage domain according to the unit identifier, and determining at least two initial edge nodes corresponding to the target storage domain, where the target storage domain corresponds to multiple of the initial edge nodes, each of the initial edge nodes belongs to multiple of the target storage domains, and the at least two initial edge nodes belong to the same edge node cluster; Determining a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information, where after determining the target edge node from the at least two initial edge nodes, it further includes: when the number of nodes filtered in any dimension of the initial edge nodes is less than a first threshold, expanding edge nodes and storage units for the target storage domain; Determining a target storage unit corresponding to the unit identifier in the target edge node, and writing the data to be written into the target storage unit, where determining the target storage unit corresponding to the unit identifier in the target edge node and writing the data to be written into the target storage unit includes: determining the target storage unit corresponding to the unit identifier in the target edge node, and at least two engine clusters in the target edge node; determining a target engine cluster according to the current running parameters of each engine cluster in the at least two engine clusters; writing the data to be written into the target storage unit according to the target engine cluster.

2. The data processing method based on edge cloud according to claim 1, where determining the target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information includes: Determining candidate edge nodes from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes, where the node attribute information includes node current state information, node current attribute information, and node current running information; Determining a target edge node from the candidate edge nodes according to the node current attribute information, node current running information of the candidate edge nodes, and the user attribute information.

3. The data processing method based on edge cloud according to claim 2, where determining the candidate edge nodes from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes includes: Filtering out candidate edge nodes from the at least two initial edge nodes according to the node current state information, node current attribute information, and node current running information of the at least two initial edge nodes.

4. The data processing method based on edge cloud according to claim 2, wherein screening candidate edge nodes from the at least two initial edge nodes according to the node current state information, node current attribute information, and node current operation information of the at least two initial edge nodes includes: Filtering out the initial edge nodes that do not match the user attribute information from the at least two initial edge nodes according to the node current attribute information; Screening out the initial edge nodes that meet the first screening condition from the filtered initial edge nodes according to the node current state information; Screening out candidate edge nodes that meet the second screening condition from the initial edge nodes that meet the first screening condition according to the node current operation information.

5. The data processing method based on edge cloud according to claim 2, wherein determining a target edge node from the candidate edge nodes according to the node current attribute information, node current operation information, and user attribute information of the candidate edge nodes includes: In the case where the number of nodes of the candidate edge nodes is greater than or equal to the first quantity threshold, determining a target edge node from the candidate edge nodes according to the node current attribute information, node current operation information, and user attribute information of the candidate edge nodes.

6. The data processing method based on edge cloud according to claim 5, wherein determining a target edge node from the candidate edge nodes according to the node current attribute information, node current operation information, and user attribute information of the candidate edge nodes includes: Determining a node sorting strategy according to the node current attribute information, node current operation information, and user attribute information of the candidate edge nodes; Sorting the candidate edge nodes according to the node sorting strategy and determining a target edge node from the sorted candidate edge nodes.

7. The data processing method based on edge cloud according to claim 1, wherein writing the data to be written into the target storage unit according to the target engine cluster includes: Writing the data to be written into the target storage unit through a writing method corresponding to the data to be written according to the target engine cluster.

8. The data processing method based on edge cloud according to claim 1 further includes: Receiving the node current operation information of the at least two initial edge nodes sent by a real-time water level statistics module, wherein the node current operation information includes the node current bandwidth, the node queries per second rate, and the node current storage space.

9. After writing the data to be written into the target storage unit in the data processing method based on edge cloud according to claim 1, it further includes: Receiving a data reading request sent by a user, wherein the data reading request carries the unit identifier of the target storage unit of the data to be read and the file name; Determining the target storage unit according to the unit identifier and reading the data to be read from the target storage unit according to the file name.

10. A data processing device based on edge cloud, applied to a node center control system, includes: A request receiving module, configured to receive a data writing request sent by a user, where the data writing request carries data to be written, user attribute information, and a unit identifier of a target storage unit corresponding to the data to be written; An initial node determination module, configured to determine a target storage domain according to the unit identifier, and determine at least two initial edge nodes corresponding to the target storage domain, where the target storage domain corresponds to a plurality of the initial edge nodes, each of the initial edge nodes belongs to a plurality of the target storage domains, and the at least two initial edge nodes belong to the same edge node cluster; A target node determination module, configured to determine a target edge node from the at least two initial edge nodes according to the node attribute information of the at least two initial edge nodes and the user attribute information, where the target node determination module is further configured to: when the number of nodes after filtering in any dimension of the initial edge nodes is less than a first threshold, expand edge nodes and storage units for the target storage domain; A data storage module, configured to determine a target storage unit corresponding to the unit identifier in the target edge node, and write the data to be written into the target storage unit, where the data storage module is further configured to: determine a target storage unit corresponding to the unit identifier in the target edge node, and at least two engine clusters in the target edge node; determine a target engine cluster according to the current running parameters of each engine cluster in the at least two engine clusters; write the data to be written into the target storage unit according to the target engine cluster.

11. A computing device, comprising: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the data processing method based on edge cloud according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the data processing method based on edge cloud according to any one of claims 1 to 9 are implemented.

Images