A data storage method, device, apparatus and medium

Abstract

The application discloses a data storage method and device, equipment and medium, and relates to the technical field of computers, and comprises the following steps: an edge computing node acquires to-be-transmitted data of a user; the edge computing node generates candidate link information based on the states of a plurality of candidate links between the edge computing node and a server and sends the candidate link information to the server; the server performs attribute prediction on all candidate links in the area where the edge computing node is located based on the candidate link information, generates a link prediction attribute table, and sends the link prediction attribute table to the edge computing node, wherein the link prediction attribute table is used at least for representing a predicted value of a user IP area; the edge computing node determines a candidate link with the maximum predicted value as an optimal link based on the link prediction attribute table; and the edge computing node sends the to-be-transmitted data to a database based on the optimal link, so that the database writes the to-be-transmitted data in a data node.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data storage method, apparatus, device, and medium. Background Technology

[0002] In business operations, unstructured data lacks a pre-defined data type or table structure, making it unsuitable for storage in traditional relational databases. Organizations generate and collect 80%-90% of unstructured data, and its volume is rapidly increasing.

[0003] However, in the process of transmitting unstructured data from the front end to the cloud platform database, it is often necessary to manually set the link transmission information based on experience or adopt a more common average transmission method. This often results in high costs and long time consumption in actual transmission, and fails to make full use of network resources, making it difficult to automatically configure more ideal link attributes. Summary of the Invention

[0004] This application provides a data storage method, apparatus, device, and medium, aiming to solve the problem of how to obtain the optimal link for unstructured data transmission.

[0005] The first aspect of this application provides a data storage method, including:

[0006] Edge computing nodes acquire user data to be transmitted;

[0007] The edge computing node generates candidate link information based on the status of multiple candidate links between itself and the server and sends it to the server.

[0008] The server performs attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, generates a link prediction attribute table and sends it to the edge computing node. The link prediction attribute table is at least used to characterize the predicted value of the user IP area.

[0009] The edge computing node determines the candidate link with the largest predicted value based on the link prediction attribute table, and selects it as the optimal link.

[0010] The edge computing node sends the data to be transmitted to the database based on the optimal link, so that the database writes the data to be transmitted into the data node.

[0011] In one optional implementation, the edge computing node is any edge computing node within the target area. The edge computing node generates candidate link information based on the status of multiple candidate links with the server and sends it to the server, including:

[0012] The edge computing node obtains the links corresponding to all edge computing nodes within the target area as candidate links;

[0013] The edge computing node obtains the candidate link information based on the target region and the link category corresponding to each candidate link;

[0014] The edge computing node sends the candidate link information to the server.

[0015] In one optional implementation, the server performs attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, generating a link prediction attribute table, including:

[0016] Based on the candidate link information, the server obtains the first historical data of each candidate link, wherein the first historical data is the historical user IP area corresponding to each candidate link;

[0017] The server performs attribute prediction based on the first historical data to obtain a first predicted value;

[0018] The server combines the first predicted values ​​corresponding to all candidate links into the link prediction attribute table.

[0019] In one optional implementation, after obtaining the first predicted value, the method further includes:

[0020] Based on the candidate link information, the server obtains second historical data and third historical data for each candidate link. The second historical data is the transmission bandwidth stability parameter value of each candidate link, and the third historical data is the transmission bandwidth value of each candidate link.

[0021] The server performs attribute prediction based on the second historical data to obtain a second predicted value, which is used to characterize the network latency between the user and the edge computing node.

[0022] The server performs attribute prediction based on the third historical data to obtain a third predicted value, which is used to characterize the network bandwidth between the user and the edge computing node.

[0023] The first predicted value, the second predicted value, and the third predicted value are fused together to obtain the link prediction attribute table.

[0024] In one optional implementation, the first predicted value, the second predicted value, and the third predicted value are fused to obtain the link prediction attribute table, including:

[0025] The first predicted value, the second predicted value, and the third predicted value corresponding to each candidate link are weighted according to preset weights to obtain the weighted value of each candidate link. The preset weight range for the first predicted value is 75% to 85%, the preset weight range for the second predicted value is 10% to 20%, and the preset weight range for the third predicted value is 0% to 10%.

[0026] The weighted values ​​of all candidate links are combined into the link prediction attribute table.

[0027] In one optional implementation, the data to be transmitted is unstructured data, and the database includes at least routing nodes and data nodes. The database writes the data to be transmitted into the data nodes, including:

[0028] The routing node obtains the write request sent by the edge computing node and the data to be transmitted;

[0029] Based on the write request, the routing node determines the fragment of the target data node in the data node for storing the data to be transmitted;

[0030] The routing node allocates the data to be transmitted to the fragment.

[0031] In one optional implementation, the data to be transmitted includes original data and service data, and the routing node allocates the data to be transmitted to the fragments, including:

[0032] The routing node distributes the original data to the configuration server of the shard;

[0033] The routing node allocates the service data to the shards, and then distributes the service data discretely across multiple availability zones through the shards. The multiple availability zones are different physical data centers within the same region, and the data is synchronized between the availability zones.

[0034] A second aspect of this application provides a data storage device, comprising:

[0035] The acquisition module is used by edge computing nodes to acquire the user's data to be transmitted;

[0036] The candidate link information module is used by the edge computing node to generate candidate link information based on the status of multiple candidate links between the edge computing node and the server, and then send it to the server.

[0037] The attribute prediction module is used by the server to perform attribute prediction on all candidate links in the area where the edge computing node is located based on the candidate link information, generate a link prediction attribute table and send it to the edge computing node. The link prediction attribute table is used to at least characterize the predicted value of the user IP area.

[0038] The optimal link module is used by the edge computing node to determine the candidate link with the largest predicted value as the optimal link based on the link prediction attribute table.

[0039] The writing module is used by the edge computing node to send the data to be transmitted to the database based on the optimal link, so that the database writes the data to be transmitted into the data node.

[0040] In one optional implementation, the candidate link information module includes:

[0041] The candidate link submodule is used by the edge computing node to obtain the links corresponding to all edge computing nodes in the target area as candidate links.

[0042] The candidate link information submodule is used by the edge computing node to obtain the candidate link information based on the target area and the link category corresponding to each candidate link;

[0043] The sending submodule is used by the edge computing node to send the candidate link information to the server.

[0044] In one optional implementation, the attribute prediction module includes:

[0045] The first historical data submodule is used by the server to obtain the first historical data of each candidate link based on the candidate link information, wherein the first historical data is the historical user IP area corresponding to each candidate link;

[0046] The first prediction submodule is used by the server to perform attribute prediction based on the first historical data and obtain a first prediction value.

[0047] The first link attribute table submodule is used by the server to combine the first predicted values ​​corresponding to all candidate links into the link prediction attribute table.

[0048] In an optional implementation, the attribute prediction module further includes:

[0049] Other historical data submodules are used by the server to obtain the second historical data and the third historical data of each candidate link based on the candidate link information. The second historical data is the transmission bandwidth stability parameter value of each candidate link, and the third historical data is the transmission bandwidth value of each candidate link.

[0050] The second prediction submodule is used by the server to perform attribute prediction based on the second historical data and obtain a second prediction value, the second prediction value being used to characterize the network latency between the user and the edge computing node;

[0051] The third prediction submodule is used by the server to perform attribute prediction based on the third historical data and obtain a third prediction value, which is used to characterize the network bandwidth between the user and the edge computing node.

[0052] The second link attribute table submodule is used to fuse the first predicted value, the second predicted value, and the third predicted value to obtain the link prediction attribute table.

[0053] In one optional implementation, the second link attribute table submodule includes:

[0054] The weighting unit is used to weight the first predicted value, the second predicted value, and the third predicted value corresponding to each candidate link according to preset weights to obtain the weighted value of each candidate link. The preset weight range corresponding to the first predicted value is 75% to 85%, the preset weight range corresponding to the second predicted value is 10% to 20%, and the preset weight range corresponding to the third predicted value is 0% to 10%.

[0055] The link prediction attribute table unit is used to combine the weighted values ​​of all candidate links into the link prediction attribute table.

[0056] In one optional implementation, the writing module includes:

[0057] The routing submodule is used by the routing node to obtain the write request sent by the edge computing node and the data to be transmitted;

[0058] The sharding submodule is used by the routing node to determine, based on the write request, a shard in the data node for storing the target data node to be transmitted.

[0059] A writing submodule is used by the routing node to allocate the data to be transmitted to the fragments.

[0060] In one optional implementation, the writing submodule includes:

[0061] The original data writing unit is used by the routing node to allocate the original data to the configuration server of the shard;

[0062] The service data writing unit is used by the routing node to allocate the service data to the shard, and to distribute the service data discretely across multiple availability zones through the shard. The multiple availability zones are different physical data centers within the same region, and the data is synchronized between the availability zones.

[0063] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the data storage method described in any one of the first aspects.

[0064] A fourth aspect of this application provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the data storage method described in any one of the first aspects.

[0065] Beneficial effects:

[0066] This application provides a data storage method, apparatus, device, and medium. The method includes: an edge computing node acquiring user data to be transmitted; the edge computing node generating candidate link information based on the status of multiple candidate links between itself and a server and sending it to the server; the server performing attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, generating a link prediction attribute table and sending it to the edge computing node, wherein the link prediction attribute table is used to characterize at least the predicted value of the user's IP area; the edge computing node determining the candidate link with the largest predicted value based on the link prediction attribute table as the optimal link; and the edge computing node sending the data to be transmitted to a database based on the optimal link, so that the database writes the data to be transmitted into a data node.

[0067] This application determines the optimal link by performing attribute prediction on all candidate links within the region and using the user's IP region as the prediction basis. This ensures that the optimal link is the link from the edge computing node closest to the user's region, thereby reducing the cost and time of transmitting unstructured data from the front end to the database. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a schematic diagram of a data storage method according to an embodiment of this application;

[0070] Figure 2 This is a schematic diagram illustrating the writing of data to be transmitted according to an embodiment of this application;

[0071] Figure 3 This is a schematic diagram of a replica set structure proposed in one embodiment of this application;

[0072] Figure 4 This is a schematic diagram of business data allocation proposed in an embodiment of this application;

[0073] Figure 5 This is a schematic diagram of a data storage device structure according to an embodiment of this application;

[0074] Figure 6 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] In related technologies, unstructured data refers to data without a pre-defined data type or table structure, and therefore cannot be stored in traditional relational databases. Text and multimedia are two common types of unstructured content. Many business documents are unstructured data, such as emails, videos, photos, web pages, and audio files.

[0077] Unstructured data stores a wealth of information used for business decision-making. For example, unstructured data is used to aid artificial intelligence and machine learning. Emerging software tools can retrieve valuable information and actionable business decisions by searching vast amounts of data. 80%-90% of the data generated and collected by organizations is unstructured, and its volume is growing rapidly, much faster than that of structured data. However, unstructured data has historically been difficult to analyze.

[0078] There are many ways to store unstructured data, such as storing it in programs, NoSQL databases, data lakes, and data warehouses. However, most storage tools cannot achieve industrial-grade security and stability, requiring industrial-scale modifications. Specifically, during the transmission of unstructured data from the front end to the cloud platform's database, manual configuration of link transmission information based on experience is often required, or a common average transmission method is adopted. This often results in high costs and long transmission times, and fails to fully utilize network resources, making it difficult to automatically configure ideal link attributes.

[0079] In view of this, embodiments of this application propose a data storage method. Figure 1 A schematic diagram of a data storage method is shown, such as Figure 1 As shown, the method includes the following steps:

[0080] S101, Edge computing nodes acquire user data to be transmitted.

[0081] In this embodiment, the edge computing node is located near the data collection point and employs an open platform integrating network, computing, storage, and application core capabilities. The edge computing node can be situated between the physical entity (data collection point) and the industrial connection (cloud platform), or it can be located at the top of the physical entity. The data collection point is the location where data to be transmitted from the user terminal is collected. For example, the data collection point can be deployed on a server in a data center in the user's region, or it can be a user terminal device based on the Internet of Things.

[0082] In specific implementation step S101, the edge computing node first acquires front-end data from at least one data acquisition point with which it has established a connection, and analyzes the front-end data to obtain the front-end data characteristics corresponding to the front-end data. These front-end data characteristics characterize the basic attributes of the front-end data, such as data size and data type. Based on these front-end data characteristics, the edge acquisition node determines whether the front-end data needs to be transmitted to the cloud platform's server. If the edge computing node determines that the front-end data does not need to be transmitted to the server based on its front-end data characteristics, no subsequent transmission and storage process is performed; if the edge computing node determines that the front-end data needs to be transmitted to the server based on its front-end data characteristics, the front-end data is designated as data to be transmitted.

[0083] S102. The edge computing node generates candidate link information based on the status of multiple candidate links between itself and the server and sends it to the server.

[0084] In specific implementation step S102, the data acquisition point and the edge computing node are located in the target area. There are multiple edge computing nodes in the target area, and there are multiple links between each edge computing node and the server. Since the links corresponding to these edge computing nodes are all located in the same target area as the data acquisition point, they are all possible links for transmitting front-end data. Therefore, the edge computing node (the edge computing node corresponding to the data acquisition point that collects front-end data) first obtains the links corresponding to all edge computing nodes in the target area as candidate links.

[0085] Subsequently, the edge computing node simultaneously initiates data transmission requests to the server through the multiple candidate links. After receiving the response information from the server, it sends information about all candidate links to the server so that the server can perform attribute prediction based on the information of the candidate links.

[0086] In this embodiment of the application, the information of the candidate link includes the information of the region where the link is located (i.e., the target region). Specifically, the information of the region is the operator base station region, the geographical location of the region, etc., and the information of the region is used to subsequently determine the candidate link closest to the user's IP region. The information of the candidate link also includes the link category corresponding to each candidate link. The link category is used by the server to obtain historical data of links with the same category.

[0087] The edge computing node uses the information of the region where the link is located and the link type as the candidate link information, and finally sends the candidate link information to the server. It should be noted that the specific candidate link information may also include other information about the link, such as the duration of data transmission requests initiated by the edge computing node based on the link. This candidate link information can be determined according to the actual situation, and this application does not impose any restrictions on it.

[0088] S103. The server performs attribute prediction on all candidate links in the area where the edge computing node is located based on the candidate link information, generates a link prediction attribute table, and sends it to the edge computing node.

[0089] The server performs attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, generates a link prediction attribute table, and sends it to the edge computing node. The link prediction attribute table is used to characterize the predicted value of the user's IP area at least.

[0090] In specific implementation step S103, the server receives candidate link information from the edge computing node, and obtains historical data for each candidate link based on this candidate link information. This historical data is used to predict the link's attributes and determine the optimal link best suited for transmitting the data to be collected from the data collection point. Specifically, firstly, the server determines the historical data of links of the same link category as the candidate links based on the region information in the candidate link information. Subsequently, from the historical data of these links of the same category, the server obtains the historical user IP region corresponding to each candidate link, as the first historical data for each candidate link.

[0091] The server performs attribute prediction based on the first historical data to obtain a first predicted value. Since users acquire resources fastest locally, when predicting links based on the first historical data, the user's IP region is used as the prediction dimension for link transmission. The obtained first predicted value can characterize the distance between different candidate links and the user's IP region; the closer the candidate link, the faster the transmission speed. The server combines the first predicted values ​​corresponding to all candidate links into a link prediction attribute table. By comparing the first predicted values ​​corresponding to different candidate links in the link prediction attribute table, the server can predict the candidate link closest to the user's IP region among the multiple candidate links corresponding to the edge computing node.

[0092] In one optional implementation, the accuracy of predicting the optimal link can be further improved by adding predicted values ​​from other dimensions, based on the original first predicted value. After obtaining the first predicted value, the server determines second and third historical data for links of the same link category as the candidate links based on the regional information in the candidate link information. The second historical data is the transmission bandwidth stability parameter value of each candidate link, and the third historical data is the transmission bandwidth value of each candidate link.

[0093] The server performs attribute prediction based on the second historical data to obtain a second predicted value. Since network latency also affects the speed at which users acquire resources, when users (data collection points) are located in the same region, it is no longer possible to distinguish the corresponding candidate links solely by the user's IP region. Candidate links with lower network latency have faster transmission speeds. Therefore, the second predicted value is introduced using the second historical data. When performing link prediction, network latency is used as a prediction dimension for link transmission. The resulting second predicted value can be used to characterize the network latency between the user and the edge computing node.

[0094] The server performs attribute prediction based on the third historical data to obtain a third predicted value. Since network bandwidth also affects the speed at which users acquire resources, when users (data collection points) are located in the same region, it is no longer possible to distinguish corresponding candidate links solely by user IP region. Furthermore, candidate links with larger network bandwidth have faster transmission speeds. Therefore, the third predicted value is introduced using the third historical data. For example, when high-throughput traffic requests occur, a high-bandwidth CDN site is prioritized. When performing link prediction, network bandwidth is used as a prediction dimension for link transmission, and the resulting third predicted value can be used to characterize the network bandwidth between the user and the edge computing node.

[0095] After obtaining the first, second, and third predicted values ​​for each candidate link across the three dimensions described above, the first, second, and third predicted values ​​are fused to obtain the link prediction attribute table. Specifically, the first, second, and third predicted values ​​for each candidate link are weighted according to preset weights to obtain a weighted value for each candidate link, and then the weighted values ​​for each candidate link are combined to form the link attribute prediction table.

[0096] In one optional implementation, the preset weight range for the first predicted value is 75% to 85%, the preset weight range for the second predicted value is 10% to 20%, and the preset weight range for the third predicted value is 0% to 10%. Preferably, the preset weight for the first predicted value is 80%, the preset weight for the second predicted value is 15%, and the preset weight for the third predicted value is 5%.

[0097] In an optional implementation, the historical data may also be other historical data, and other predicted values ​​may be obtained from the other historical data. The weighted value is then obtained by combining the first predicted value, the second predicted value, and the third predicted value according to the following formula:

[0098]

[0099] in, x1 is the weighted value; f1 is the first predicted value; f2 is the preset weight corresponding to the first predicted value; x2 is the second predicted value; f3 is the third predicted value; f3 is the preset weight corresponding to the third predicted value; x n f is the nth predicted value; n The preset weight is the weight corresponding to the nth predicted value; n is the number of predicted values.

[0100] S104. The edge computing node determines the candidate link with the largest predicted value based on the link prediction attribute table, and selects it as the optimal link.

[0101] In specific implementation step S104, the link prediction attribute table contains the weighted values ​​of all candidate links corresponding to all edge computing nodes within the target area. These weighted values ​​are obtained by weighting different prediction values, and each candidate link is rated as the optimal link from different dimensions using these different prediction values. Based on the link prediction attribute table, the edge computing node determines the candidate link corresponding to the largest weighted value as the optimal link.

[0102] For example, the weighted value is obtained by weighting and fusing a first predicted value, a second predicted value, and a third predicted value. The candidate link corresponding to the largest weighted value is determined in the link prediction attribute table. The first predicted value corresponding to the largest weighted value indicates that the user's IP region is the same as the region of the candidate link corresponding to the largest weighted value. The second predicted value indicates that among all candidate links with the same user IP region, the candidate link corresponding to the largest weighted value has the lowest network latency. The third predicted value indicates that among all candidate links with the same user IP region, the candidate link corresponding to the largest weighted value has the highest bandwidth.

[0103] S105. The edge computing node sends the data to be transmitted to the database based on the optimal link, so that the database writes the data to be transmitted into the data node.

[0104] In this embodiment, the data to be transmitted is unstructured data, which includes original data and business data. The original data usually does not change, while the business data changes based on the different types of data to be transmitted. The database includes at least a router and a data node. The router is used to determine the target data node where the shard corresponding to the data to be transmitted is located based on the write request of the data to be transmitted, and sends the write request to the target data node so that the target data node responds to the write request by storing the data to be transmitted in the shard in the target data node.

[0105] Figure 2 This invention provides a schematic diagram illustrating the writing of data to be transmitted according to an embodiment of this application. Figure 2As shown, in specific implementation step S105, the edge computing node generates a corresponding database write request for the data to be transmitted. Based on the optimal link, the edge computing node sends the data to be transmitted and the corresponding write request to the routing node of the unified database network management system. The routing node obtains the write request sent by the edge computing node and the data to be transmitted. Based on the write request of the data to be transmitted, it determines the fragment of the target data node in the data node to be used to store the data to be transmitted.

[0106] Figure 3 This application illustrates a schematic diagram of a replica set structure according to an embodiment of the present application, as shown below. Figure 3 As shown, the data to be transmitted includes both raw data and business data. The raw data is relatively small and can be stored in the configuration server of the target data node. Specifically, the routing node distributes the raw data to the configuration server of the shard and uses a replica set for backup, providing multiple redundancy features. The replica set is a master-slave cluster with automatic fault recovery, consisting of one primary node and at least one secondary node. The primary node is the primary database instance; instances are independent and resource-isolated, and there are no preemption issues related to CPU, memory, or I / O. The secondary nodes are database instances; instances are independent and resource-isolated, and there are no preemption issues related to CPU, memory, or I / O. The primary node synchronizes data to the secondary nodes using offsets to achieve redundant backup and improve data high availability. Simultaneously, heartbeat detection is set up between the secondary nodes to obtain the monitoring status of each instance, ensuring storage security.

[0107] Because business data changes, minimizing network latency for data storage is crucial to ensure continuous online service. Therefore, the routing node allocates the business data to shards, which then distribute the data across multiple availability zones. These availability zones are different physical data centers within the same region, interconnected by their internal networks. Minimal network latency within each availability zone ensures fault isolation between them (except in the event of major disasters or power outages), preventing fault propagation and ensuring continuous online service for users. Figure 4 This application illustrates a schematic diagram of business data allocation according to an embodiment of the present application, as shown below. Figure 4 As shown, through different availability zones ( Figure 4Different zones are used to distinguish the request sources of data collection points or edge computing nodes in different regions. For example, a write request from Beijing with a zone identifier "zone:["A"] will be assigned to the availability zone of Beijing. The data to be transmitted corresponding to the write request with different zone identifiers will be distributed accordingly. Figure 4 The data in the upper middle section is written to the corresponding availability zone.

[0108] In one optional implementation, the routing node is configured with a routing service for sharding. When querying unstructured data (stored data to be transmitted) stored in the database, the routing service receives and responds to the query request from the application layer, determines the storage location of the data in the sharded cluster, and completes the unstructured data query.

[0109] This application provides a data storage method, the method comprising: an edge computing node acquiring user data to be transmitted; the edge computing node generating candidate link information based on the status of multiple candidate links between itself and a server and sending it to the server; the server performing attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, generating a link prediction attribute table and sending it to the edge computing node, the link prediction attribute table being used at least to characterize the predicted value of the user's IP area; the edge computing node determining the candidate link with the largest predicted value based on the link prediction attribute table, as the optimal link; and the edge computing node sending the data to be transmitted to a database based on the optimal link, so that the database writes the data to be transmitted into a data node.

[0110] This application determines the optimal link by performing attribute prediction on all candidate links within the region and using the user's IP region as the prediction basis. This ensures that the optimal link is the link from the edge computing node closest to the user's region, thereby reducing the cost and time of transmitting unstructured data from the front end to the database.

[0111] Based on the same inventive concept, embodiments of this application disclose a data storage device. Figure 5 A schematic diagram of a data storage device structure according to an embodiment of this application is shown, as follows: Figure 5 As shown, the device includes:

[0112] The acquisition module is used by edge computing nodes to acquire the user's data to be transmitted;

[0113] The candidate link information module is used by the edge computing node to generate candidate link information based on the status of multiple candidate links between the edge computing node and the server, and then send it to the server.

[0114] The attribute prediction module is used by the server to perform attribute prediction on all candidate links in the area where the edge computing node is located based on the candidate link information, generate a link prediction attribute table and send it to the edge computing node. The link prediction attribute table is used to at least characterize the predicted value of the user IP area.

[0115] The optimal link module is used by the edge computing node to determine the candidate link with the largest predicted value as the optimal link based on the link prediction attribute table.

[0116] The writing module is used by the edge computing node to send the data to be transmitted to the database based on the optimal link, so that the database writes the data to be transmitted into the data node.

[0117] In one optional implementation, the candidate link information module includes:

[0118] The candidate link submodule is used by the edge computing node to obtain the links corresponding to all edge computing nodes in the target area as candidate links.

[0119] The candidate link information submodule is used by the edge computing node to obtain the candidate link information based on the target area and the link category corresponding to each candidate link;

[0120] The sending submodule is used by the edge computing node to send the candidate link information to the server.

[0121] In one optional implementation, the attribute prediction module includes:

[0122] The first historical data submodule is used by the server to obtain the first historical data of each candidate link based on the candidate link information, wherein the first historical data is the historical user IP area corresponding to each candidate link;

[0123] The first prediction submodule is used by the server to perform attribute prediction based on the first historical data and obtain a first prediction value.

[0124] The first link attribute table submodule is used by the server to combine the first predicted values ​​corresponding to all candidate links into the link prediction attribute table.

[0125] In an optional implementation, the attribute prediction module further includes:

[0126] Other historical data submodules are used by the server to obtain the second historical data and the third historical data of each candidate link based on the candidate link information. The second historical data is the transmission bandwidth stability parameter value of each candidate link, and the third historical data is the transmission bandwidth value of each candidate link.

[0127] The second prediction submodule is used by the server to perform attribute prediction based on the second historical data and obtain a second prediction value, the second prediction value being used to characterize the network latency between the user and the edge computing node;

[0128] The third prediction submodule is used by the server to perform attribute prediction based on the third historical data and obtain a third prediction value, which is used to characterize the network bandwidth between the user and the edge computing node.

[0129] The second link attribute table submodule is used to fuse the first predicted value, the second predicted value, and the third predicted value to obtain the link prediction attribute table.

[0130] In one optional implementation, the second link attribute table submodule includes:

[0131] The weighting unit is used to weight the first predicted value, the second predicted value, and the third predicted value corresponding to each candidate link according to preset weights to obtain the weighted value of each candidate link. The preset weight range corresponding to the first predicted value is 75% to 85%, the preset weight range corresponding to the second predicted value is 10% to 20%, and the preset weight range corresponding to the third predicted value is 0% to 10%.

[0132] The link prediction attribute table unit is used to combine the weighted values ​​of all candidate links into the link prediction attribute table.

[0133] In one optional implementation, the writing module includes:

[0134] The routing submodule is used by the routing node to obtain the write request sent by the edge computing node and the data to be transmitted;

[0135] The sharding submodule is used by the routing node to determine, based on the write request, a shard in the data node for storing the target data node to be transmitted.

[0136] A writing submodule is used by the routing node to allocate the data to be transmitted to the fragments.

[0137] In one optional implementation, the writing submodule includes:

[0138] The original data writing unit is used by the routing node to allocate the original data to the configuration server of the shard;

[0139] The service data writing unit is used by the routing node to allocate the service data to the shard, and to distribute the service data discretely across multiple availability zones through the shard. The multiple availability zones are different physical data centers within the same region, and the data is synchronized between the availability zones.

[0140] Based on the same inventive concept, this application discloses an electronic device. Figure 6 A schematic diagram of an electronic device disclosed in an embodiment of this application is shown, such as... Figure 6 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus. The memory 110 stores a computer program that can run on the processor 120 to implement the steps in the data storage method disclosed in the embodiments of this application.

[0141] Based on the same inventive concept, embodiments of this application disclose a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, implements the steps in the data storage method described in the embodiments of this application.

[0142] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0143] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0144] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0146] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0147] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0148] The data storage method, apparatus, device, and medium provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data storage method, characterized in that, The method includes: Edge computing nodes acquire user data to be transmitted; the data to be transmitted is unstructured data. The edge computing node generates candidate link information based on the status of multiple candidate links between itself and the server and sends it to the server. The server performs attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, generates a link prediction attribute table and sends it to the edge computing node. The link prediction attribute table is at least used to characterize the predicted value of the user IP area. The edge computing node determines the candidate link with the largest predicted value based on the link prediction attribute table, and selects it as the optimal link. The edge computing node sends the data to be transmitted to the database based on the optimal link, so that the database writes the data to be transmitted into the data node; the database includes at least a routing node and the data node; The database writes the data to be transmitted into the data nodes, including: The routing node obtains the write request sent by the edge computing node and the data to be transmitted; Based on the write request, the routing node determines the fragment of the target data node in the data node for storing the data to be transmitted; The routing node allocates the data to be transmitted to the fragment.

2. The data storage method according to claim 1, characterized in that, The edge computing node is any edge computing node within the target area. Based on the status of multiple candidate links between the edge computing node and the server, the edge computing node generates candidate link information and sends it to the server, including: The edge computing node obtains the links corresponding to all edge computing nodes within the target area as candidate links; The edge computing node obtains the candidate link information based on the target region and the link category corresponding to each candidate link; The edge computing node sends the candidate link information to the server.

3. The data storage method according to claim 1, characterized in that, The server performs attribute prediction on all candidate links within the area where the edge computing node is located based on the candidate link information, and generates a link prediction attribute table, including: Based on the candidate link information, the server obtains the first historical data of each candidate link, wherein the first historical data is the historical user IP area corresponding to each candidate link; The server performs attribute prediction based on the first historical data to obtain a first predicted value; The server combines the first predicted values ​​corresponding to all candidate links into the link prediction attribute table.

4. The data storage method according to claim 3, characterized in that, After obtaining the first predicted value, the method further includes: Based on the candidate link information, the server obtains second historical data and third historical data for each candidate link. The second historical data is the transmission bandwidth stability parameter value of each candidate link, and the third historical data is the transmission bandwidth value of each candidate link. The server performs attribute prediction based on the second historical data to obtain a second predicted value, which is used to characterize the network latency between the user and the edge computing node. The server performs attribute prediction based on the third historical data to obtain a third predicted value, which is used to characterize the network bandwidth between the user and the edge computing node. The first predicted value, the second predicted value, and the third predicted value are fused together to obtain the link prediction attribute table.

5. The data storage method according to claim 4, characterized in that, The first predicted value, the second predicted value, and the third predicted value are fused to obtain the link prediction attribute table, which includes: The first predicted value, the second predicted value, and the third predicted value corresponding to each candidate link are weighted according to preset weights to obtain the weighted value of each candidate link. The preset weight range for the first predicted value is 75% to 85%, the preset weight range for the second predicted value is 10% to 20%, and the preset weight range for the third predicted value is 0% to 10%. The weighted values ​​of all candidate links are combined into the link prediction attribute table.

6. The data storage method according to claim 1, characterized in that, The data to be transmitted includes original data and service data. The routing node allocates the data to be transmitted to the fragments, including: The routing node distributes the original data to the configuration server of the shard; The routing node allocates the service data to the shards, and then distributes the service data discretely across multiple availability zones through the shards. The multiple availability zones are different physical data centers within the same region, and the data is synchronized between the availability zones.

7. A data storage device, characterized in that, include: The acquisition module is used by edge computing nodes to acquire the user's data to be transmitted; The data to be transmitted is unstructured data; The candidate link information module is used by the edge computing node to generate candidate link information based on the status of multiple candidate links between the edge computing node and the server, and then send it to the server. The attribute prediction module is used by the server to perform attribute prediction on all candidate links in the area where the edge computing node is located based on the candidate link information, generate a link prediction attribute table and send it to the edge computing node. The link prediction attribute table is used to at least characterize the predicted value of the user IP area. The optimal link module is used by the edge computing node to determine the candidate link with the largest predicted value as the optimal link based on the link prediction attribute table. The writing module is used by the edge computing node to send the data to be transmitted to the database based on the optimal link, so that the database writes the data to be transmitted into the data node; The database includes at least routing nodes and data nodes; The writing module includes: The routing submodule is used by the routing node to obtain the write request sent by the edge computing node and the data to be transmitted; The sharding submodule is used by the routing node to determine, based on the write request, a shard in the data node for storing the target data node to be transmitted. A writing submodule is used by the routing node to allocate the data to be transmitted to the fragments.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps of the data storage method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the data storage method according to any one of claims 1-6.

Images