Distributed internet of things edge layer dynamic fragmentation optimization method

Abstract

The invention relates to a distributed internet of things edge layer dynamic fragmentation optimization method. The method comprises the following steps: extracting edge node features from a block chain network of the Internet of Things, performing a first round of fragmentation division by adopting a label propagation algorithm based on a limited community scale, and training an ELM classifier based on a first round of fragmentation division result; in each division period, an ELM classifier is adopted to detect the network state of the block chain, and if the network state of the block chain is changed, an incremental learning strategy is adopted to carry out dynamic fragmentation classification on the nodes; and when the block chain network is changed in a large scale, recalculating global optimal fragment division by adopting a label propagation algorithm based on a limited community scale, and updating the ELM classifier. A feature extraction mechanism integrating node communication frequency, geographic position and equipment performance is designed, the efficient classification capability and incremental learning characteristics of an extreme learning machine are utilized, fragment structure dynamic adjustment suitable for the IoT environment is achieved, and meanwhile high calculation overhead caused by frequent global recombination is avoided.

Description

Technical Field

[0001] This application relates to the field of IoT blockchain sharding technology, and in particular to a distributed IoT edge layer dynamic sharding optimization method. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) industry, the number of IoT terminal devices has exploded, stimulating the demand for massive data interaction and sharing. Blockchain technology, with its advantages of decentralization, tamper-proofing, and traceability, provides a potential solution for secure data sharing in IoT scenarios. However, existing IoT blockchain architectures generally face performance bottlenecks such as low transaction throughput and heavy node storage load, severely limiting their applicability in large-scale IoT applications. To improve system processing capabilities, researchers have introduced sharding technology into blockchain. By dividing the network into multiple independent shards, each node only needs to process transactions within its shard, effectively alleviating scalability bottlenecks. Blockchain sharding solutions have thus become a key focus in the current field of secure IoT data sharing.

[0003] However, due to the global reach of IoT applications, edge computing nodes are often widely distributed across diverse geographical environments, exhibiting significant heterogeneity in computing power, storage capacity, and communication conditions. Ensuring the secure sharing and efficient management of massive amounts of data presents a critical challenge. Blockchain sharding technology, by dividing the blockchain network into multiple functionally independent shards, allows each shard to process only a specific subset of transactions, potentially improving the scalability and throughput of IoT blockchains. However, current mainstream sharding strategies primarily target general blockchain scenarios, neglecting the performance heterogeneity and physical mobility of edge nodes in IoT environments. Directly applying these strategies to IoT scenarios will struggle to meet the demands for low latency, high throughput, and efficient resource utilization.

[0004] Therefore, there is an urgent need in related technologies for a partitioning method that can control communication overhead, ensure balanced resource scheduling, and improve system performance. Summary of the Invention

[0005] Therefore, it is necessary to provide a distributed IoT edge layer dynamic sharding optimization method that can control communication overhead, ensure balanced resource scheduling, and improve system performance, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for dynamic sharding optimization at the edge layer of a distributed Internet of Things (IoT). The method includes: Edge node features are extracted from the IoT blockchain network, and a label propagation algorithm based on limited community size is used for the first round of partitioning. An ELM classifier is trained based on the results of the first round of partitioning. In each partitioning cycle, the ELM classifier is used to detect the state of the blockchain network. If the state of the blockchain network changes, an incremental learning strategy is used to dynamically partition and classify the nodes. When a large-scale change occurs in the blockchain network, a label propagation algorithm based on a limited community size is used to recalculate the globally optimal sharding partition and update the ELM classifier.

[0007] Optionally, in one embodiment of this application, the extraction of edge node features from the IoT blockchain network includes: The performance indicators of all devices are normalized using an exponential decay function, and the normalization results are merged to obtain the static features of the edge nodes. The number of data sharing interactions between each node is counted, and a communication frequency matrix is ​​constructed as a dynamic feature of the edge nodes.

[0008] Optionally, in one embodiment of this application, the first-round segmentation using a label propagation algorithm based on a limited community size includes: The edge weights in the network topology are calculated based on the characteristics of edge nodes, and a weighted undirected graph is constructed to initialize the sum of edge weights for each segment. The scoring function is called to calculate the score of each neighbor label, update the edge node fragment label, and perform edge node fragmentation based on the fragment label.

[0009] Optionally, in one embodiment of this application, the step of calculating edge weights in the network topology based on edge node features includes: Calculate the physical distance, performance similarity, and communication frequency between edge nodes to determine the edge weights in the network topology.

[0010] Optionally, in one embodiment of this application, training the ELM classifier based on the first-round partitioning result includes: The input matrix is ​​defined based on the edge node features; The input matrix is ​​input into the hidden layer of the ELM model, and the hidden layer matrix is ​​output. The hidden layer output weights are calculated based on the pseudo-inverse matrix of the hidden layer matrix and the edge node fragment labels.

[0011] Optionally, in one embodiment of this application, the step of using an incremental learning strategy to dynamically segment and classify nodes includes: Calculate the hidden layer output of the new data based on the newly added or changed nodes, and update the hidden layer matrix; The new hidden layer pseudo-inverse is calculated using incremental updates, and the new hidden layer output weights are calculated based on the new hidden layer pseudo-inverse. Calculate the target fragment label of the new data based on the new hidden layer output weights, and update the fragmentation of the edge nodes.

[0012] Optionally, in one embodiment of this application, the method further includes: The data storage of edge nodes is dynamically adjusted based on a weighted consistent hashing algorithm and a weighted scoring mechanism.

[0013] Optionally, in one embodiment of this application, the dynamic adjustment of edge node data storage based on a weighted consistent hashing algorithm and a weighted scoring mechanism includes: The storage score of each edge node is calculated based on its storage space, computing power, and bandwidth, and a global storage score is obtained. The global storage score is fine-tuned based on node communication latency, and a weighted consistent hashing mechanism is used to map storage nodes to hash space and allocate virtual nodes.

[0014] Secondly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the steps of the methods described in the various embodiments above.

[0015] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the methods described in the various embodiments above.

[0016] The aforementioned distributed IoT edge layer dynamic sharding optimization method first extracts edge node features from the IoT blockchain network and performs an initial sharding partitioning using a label propagation algorithm based on a limited community size. An ELM classifier is then trained based on the initial partitioning results. Next, in each partitioning cycle, the ELM classifier detects the blockchain network state. If the blockchain network state changes, an incremental learning strategy is used to dynamically shard and classify nodes. Finally, when the blockchain network undergoes large-scale changes, the globally optimal sharding partitioning is recalculated using a label propagation algorithm based on a limited community size, and the ELM classifier is updated. In other words, a feature extraction mechanism integrating node communication frequency, geographical location, and device performance is designed. Utilizing the efficient classification capabilities and incremental learning characteristics of Extreme Learning Machines (ELM), a dynamic adjustment of the sharding structure suitable for IoT environments is achieved, while avoiding the high computational overhead caused by frequent global reorganization. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a distributed IoT edge layer dynamic sharding optimization method in one embodiment; Figure 2 This is a schematic diagram of the consistent hash storage mapping process in one embodiment; Figure 3This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In one embodiment, such as Figure 1 As shown, a dynamic sharding optimization method for distributed IoT edge layers is provided, including the following steps: S101: Extract edge node features from the IoT blockchain network, perform the first round of sharding using a label propagation algorithm based on limited community size, and train an ELM classifier based on the results of the first round of sharding.

[0020] In this embodiment, firstly, in the first round of sharding, necessary information is collected from each edge computing node in the IoT blockchain network and processed to construct edge node features for sharding classification, including static features and dynamic features. Then, global optimization is performed based on the Label Propagation Algorithm (LPA) with limited community size to form the initial sharding partition. The partitioning results are used to train the ELM classifier.

[0021] The input features for training include the feature vector of each node. Including equipment performance Physical location and the original number of communications The output label is the fragment attribution label generated by LPA, using One-Hot encoding. Assuming there are K fragments, then:

[0022] in, (if It belongs to fragmentation ),otherwise .

[0023] Ultimately, the ELM training dataset is defined as follows:

[0024] In IoT scenarios, edge computing nodes typically have low device performance update frequency, weak mobility, and usually interact with fixed groups of devices. Therefore, the sharding scheme should minimize the overhead of global re-sharding and optimize only nodes that truly require adjustment. However, traditional community partitioning methods usually require recalculating the sharding structure of the entire network in each optimization cycle, leading to unnecessary computational costs and sharding oscillations. Therefore, this application requires a classification method that can efficiently perform local updates and adapt to the long-term stability of the network, so as to reduce computational overhead while ensuring the rationality of sharding.

[0025] ELM is a highly efficient single-hidden-layer feedforward neural network classifier suitable for the sharding optimization requirements of this application. On one hand, ELM does not require iterative optimization; it only needs one forward computation to obtain the classification result. On some datasets, its training speed is hundreds of times faster than traditional deep learning methods, with lower computational resource consumption. On the other hand, ELM supports online sequence learning, enabling incremental training only for newly added or changed nodes while maintaining historical learning results, without global retraining, thus significantly reducing the computational cost of sharding adjustments. Combining ELM's efficient classification capabilities and incremental learning characteristics, this application uses it for the dynamic optimization of IoT blockchain sharding to achieve fast, stable, and low-cost sharding management.

[0026] Specifically, in one embodiment of this application, the extraction of edge node features from the Internet of Things blockchain network includes: S201: The performance indicators of all devices are normalized using an exponential decay function, and the normalization results are merged to obtain the static features of the edge nodes.

[0027] S202: Count the number of data sharing interactions between each node and construct a communication frequency matrix as a dynamic feature of the edge nodes.

[0028] In one embodiment of this application, static features reflect the long-term stability of nodes, including information such as computing resources and geographical location. These features typically do not change drastically in a short period of time, and therefore can be used as steady-state parameters input to the ELM classifier during the partitioning process. Since the numerical ranges of different performance indicators vary significantly, all device performance indicators first need to be normalized. Due to the large performance differences between different edge devices, an exponential decay function is used for normalization to reduce the impact of extreme values ​​on the normalization results. The exponential decay function effectively limits the influence of extreme values, making the data distribution more robust.

[0029] in, For the attenuation coefficient, an empirical value is selected based on the median performance value. The calculation shows that:

[0030] Multiple normalized equipment performance indicators are combined into a single comprehensive equipment performance value. The definition is as follows:

[0031] in, These are the weighting coefficients for each indicator, which can be adjusted according to the specific network environment. For edge nodes Computational ability score For edge nodes Memory performance rating For edge nodes The storage space score indicates the remaining storage space capacity. For edge nodes Disk input / output performance rating, For edge nodes The bandwidth score. Finally, during the ELM training and prediction phases, the static feature vector of each node is defined as:

[0032] in, Includes only the overall performance value of the equipment and physical location coordinates This reduces computational complexity.

[0033] Dynamic features measure the interaction patterns between nodes, especially transaction frequency and data access behavior. The number of data sharing interactions between each node is counted, and a communication frequency matrix is ​​constructed. In the training and prediction phases of the ELM classifier, nodes are used directly. With nodes Number of communications between These features are used as input features without normalization. This preserves the original information about the transaction relationships, allowing ELM to predict shard partitions more accurately. Finally, the complete feature vector for each node is defined as:

[0034] This feature vector includes both the device's computing power and physical location, as well as the communication relationships between nodes, enabling the ELM classifier to accurately predict the sharding affiliation of a node.

[0035] In one embodiment of this application, the first-round segmentation using a label propagation algorithm based on a limited community size includes: S301: Calculate the edge weights in the network topology based on the characteristics of edge nodes, construct a weighted undirected graph, and initialize the sum of edge weights for each segment.

[0036] S302: Call the scoring function to calculate the score of each neighbor label, update the edge node fragment label, and perform edge node fragmentation based on the fragment label.

[0037] In one embodiment of this application, when calculating the global partitioning using the LPA label propagation algorithm, it is necessary to calculate the physical distance and performance similarity between nodes using the static and dynamic feature vectors of edge nodes, thereby calculating the edge weights in the network topology for each node. Assigned a unique initial label Simultaneously construct a weighted undirected graph ,in The edge weights are calculated using the edge weight formula, and the sum of edge weights for each partition is initialized. .

[0038] In each iteration, each node By calling the scoring function Calculate the score of each neighbor's label and update its fragment label accordingly. Here, we use... Indicates and The set of directly connected neighbor nodes. A node represents the set of labels of all its neighbors. New label It can be expressed by the following formula:

[0039]

[0040] in, Labels indicating neighboring nodes Is it equal to the currently considered label? ,if, The value is 1 otherwise 0. This normalizes the influence of neighboring nodes, preventing certain high-weight edges from having an excessively dominant effect on the propagation process.

[0041] It is an adjustable parameter, range This is used to control the degree of optimization for cross-shard communication, preventing some shards from becoming too large. When a shard... When the load is too high (indicating heavy load or excessive cross-shard communication), its attractiveness should be reduced to encourage more nodes to join shards with lower loads.

[0042] Once the algorithm reaches its maximum number of iterations The algorithm will terminate if the following convergence conditions are met:

[0043] in For the convergence threshold, if the graph The percentage change in labels for all nodes is less than If the condition is met, the algorithm terminates.

[0044] To avoid creating small, isolated fragments during the partitioning process and thus facing the risk of a 1% attack, a minimum fragment merging strategy is implemented after LPA (Limited Partition A) ends: a minimum fragment size threshold is set. If the number of nodes in a certain partition is less than If it is a fragment, it will be merged into the adjacent fragment with the highest edge weight.

[0045] In one embodiment of this application, the step of calculating edge weights in the network topology based on edge node features includes: Calculate the physical distance, performance similarity, and communication frequency between edge nodes to determine the edge weights in the network topology.

[0046] In one embodiment of this application, the physical distance is typically a large value (in kilometers), which may lead to errors when calculating the correlation between nodes in the shards. Therefore, an exponential decay normalization method is used for processing:

[0047] in, It is calculated using the spherical cosine theorem:

[0048] It is the Earth's radius (6371 km). Let be the attenuation coefficient, based on the median physical distance. The calculation shows that:

[0049] The formula for calculating performance similarity is as follows:

[0050] in, , They are nodes , The overall equipment performance value. When this value is close to 1, it indicates that the equipment performance of the two nodes is similar and they are suitable to be assigned to the same shard.

[0051] The number of initial communications between different edge nodes can vary significantly, by orders of magnitude larger than other features. This can cause the frequency of communications to dominate the calculation of LPA edge weights, leading to an over-reliance on communication frequency in sharding decisions while ignoring device performance and physical distance. Therefore, normalization is necessary. Since the number of communications in edge computing networks typically exhibits a long-tail distribution, meaning that a small number of nodes communicate with extremely high frequencies, Log normalization is chosen for this purpose. The formula for calculating communication frequency is as follows:

[0052] in, For nodes With nodes The number of original communications between them Ultimately, the edge weights The definition is as follows.

[0053]

[0054] By setting the weights as described above, LPA can prioritize nodes with high communication frequencies, similar computing capabilities, and close physical locations to be assigned to the same shard during propagation, thereby reducing the proportion of cross-shard transactions.

[0055] In one embodiment of this application, training the ELM classifier based on the first round of partitioning results includes: S401: Define the input matrix based on the edge node features.

[0056] S402: Input the input matrix into the hidden layer of the ELM model and output the hidden layer matrix.

[0057] S403: Calculate the hidden layer output weights based on the pseudo-inverse matrix of the hidden layer matrix and the edge node fragment labels.

[0058] In one embodiment of this application, after LPA generates the initial fragmentation, ELM is used for one-time training to learn the mapping relationship between device features and fragment labels. Let each node... have 3D input features, the number of nodes in the initial LPA shard is After normalizing the features in each dimension, the input matrix is ​​defined as follows:

[0059] Fragmentation Tag Matrix Calculated using the LPA algorithm, assuming K partitions are obtained:

[0060] The ELM model uses an SLFN structure in the hidden layers, including One hidden neuron, The initial value is set to When calculating the hidden layer output, the hidden layer input weights... and bias The values ​​are uniformly and randomly generated between [-1, 1] and remain unchanged throughout the training process. Dimensions , Dimensions :

[0061] in, It is the hidden layer output matrix. It is an activation function, using ReLU: .

[0062] In the output layer, the output weights of the hidden layers need to be calculated. :

[0063] in, It is the weight matrix connecting the hidden layer and the output layer. yes The Moore-Penrose pseudo-inverse matrix. After training, the trained matrix is ​​stored. and As a classification model for ELM, the training process of ELM does not involve iterative optimization and can be completed with only one forward computation.

[0064] S102: In each partitioning cycle, the ELM classifier is used to detect the state of the blockchain network. If the state of the blockchain network changes, an incremental learning strategy is used to dynamically partition and classify the nodes.

[0065] In this embodiment of the application, during the operation phase, at the beginning of each cycle, the ELM classifier detects changes in node computing power, physical location, or a significant increase in cross-shard transaction frequency in the blockchain network. It then uses an incremental learning strategy to re-shard some nodes for prediction, and the blockchain sharding network module performs node migration to optimize shard balance.

[0066] In each partitioning cycle, nodes are classified. To reduce unnecessary overhead, ELM only partitions newly added nodes and nodes with significant state changes.

[0067] For nodes that have changed, the feature vectors need to be recalculated:

[0068]

[0069] It is then input into ELM for piecewise prediction:

[0070] ELM calculates the hidden layer output and predicts the updated fragment labels:

[0071]

[0072] Predicted fragment tags for The shard index corresponding to the maximum value in the middle, if If so, the node will be migrated from the old shard to the new shard.

[0073] For a newly joined node, the number of communications it makes with other nodes in the network. Initially, the value is 0, and segmentation is performed solely based on static features. In the next round of feature extraction, if the number of communication attempts changes and exceeds a threshold, segmentation will be performed based on the changed nodes.

[0074] In one embodiment of this application, the step of using an incremental learning strategy to dynamically segment and classify nodes includes: S501: Calculate the hidden layer output of the new data based on the newly added or changed nodes, and update the hidden layer matrix.

[0075] S502: Calculate the new hidden layer pseudo-inverse using incremental update, and calculate the new hidden layer output weights based on the new hidden layer pseudo-inverse.

[0076] S503: Calculate the target fragment label of the new data based on the new hidden layer output weights, and update the fragmentation of the edge nodes.

[0077] In one embodiment of this application, although ELM can efficiently predict node segments after initial training, the model's prediction accuracy may decrease over time. Therefore, an Online Sequential Extreme Learning Machine (OS-ELM) is introduced to incrementally train the ELM classifier, enabling it to continuously adapt to dynamic changes in the network environment without forgetting historical training results, thereby improving model adaptability.

[0078] Since OS-ELM requires a fixed input feature dimension, a "fixed input feature dimension" strategy is adopted to avoid changes in the feature dimension caused by the addition of new nodes in communication, thus ensuring the long-term feasibility of the incremental learning mechanism. This strategy is applied to the feature vectors. The communication frequency portion only retains the node's... Important neighbors, i.e., those with the highest communication frequency Each node has neighbors, such that the input features of each node are... The dimension is always fixed. .

[0079] To reduce unnecessary computation, the OS-ELM mechanism is set to trigger only when the following conditions are met cumulatively since the last incremental learning: (1) The cumulative number of newly added nodes exceeds the threshold. .

[0080] (2) The cumulative proportion of existing nodes that have undergone sharding migration exceeds .

[0081] (3) Decreased ELM prediction accuracy: In the last 5 rounds of model operation, the cross-shard communication rate has continuously exceeded .

[0082] in, The value can be determined by actual performance. When one of the above conditions is met, OS-ELM is triggered, and the partitioning results are used for incremental training in the next round of partitioning to update the model weights and improve classification accuracy.

[0083] In the incremental learning phase, OS-ELM needs to construct a new incremental training set for newly added or changed nodes.

[0084] in, This is the original training dataset, and the new node. and the latest state of the changing node It was added to the training set.

[0085] For newly added or changed nodes, calculate the hidden layer output of the new data:

[0086] in, The input feature vector representing the newly added or changed node. This is the output of the hidden layer corresponding to the new data.

[0087] When constructing the updated hidden layer matrix, it is necessary to add the hidden layer output with the new data:

[0088] Target output matrix updated synchronously:

[0089] in, It is the target shard label for the new data.

[0090] Since OS-ELM uses the Moore-Penrose pseudo-inverse matrix to solve for the output weights Calculating the entire new pseudo-inverse matrix directly could be computationally expensive. Therefore, an incremental matrix update formula is used to avoid recalculating the entire pseudo-inverse matrix.

[0091] For newly added data The new hidden layer pseudo-inverse is calculated using incremental updates:

[0092] in, It is the Moore-Penrose pseudo-inverse matrix of the old data. It is a dimension of The identity matrix.

[0093] Finally, calculate the new output weights:

[0094] In this way, OS-ELM achieves rapid learning of new and changed nodes by expanding the training data incrementally and updating the pseudo-inverse matrix, avoiding repeated calculations on the entire historical dataset.

[0095] S103: When the blockchain network undergoes large-scale changes, a label propagation algorithm based on a limited community size is used to recalculate the globally optimal sharding partition and update the ELM classifier.

[0096] In this embodiment of the application, when the network undergoes large-scale changes (such as the addition of a large number of new nodes or the migration of existing nodes leading to sharding load imbalance), a label propagation algorithm based on a limited community size is used to recalculate the globally optimal sharding partition and update the ELM classifier to ensure the long-term stability and efficiency of the sharding structure.

[0097] In one embodiment of this application, the method further includes: The data storage of edge nodes is dynamically adjusted based on a weighted consistent hashing algorithm and a weighted scoring mechanism.

[0098] In one embodiment of this application, in IoT data sharing, edge nodes, acting as distributed storage nodes, undertake a large number of data storage and computing tasks. However, due to the varying computing resources, storage capacity, and network bandwidth of edge nodes, long-term operation may lead to storage resource overload on some edge nodes within a shard, while other servers remain under low load, causing a sharding performance bottleneck. To optimize this problem, a novel load balancing method is proposed, combining a consistent hashing algorithm and a weighted scoring mechanism. This method enables data storage to be dynamically adjusted based on the actual storage capacity and network latency of the edge nodes, ensuring a reasonable distribution of storage tasks among them. Figure 2 As shown, storage scores are distributed to guide each edge node in performing weighted consistent hash calculations and allocating storage tasks locally. Unlike the traditional global hash ring construction method, the cloud only calculates and distributes storage scores to each edge node. The edge nodes then fine-tune their scores based on these scores and their own communication latency information, independently calculate their own local consistent hash rings, and finally select suitable storage nodes.

[0099] Specifically, in one embodiment of this application, the dynamic adjustment of data storage of edge nodes based on a weighted consistent hashing algorithm and a weighted scoring mechanism includes: S601: Calculate the storage score of each edge node based on its storage space, computing power, and bandwidth scores to obtain the global storage score.

[0100] S602: Fine-tune the global storage score based on node communication latency, and use a weighted consistent hashing mechanism to map storage nodes to the hash space and allocate virtual nodes.

[0101] In one embodiment of this application, a storage score is calculated based on performance metrics such as storage space, computing power, bandwidth, and latency of each edge node, which is used to construct the storage weight of the node mapping on the consistent hash ring. The system then assigns a storage score to each node. Storage rating This primarily reflects the performance of edge nodes, taking into account factors such as their remaining storage space, computing power, and bandwidth. First, in each epoch, the cloud is responsible for performing a preliminary calculation of the storage score for all edge nodes, based on the following formula:

[0102] in, It is an edge node The storage space score indicates the remaining storage space capacity. and Representing edge nodes respectively The computing power score and bandwidth score. These correspond to the weights of storage space, computing power, and bandwidth, representing the degree of influence of different metrics on the final score. Typically, remaining storage space... It has a significant impact on storage load, therefore It should be set to a higher value, while the impact on computing power and bandwidth is relatively small, so it can be appropriately reduced. The value can be adjusted according to different application scenarios and system requirements.

[0103] After evaluating all edge nodes, the cloud platform generates a preliminary global storage score. This is used to determine the initial storage load capacity of each edge node. Each edge node also needs to further fine-tune its score based on the storage score issued by the cloud, combined with communication latency information between it and other nodes, to optimize storage node selection and data distribution. The specific adjustment formula is as follows:

[0104] in, It is a node and The communication latency between nodes reflects the network performance between nodes. This is a communication latency adjustment factor that controls the impact of latency on storage rating. Generally, higher latency reduces a node's priority. It will also decrease accordingly.

[0105] The storage score is calculated globally and distributed through the cloud. The edge nodes then fine-tune the storage score based on the local network conditions, taking into account communication latency to make the load distribution more accurate.

[0106] Consistent hashing is a classic data distribution method that constructs a virtual hash ring, mapping storage nodes to a hash space, and finding the nearest storage node for data based on its hash value. To ensure the rational allocation of storage tasks, a weighted consistent hashing mechanism is employed, combined with storage scoring for edge nodes. When mapping storage nodes to hash space, different numbers of virtual nodes are allocated, and these virtual nodes and actual nodes are distributed from 0 to... In a consistent hashing ring, edge nodes with high storage scores take on more storage tasks, ensuring load balancing.

[0107] Number of virtual nodes By node The ratio of the storage score to the sum of the storage scores of all edge nodes determines:

[0108] in, This is the initial number of virtual nodes, used to control the overall distribution density of the hash ring. The sum of storage scores for all edge nodes.

[0109] This method allocates more virtual nodes to edge nodes with high storage scores, increasing the probability of data falling onto those nodes, while assigning fewer virtual nodes to edge nodes with low storage scores. This effectively avoids the hot / cold sharding problem during data storage and ensures balanced storage load across shards. During storage task allocation, data... hash value The nearest edge node is found clockwise on the hash ring, and this node becomes the storage node for the data slice. If storage overload is detected for an edge node (remaining storage space is below a set threshold), the number of its virtual nodes is dynamically reduced in the next round of mapping to avoid storage overflow.

[0110] In the aforementioned distributed IoT edge layer dynamic sharding optimization method, firstly, edge node features are extracted from the IoT blockchain network, and an initial sharding partitioning is performed using a label propagation algorithm based on a limited community size. An ELM classifier is then trained based on the initial sharding partitioning results. Subsequently, in each partitioning cycle, the ELM classifier is used to detect the blockchain network state. If the blockchain network state changes, an incremental learning strategy is employed to dynamically shard and classify the nodes. Finally, when a large-scale change occurs in the blockchain network, the globally optimal sharding partitioning is recalculated using a label propagation algorithm based on a limited community size, and the ELM classifier is updated. In other words, a feature extraction mechanism integrating node communication frequency, geographical location, and device performance is designed. Utilizing the efficient classification capabilities and incremental learning characteristics of the Extreme Learning Machine (ELM), a dynamic adjustment of the sharding structure suitable for the IoT environment is achieved, while avoiding the high computational overhead caused by frequent global reorganization.

[0111] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a distributed Internet of Things (IoT) edge layer dynamic sharding optimization method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0113] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0114] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0115] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0116] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0118] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0119] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for dynamic sharding optimization at the edge layer of a distributed Internet of Things (IoT), characterized in that, The method includes: Edge node features are extracted from the IoT blockchain network, and a label propagation algorithm based on limited community size is used for the first round of partitioning. An ELM classifier is trained based on the results of the first round of partitioning. In each partitioning cycle, the ELM classifier is used to detect the state of the blockchain network. If the state of the blockchain network changes, an incremental learning strategy is used to dynamically partition and classify the nodes. When a large-scale change occurs in the blockchain network, a label propagation algorithm based on a limited community size is used to recalculate the globally optimal sharding partition and update the ELM classifier.

2. The distributed IoT edge layer dynamic sharding optimization method according to claim 1, characterized in that, The extraction of edge node features from the Internet of Things blockchain network includes: The performance indicators of all devices are normalized using an exponential decay function, and the normalization results are merged to obtain the static features of the edge nodes. The number of data sharing interactions between each node is counted, and a communication frequency matrix is ​​constructed as a dynamic feature of the edge nodes.

3. The distributed IoT edge layer dynamic sharding optimization method according to claim 1, characterized in that, The first round of fragmentation using a tag propagation algorithm based on a limited community size includes: The edge weights in the network topology are calculated based on the characteristics of edge nodes, and a weighted undirected graph is constructed to initialize the sum of edge weights for each segment. The scoring function is called to calculate the score of each neighbor label, update the edge node fragment label, and perform edge node fragmentation based on the fragment label.

4. The distributed IoT edge layer dynamic sharding optimization method according to claim 3, characterized in that, The calculation of edge weights in the network topology based on edge node features includes: Calculate the physical distance, performance similarity, and communication frequency between edge nodes to determine the edge weights in the network topology.

5. The distributed IoT edge layer dynamic sharding optimization method according to claim 1, characterized in that, The training of the ELM classifier based on the first round of partitioning results includes: The input matrix is ​​defined based on the edge node features; The input matrix is ​​input into the hidden layer of the ELM model, and the hidden layer matrix is ​​output. The hidden layer output weights are calculated based on the pseudo-inverse matrix of the hidden layer matrix and the edge node fragment labels.

6. The distributed IoT edge layer dynamic sharding optimization method according to claim 1, characterized in that, The incremental learning strategy for dynamic segmentation and classification of nodes includes: Calculate the hidden layer output of the new data based on the newly added or changed nodes, and update the hidden layer matrix; The new hidden layer pseudo-inverse is calculated using incremental updates, and the new hidden layer output weights are calculated based on the new hidden layer pseudo-inverse. Calculate the target fragment label of the new data based on the new hidden layer output weights, and update the fragmentation of the edge nodes.

7. The distributed IoT edge layer dynamic sharding optimization method according to claim 1, characterized in that, The method further includes: The data storage of edge nodes is dynamically adjusted based on a weighted consistent hashing algorithm and a weighted scoring mechanism.

8. The distributed IoT edge layer dynamic sharding optimization method according to claim 7, characterized in that, The dynamic adjustment of edge node data storage based on weighted consistent hashing algorithm and weighted scoring mechanism includes: The storage score of each edge node is calculated based on its storage space, computing power, and bandwidth, and a global storage score is obtained. The global storage score is fine-tuned based on node communication latency, and a weighted consistent hashing mechanism is used to map storage nodes to hash space and allocate virtual nodes.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images