Network data efficient caching method and system based on edge computing

Abstract

The invention relates to the technical field of edge computing, and discloses a network data efficient caching method and system based on edge computing, which realize dynamic identification of hotspot data and optimal caching position decision by analyzing edge node data flow characteristics and network topology states in real time. Firstly, data packet meta-information is extracted from an edge node, a content feature vector is generated, and data popularity is evaluated; secondly, analyzing a network topology structure and a link state, and screening candidate cache nodes; then, factors such as service priority, network reliability and topology stability are fused, and an optimal cache position is determined by adopting a reinforcement learning algorithm; and finally, the cache content is updated, the cache state is synchronized between the edge nodes, and the cache distribution is optimized. The data access efficiency and the resource utilization rate of the edge network can be effectively improved, and the method adapts to dynamically changing network environments and user requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of edge computing, and in particular to a method and system for efficiently caching network data based on edge computing. Background Art

[0002] As a core pillar of next-generation network architecture, edge computing plays an irreplaceable role in promoting low-latency, high-efficiency data processing and distributed intelligence. With the explosive growth of applications such as the Internet of Things, autonomous driving, and smart cities, the demand for network data processing is increasing exponentially. Edge computing significantly alleviates the pressure on central cloud by moving computing and storage capabilities to edge nodes close to data sources.

[0003] Existing edge computing caching methods currently exhibit significant shortcomings when coping with complex network environments. Many solutions rely on static caching strategies, ignoring the dynamic nature of data and the real-time changes in network status. This results in low cache resource utilization, frequent data duplication, and cache failures. This not only wastes the limited storage space of edge nodes but also increases network transmission latency.

[0004] In an existing technology for cache optimization in edge computing, dynamic perception of data features and intelligent decision-making on cache locations are two core challenges. Data features include content type, size, and popularity. These attributes change rapidly in actual networks, making it difficult for traditional methods to extract them in real time and form effective feature vectors for decision-making. As a result, the cache system is unable to dynamically adjust storage strategies based on the importance of the data, causing popular data to be accidentally replaced and unpopular data to occupy valuable space. Furthermore, the complexity of network topology and link status exacerbates this problem. Edge nodes are widely distributed, and network connection status often fluctuates. If these factors are not comprehensively considered, the choice of cache location is often not accurate enough, resulting in increased data access latency and even service interruption. Furthermore, the lack of an efficient cache status sharing mechanism between edge nodes means that the same data may be stored repeatedly on multiple nodes, wasting resources.

[0005] Therefore, how to intelligently determine the cache location and realize cache status sharing among edge nodes based on the dynamic feature vector of the content, combined with network topology and link status, to improve resource utilization and reduce access latency has become a key issue for efficient data caching in edge computing. Summary of the Invention

[0006] The present invention provides a method and system for efficiently caching network data based on edge computing, so as to realize efficient data caching in edge computing.

[0007] In a first aspect, in order to solve the above technical problems, the present invention provides a method for efficiently caching network data based on edge computing, comprising:

[0008] Obtain real-time data streams from edge nodes, generate content feature vectors, and obtain a quantitative representation of the data's dynamic characteristics;

[0009] Calculating the popularity value of the data based on the content feature vector; if the popularity value exceeds a preset popularity threshold, marking it as hot data, and obtaining a classification result of the data popularity change;

[0010] Obtaining the current network topology and the distribution information of the edge nodes from the network management system, and calculating the link state parameters between the nodes;

[0011] Obtaining a data source identifier in the content feature vector and a node connectivity weight in the link state parameter; if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, selecting the corresponding node as a candidate cache node, thereby obtaining a candidate node set that integrates service priority and network reliability;

[0012] Obtaining the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter; if the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, filtering out the corresponding stable nodes in the candidate node set to obtain a target cache node set that combines topological stability and request regularity;

[0013] Determining a target cache node according to the classification result of the data heat change and the target cache node set;

[0014] Checking the storage space of the node according to the target cache node, and if the remaining space is lower than a preset space threshold, releasing the storage space to obtain the updated cache content;

[0015] Generate a cache status table according to the updated cache content to obtain a global cache status sharing result;

[0016] According to the global cache state sharing result, it is detected whether the data identifier is repeatedly stored in multiple nodes. If repeated storage exists, the node storage with the highest heat value is retained to obtain an optimized cache distribution.

[0017] Preferably, the data popularity value is calculated based on the content feature vector. If the popularity value exceeds a preset popularity threshold, the data is marked as hot data. The classification result of the data popularity change is obtained, including:

[0018] Obtain user interaction data from a data source, extract feature dimensions such as page views, likes, and comments, and perform vector normalization to generate the initial content feature vector;

[0019] According to a pre-established popularity evaluation model, the initial content feature vector is input, combined with the weight coefficient, and the weighted sum of each feature dimension is calculated through a linear regression algorithm to obtain the popularity value of the data;

[0020] If the popularity value exceeds a preset popularity threshold, it is marked as hot data, otherwise it is marked as non-hot data, and the classification result is generated based on the mark;

[0021] The heat values ​​of consecutive time points corresponding to the classification results are obtained, and the initial content feature vector is adjusted through a dynamic update mechanism to generate a classification result of the data heat change.

[0022] Preferably, obtaining the current network topology and the distribution information of the edge nodes from the network management system and calculating the link state parameters between the nodes includes:

[0023] Obtaining the current network topology and distribution information of edge nodes from the network management system, combining it with real-time link monitoring data, and storing it in a distributed database to obtain a structured network data set;

[0024] Based on the structured network data set, a graph analysis algorithm is used to calculate the link state parameters between each node, including the delay distribution between nodes and the link bandwidth utilization, and the connectivity weight is determined to obtain a link state parameter set;

[0025] Afterwards, the method further includes: if the inter-node delay in the link state parameter set exceeds a preset delay threshold, calculating the path dynamic change frequency to obtain a network state fluctuation data set;

[0026] According to the network status fluctuation data set, a statistical analysis tool is used to calculate the fluctuation amplitude and frequency distribution, and a quantitative description report is generated through a visualization tool to obtain a quantitative description of the network status fluctuation.

[0027] Preferably, the step of obtaining the data source identifier in the content feature vector and the node connectivity weight in the link state parameter, and selecting the corresponding node as a candidate cache node if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, thereby obtaining a candidate node set that integrates service priority and network reliability, includes:

[0028] Extracting a data source identifier from the content feature vector, querying a pre-established service priority mapping table through a distributed database, and generating a service priority data set if the data source identifier corresponds to a high-priority service;

[0029] According to the service priority data set and in combination with the node connectivity weight in the link state parameter, if the service priority is higher than a preset priority threshold and the node connectivity weight is higher than a preset weight threshold, determining a candidate cache node set by a path calculation tool;

[0030] According to the candidate cache node set, cache node capacity and service traffic distribution data of each node are obtained and processed through a clustering tool to obtain an optimized node subset;

[0031] Based on the optimized node subset, a weighted scoring mechanism is used to comprehensively consider the service priority, the node connectivity weight, and the cache node capacity to calculate the score of each node and obtain the final candidate node set. Preferably, the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter are obtained. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain the target cache node set that combines topological stability and request regularity, including:

[0032] Obtaining time series characteristics from the data request log, extracting the periodic pattern of the data request using a Fourier transform algorithm, generating a periodic feature vector, and obtaining a quantized result of the periodic feature vector;

[0033] Calculating a path dynamic change frequency according to the link state parameters, and if the dynamic frequency is lower than a preset change threshold, screening nodes that meet the conditions from the candidate node set to determine the initial stable node set;

[0034] According to the initial stable node set, a topological stability index of each node is obtained, and the periodic eigenvector and the topological stability index are fused by a weighted average algorithm to obtain the fused eigenvector;

[0035] If the value of the fused feature vector is higher than a preset vector threshold, the corresponding node is included in the target cache node set to generate the target cache node set that is a fusion of the topology stability and request regularity.

[0036] Preferably, determining the target cache node according to the classification result of the data heat change and the target cache node set includes:

[0037] Obtain content feature vectors from data popularity and traffic distribution, and use clustering algorithms to group classification results;

[0038] If the heat of the grouped data exceeds a preset heat threshold, the initial candidate node set is obtained;

[0039] According to the topology structure of the initial candidate node set, link status and node performance parameters are obtained, and if the delay of the link status is lower than a preset delay threshold, the candidate node with high priority is determined by a sorting algorithm;

[0040] Iteratively optimizing the candidate nodes with high priority by using a reinforcement learning algorithm, inputting content feature vectors, link status, and storage capacity, to obtain an optimal cache location decision;

[0041] Through the optimal cache location decision and storage capacity, it is determined whether the cache node meets the traffic distribution requirements, and the updated target cache node is determined.

[0042] Preferably, generating a cache status table according to the updated cache content to obtain a global cache status sharing result includes:

[0043] Get cache content from the target node, query the storage location through data identification, and use distributed hash tables to locate edge nodes;

[0044] If the updated cache content is updated, then the updated data is extracted to obtain the latest cache content;

[0045] Generate a status table based on the latest cache content, organize the data identifier, storage location and popularity value in a structured manner, and store them in key-value pairs;

[0046] If the heat value exceeds a preset heat threshold, it is marked as high-frequency data and a state table is obtained;

[0047] Synchronize the state table using a distributed consensus algorithm and broadcast the state table among edge nodes;

[0048] If the data between nodes is inconsistent, the conflict resolution mechanism is triggered to obtain the synchronized state table;

[0049] The edge node data is merged through the synchronized state table to obtain the global state, and the sharing result is calculated using an aggregation function. If there is a data conflict, the latest data is selected according to the timestamp to obtain the global cache state sharing result.

[0050] In a second aspect, the present invention provides an efficient network data caching system based on edge computing, which is characterized by comprising:

[0051] The generation module is used to obtain real-time data streams from edge nodes, generate content feature vectors, and obtain a quantitative representation of the dynamic characteristics of the data;

[0052] a marking module, configured to calculate the popularity value of the data based on the content feature vector, and mark the data as hot data if the popularity value exceeds a preset popularity threshold, thereby obtaining a classification result of the data popularity change;

[0053] A calculation module is used to obtain the current network topology and the distribution information of the edge nodes from the network management system and calculate the link state parameters between the nodes;

[0054] a fusion module, configured to obtain a data source identifier in the content feature vector and a node connectivity weight in the link state parameter; if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, select the corresponding node as a candidate cache node, and obtain a candidate node set that integrates service priority and network reliability;

[0055] An acquisition module is configured to obtain the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain a target cache node set that combines topological stability with request regularity.

[0056] a determination module, configured to determine a target cache node according to the classification result of the data heat change and the target cache node set;

[0057] An update module is configured to check the storage space of a node according to the target cache node, and if the remaining space is lower than a preset space threshold, release the storage space to obtain updated cache content;

[0058] A first cache module, configured to generate a cache status table according to the updated cache content to obtain a global cache status sharing result;

[0059] The second cache module is used to detect whether the data identifier is repeatedly stored in multiple nodes based on the global cache state sharing result. If there is repeated storage, the node storage with the highest heat value is retained to obtain an optimized cache distribution.

[0060] In a third aspect, the present invention also provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the efficient network data caching method based on edge computing as described above is implemented.

[0061] In a fourth aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned edge computing-based network data efficient caching methods.

[0062] Compared with the existing technology, the present invention provides an efficient network data caching method and system based on edge computing, which realizes the dynamic identification of hotspot data and the optimal cache location decision by real-time analysis of edge node data flow characteristics and network topology status. First, extract the data packet metadata from the edge node, generate the content feature vector, and evaluate the data heat; secondly, analyze the network topology and link status, and screen the candidate cache nodes; then, integrate business priority, network reliability, topology stability and other factors, and use the reinforcement learning algorithm to determine the optimal cache location; finally, update the cache content and synchronize the cache status between edge nodes to optimize the cache distribution. The present invention can effectively improve the data access efficiency and resource utilization of the edge network, and adapt to the dynamically changing network environment and user needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a flow chart of a method for efficient caching of network data based on edge computing provided by the first embodiment of the present invention;

[0064] Figure 2 This is a schematic diagram of the structure of an efficient network data caching system based on edge computing provided by the second embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] Reference Figure 1 The first embodiment of the present invention provides a flow chart of a method for efficiently caching network data based on edge computing, including the following steps:

[0067] S11, obtains real-time data streams from edge nodes, generates content feature vectors, and obtains a quantitative representation of the data's dynamic characteristics;

[0068] S12, calculating the popularity value of the data based on the content feature vector, and if the popularity value exceeds a preset popularity threshold, marking it as hot data, and obtaining a classification result of the data popularity change;

[0069] S13, obtaining the current network topology and the distribution information of the edge nodes from the network management system, and calculating the link state parameters between the nodes;

[0070] S14, obtaining the data source identifier in the content feature vector and the node connectivity weight in the link state parameter. If the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, selecting the corresponding node as a candidate cache node, thereby obtaining a candidate node set that integrates service priority and network reliability.

[0071] S15, obtaining the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain a target cache node set that combines topological stability and request regularity.

[0072] S16, determining a target cache node according to the classification result of the data heat change and the target cache node set;

[0073] S17, checking the storage space of the node according to the target cache node, and if the remaining space is lower than a preset space threshold, releasing the storage space to obtain the updated cache content;

[0074] S18, generating a cache status table according to the updated cache content to obtain a global cache status sharing result;

[0075] S19, based on the global cache state sharing result, detecting whether the data identifier is repeatedly stored in multiple nodes, and if so, retaining the node storage with the highest heat value to obtain an optimized cache distribution.

[0076] In step S11, a real-time data stream is acquired from an edge node, a content feature vector is generated, and a quantitative representation of the dynamic characteristics of the data is obtained.

[0077] Acquire real-time data streams from edge nodes, parse data packets, extract metadata of the data packets, including content type, data size, request frequency, data source identifier, and time series characteristics, and store them in a preset database to obtain a structured representation of the metadata of the data packets. Use a preset feature extraction algorithm to process the content type, data size, request frequency, data source identifier, and time series characteristics in the structured representation to generate content feature vectors, store them in a vector database, and obtain a set of content feature vectors. If the request frequency of a feature vector in the set of content feature vectors exceeds a preset threshold, the feature vectors are grouped using a clustering algorithm to determine a classification pattern of the data dynamic characteristics. Based on the classification pattern, calculate the weighted average of the time series characteristics and request frequency of each group in the classification pattern to generate a quantitative representation of the data dynamic characteristics, store them in a quantitative database, and obtain a final representation of the data dynamic characteristics.

[0078] Specifically, real-time data stream processing has important applications in edge computing scenarios, especially in the field of IoT device monitoring.

[0079] For example, consider a smart factory sensor network where edge nodes acquire data streams from temperature sensors on the production line. Data packets contain information such as temperature values, timestamps, and sensor IDs. When parsing data packets, protocol analysis tools can be used to extract metadata. The content type refers to the type of data carried by the packet, such as temperature data; the data size is the number of bytes per packet, such as 64 bytes; the request frequency is the number of packets per second, such as 10 times / second; the data source is identified by the sensor ID, such as S001; and the time series characteristics are the trends in data over time, such as periodic temperature fluctuations. This metadata is stored in a MySQL database using JSON format, creating a structured representation that facilitates querying and analysis. Stored procedures can optimize performance through batch inserts.

[0080] Specifically, the feature extraction algorithm can use principal component analysis to generate feature vectors based on the content type, data size, request frequency, data source identification, and time series characteristics in the structured representation.

[0081] For example, the content type can be encoded as a numeric value, with temperature data as 1; data size is normalized to a range of 0 to 1; request frequency is averaged using a sliding window; and time series features are extracted using a Fourier transform to extract frequency components. The resulting feature vector, such as [1, 0.64, 0.1, 0.001, 0.5], is stored in a vector database such as Milvus, enabling efficient similarity retrieval. This vector representation preserves the core characteristics of the data packet and facilitates subsequent analysis.

[0082] In one possible implementation, if the request frequency of a feature vector exceeds a threshold, such as 20 times per second, it indicates possible abnormal traffic. A K-means clustering algorithm is used to group the feature vectors, setting the number of clusters to 3 and generating three classification modes: normal, abnormal, and transitional.

[0083] For example, normal patterns correspond to low frequencies and stable time series, while abnormal patterns correspond to high frequencies and violent fluctuations. Clustering results reveal the distribution patterns of data dynamic characteristics, facilitating fault detection.

[0084] Preferably, based on the classification pattern, a weighted average of the time series characteristics and request frequency is calculated for each group. Assuming a weight of 0.7 for the request frequency in normal mode and 0.3 for the time series characteristics, a quantized value, such as 0.85, is obtained and stored in a quantitative database such as Redis. This quantitative representation reflects the dynamic behavior of the data stream and supports real-time monitoring and forecasting.

[0085] For example, an abnormal increase in a quantitative value may indicate equipment overload, triggering an alarm. The final representation integrates multi-dimensional features, achieving a complete mapping from raw data to dynamic characteristics.

[0086] It should be noted that the above process significantly improves the efficiency and accuracy of data processing through structured storage, feature extraction and quantitative analysis.

[0087] For example, cluster analysis of feature vectors can quickly identify abnormal patterns, while quantitative representation provides an intuitive basis for decision-making. These technologies can reduce equipment failure rates and improve production efficiency in smart factories, forming a closed loop from data collection to decision support.

[0088] In step S12, the popularity value of the data is calculated based on the content feature vector. If the popularity value exceeds a preset popularity threshold, the data is marked as hot data, and a classification result of the data popularity change is obtained.

[0089] The data popularity value is calculated based on the content feature vector. If the popularity value exceeds a preset popularity threshold, the data is marked as popular data. The classification result of the data popularity change is obtained, including:

[0090] Obtain user interaction data from a data source, extract feature dimensions such as page views, likes, and comments, and perform vector normalization to generate the initial content feature vector;

[0091] According to a pre-established popularity evaluation model, the initial content feature vector is input, combined with the weight coefficient, and the weighted sum of each feature dimension is calculated through a linear regression algorithm to obtain the popularity value of the data;

[0092] If the popularity value exceeds a preset popularity threshold, it is marked as hot data, otherwise it is marked as non-hot data, and the classification result is generated based on the mark;

[0093] The heat values ​​of consecutive time points corresponding to the classification results are obtained, and the initial content feature vector is adjusted through a dynamic update mechanism to generate a classification result of the data heat change.

[0094] For example, when obtaining user interaction data from social media platforms, the key is to extract feature dimensions such as views, likes, and comments. These features reflect user attention and engagement with content. Views indicate how many times a piece of content has been viewed, likes reflect user approval, and comments indicate the depth of interaction.

[0095] For example, on a short video platform, a video has 100,000 views, 5,000 likes, and 1,000 comments. These data constitute the initial feature dimensions. Vector normalization is used to normalize feature values ​​of different magnitudes to the same scale.

[0096] For example, the raw values ​​of views, likes, and comments may vary greatly. After standardization, they are mapped to the range of 0 to 1 to facilitate subsequent model processing.

[0097] In one possible implementation, vector normalization uses the maximum-minimum normalization method. Assume a video has 100,000 views, with a platform view count range of 10,000 to 1,000,000, and a normalized value of 0.9; 5,000 likes, with a range of 1,000 to 10,000, and a normalized value of 0.49; and 1,000 comments, with a range of 10 to 5,000, and a normalized value of 0.2. These values ​​constitute the initial content feature vector.

[0098] It should be noted that standardization ensures fair comparison of different features and avoids model bias due to magnitude differences.

[0099] Specifically, the heat evaluation model is based on the linear regression algorithm and combines the preset weight coefficients to calculate the weighted sum of the feature vectors.

[0100] For example, the model may set the weight of views to 0.5, the number of likes to 0.3, and the number of comments to 0.2. For the above feature vector, the weighted sum is calculated as 0.9×0.5+0.49×0.3+0.2×0.2=0.637. If the preset threshold is 0.6, the heat value of the video 0.637 exceeds the threshold and is marked as hot data. This marking method makes it easier for the platform to quickly screen high-profile content. Preferably, for time series data, the dynamic update mechanism adjusts the feature vector by the heat value of consecutive time points.

[0101] For example, a video's popularity score was 0.637 24 hours after its release. 48 hours later, as the number of comments reached 2,000, the normalized score became 0.4, and the recalculated popularity score was 0.73. Dynamic updates capture changing popularity trends, helping the platform adjust its recommendation strategies in real time.

[0102] It can be understood that this mechanism avoids the limitations of static evaluation and maintains the timeliness of feature vectors.

[0103] In one embodiment, the classification result may be further refined into subcategories of popular data.

[0104] For example, based on the rate of comment growth, popular data can be categorized as either "explosive" or "stable." If a video's comments increase from 1,000 to 2,000 within 24 hours, it's marked as explosive and prioritized for push notifications to maintain its popularity. This refined categorization enhances the accuracy of content recommendations.

[0105] For example, the combination of dynamic updates and classification results enables the platform to promptly discover potential explosive content and optimize the user experience.

[0106] For example, data from consecutive time points in the video above shows that its popularity continues to rise over the 72-hour period, indicating increased user engagement. The dynamic update mechanism recalculates feature vectors to ensure that classification results reflect the latest trends. This approach maintains consistent data processing and logical rigor within a single business scenario.

[0107] In step S13, the current network topology and the distribution information of the edge nodes are obtained from the network management system, and the link state parameters between the nodes are calculated.

[0108] The obtaining of the current network topology and the distribution information of the edge nodes from the network management system and the calculation of the link state parameters between the nodes include:

[0109] Obtaining the current network topology and distribution information of edge nodes from the network management system, combining it with real-time link monitoring data, and storing it in a distributed database to obtain a structured network data set;

[0110] Based on the structured network data set, a graph analysis algorithm is used to calculate the link state parameters between each node, including the delay distribution between nodes and the link bandwidth utilization, and the connectivity weight is determined to obtain a link state parameter set;

[0111] Afterwards, the method further includes: if the inter-node delay in the link state parameter set exceeds a preset delay threshold, calculating the path dynamic change frequency to obtain a network state fluctuation data set;

[0112] According to the network status fluctuation data set, a statistical analysis tool is used to calculate the fluctuation amplitude and frequency distribution, and a quantitative description report is generated through a visualization tool to obtain a quantitative description of the network status fluctuation.

[0113] For example, when obtaining network topology and edge node distribution information from a network management system, data can be collected through a software-defined network controller. For example, consider an enterprise network consisting of 10 edge nodes distributed across three data centers. The controller captures the connectivity and geographic locations between nodes in real time, generating a topology map. The map shows that the link from node A to node B passes through two switches, with a link length of 100 kilometers. This approach ensures a clear representation of the network structure, facilitating subsequent analysis.

[0114] In one possible implementation, bandwidth usage and latency data may be collected through a network probe tool in combination with real-time link monitoring data.

[0115] For example, the link bandwidth between node A and node B is 1 Gbps, currently 70% utilized, and has a latency of 5 milliseconds. The probe updates data every 5 minutes and uploads it to a distributed database such as Cassandra. Distributed storage efficiently processes large amounts of network data, ensuring low latency and high availability for data queries.

[0116] Specifically, a graph analysis algorithm is used to calculate link state parameters based on a structured network dataset. The graph database Neo4j is used to store topology data, and the PageRank algorithm is used to evaluate node importance. Connectivity weights are calculated by combining latency and bandwidth utilization.

[0117] For example, node A has a higher importance score than node C because it connects more critical links. The Dijkstra algorithm is used to find the shortest path. Assume the optimal path from node A to node B has a latency of 4 milliseconds, which is better than the suboptimal path's latency of 6 milliseconds. This method improves path selection accuracy.

[0118] It's important to note that if inter-node latency exceeds a preset threshold, such as 10 milliseconds, you can use a time series analysis tool like Prometheus to analyze the frequency of path changes. For example, if the latency of a link fluctuates from 5 milliseconds to 15 milliseconds within an hour, analysis shows that the fluctuation frequency occurs every 10 minutes. The resulting dataset reveals the dynamics of network bottlenecks and helps quickly locate problematic links.

[0119] Preferably, based on the network status fluctuation dataset, a statistical analysis tool such as Python's Pandas library is used to calculate the fluctuation amplitude and frequency distribution.

[0120] For example, statistics show that 80% of fluctuations are between 2 and 5 milliseconds, with a normal frequency distribution. Combining this with visualization tools like Grafana generates reports containing latency curves and bandwidth heatmaps, visually displaying network status. This quantitative description facilitates quick decision-making for administrators.

[0121] In one embodiment, the combination of the above methods can dynamically optimize network performance.

[0122] For example, by analyzing reports, an administrator discovered that a certain link frequently exceeded its threshold. The administrator promptly adjusted the routing policy and switched traffic to an alternate path, reducing latency from 15 milliseconds to 6 milliseconds. This approach significantly improved network stability and user experience.

[0123] As you can see, the above example, from topology acquisition to report generation, forms a complete network status analysis chain. The implementation methods of each link support each other, ensuring the reliability and practicality of the analysis results and providing efficient technical support for network management.

[0124] In step S14, the data source identifier in the content feature vector and the node connectivity weight in the link state parameter are obtained. If the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, the corresponding node is selected as a candidate cache node to obtain a candidate node set that integrates service priority and network reliability.

[0125] The step of obtaining the data source identifier in the content feature vector and the node connectivity weight in the link state parameter, and selecting the corresponding node as a candidate cache node if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, thereby obtaining a candidate node set that integrates service priority and network reliability, including:

[0126] Extracting a data source identifier from the content feature vector, querying a pre-established service priority mapping table through a distributed database, and generating a service priority data set if the data source identifier corresponds to a high-priority service;

[0127] According to the service priority data set and in combination with the node connectivity weight in the link state parameter, if the service priority is higher than a preset priority threshold and the node connectivity weight is higher than a preset weight threshold, determining a candidate cache node set by a path calculation tool;

[0128] According to the candidate cache node set, cache node capacity and service traffic distribution data of each node are obtained and processed through a clustering tool to obtain an optimized node subset;

[0129] According to the optimized node subset, a weighted scoring mechanism is adopted to comprehensively consider the service priority, the node connectivity weight and the cache node capacity to calculate the score of each node and obtain the final candidate node set.

[0130] For example, when extracting data source identifiers from content feature vectors, network data packets can be parsed using a streaming tool such as Apache Kafka. For example, assume that data packets transmitted across an enterprise network contain service type identifiers, such as video streaming or database queries. Kafka extracts these identifiers in real time and generates structured logs.

[0131] For example, the video stream is identified as V001 and the database query is D002. This approach ensures accurate identification of data sources and facilitates subsequent priority mapping.

[0132] In one possible implementation, a service priority mapping table is queried through a distributed database, using MongoDB to store predefined mappings. Suppose the mapping table specifies video stream V001 as high priority with a value of 8, while the database query specifies D002 as medium priority with a value of 5. The query process identifies V001 as matching high priority and generates a service priority dataset containing service types and priority values. This approach supports fast queries and adapts to dynamic network environments.

[0133] Specifically, when combining the node connectivity weight in the link state parameter to determine whether the priority and weight thresholds are met, the network topology data stored in a graph database such as Neo4j can be used.

[0134] For example, the connectivity weight of node A is 0.9, and that of node B is 0.6. The preset weight threshold is 0.7, and the priority threshold is 7. Video stream V001 has a priority of 8, and node A's weight meets the requirements. Therefore, a path calculation tool such as the ONOS controller determines a candidate cache node set, which includes node A and its directly connected nodes. This screening ensures that high-priority services are assigned to reliable nodes.

[0135] It's important to note that when obtaining cache node capacity and traffic distribution data for a candidate cache node set, real-time data can be collected using monitoring tools such as Zabbix. Assume that node A has a cache capacity of 500GB and a current traffic flow of 200Mbps, while node B has a capacity of 300GB and a traffic flow of 150Mbps. Clustering tools such as the K-means algorithm can be used to process the data, dividing the nodes into two groups based on capacity and traffic distribution. This yields an optimized node subset, such as a high-performance subset that includes node A. This clustering approach improves the targeted nature of node selection.

[0136] Preferably, when a weighted scoring mechanism is used to calculate the final candidate node set, service priority, node connectivity weight, and cache node capacity may be comprehensively considered.

[0137] For example, let's set the priority weight factor to 0.4, the connectivity weight factor to 0.3, and the capacity factor to 0.3. Node A has a priority of 8, a weight of 0.9, and a capacity of 500GB, resulting in a calculated score of 8.2. Node B has a lower score of 6.5. The final candidate node set selects Node A, which has a score of 8.2. This scoring mechanism ensures that the node with the best overall performance is selected.

[0138] It's clear that the above-described embodiment forms a complete service optimization chain, from data source extraction to final node selection. Each link supports each other through distributed processing, real-time monitoring, and algorithmic analysis, ensuring efficient allocation of high-priority services within the network. This approach significantly improves resource utilization and service response speed, providing reliable support for network management.

[0139] In step S15, the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter are obtained. If the data request is periodic and the path dynamic change frequency is lower than the preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain the target cache node set that integrates topological stability and request regularity.

[0140] The time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter are obtained. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain a target cache node set that combines topological stability and request regularity, including:

[0141] Obtaining time series characteristics from the data request log, extracting the periodic pattern of the data request using a Fourier transform algorithm, generating a periodic feature vector, and obtaining a quantized result of the periodic feature vector;

[0142] Calculating a path dynamic change frequency according to the link state parameters, and if the dynamic frequency is lower than a preset change threshold, screening nodes that meet the conditions from the candidate node set to determine the initial stable node set;

[0143] According to the initial stable node set, a topological stability index of each node is obtained, and the periodic eigenvector and the topological stability index are fused by a weighted average algorithm to obtain the fused eigenvector;

[0144] If the value of the fused feature vector is higher than a preset vector threshold, the corresponding node is included in the target cache node set to generate the target cache node set that is a fusion of the topology stability and request regularity.

[0145] For example, when obtaining time series characteristics from data request logs, the timestamp distribution of the requests can be extracted by analyzing the user request logs in the distributed storage system.

[0146] For example, logs from a video streaming platform show that user requests from 6:00 PM to 10:00 PM daily are significantly higher than at other times. Using a Fourier transform algorithm, time series data can be decomposed into sinusoidal components of varying frequencies, identifying peak request patterns with a 24-hour period. By generating a periodic feature vector, the intensity of the period can be quantified, e.g., peak requests account for 60%, while off-peak requests account for only 20%. Generating this periodic feature vector helps accurately predict cache demand.

[0147] In one possible implementation, when calculating the path dynamic change frequency in the link state parameters, the path stability can be determined by monitoring the link delay and packet loss rate of each node in the network topology in real time.

[0148] For example, if a node's link latency fluctuates between 5 and 10 milliseconds and its packet loss rate is less than 1%, its dynamic change frequency is low, indicating a stable path. If the preset threshold is two changes per minute, the node meets the criteria and is included in the initial stable node set. This screening method ensures the reliability of cache nodes under network fluctuations.

[0149] Specifically, when obtaining the topological stability index for the initial stable node set, the number of neighbor nodes and the link interruption frequency of the node can be analyzed.

[0150] For example, node A has eight neighboring nodes and no link interruptions in the past 24 hours, indicating a high topology stability index. However, node B has only three neighboring nodes and has experienced two brief outages, indicating low stability. When fusing the periodic feature vectors with the topology stability index using a weighted average algorithm, the weight for the periodic feature can be set to 0.6, and the weight for topology stability can be set to 0.4.

[0151] For example, if the periodic eigenvalue of node A is 0.8, the topological stability value is 0.9, and the fused eigenvector value is 0.8×0.6+0.9×0.4=0.84, then if the preset threshold is 0.7, node A is included in the target cache node set.

[0152] Preferably, when generating a target cache node set that integrates topology stability and request regularity, the cache hit rate of the node can be further verified.

[0153] For example, node A's cache hit rate during peak hours reaches 85%, indicating that it can efficiently respond to periodic requests, while node B's hit rate is only 50%, possibly due to its poor topological location. This node set can better match business needs and improve cache efficiency.

[0154] It can be understood that the above method takes into account both request regularity and network reliability by integrating periodic characteristics with topological stability.

[0155] For example, video streaming platforms can use this method to prioritize stable nodes with high hit rates as cache nodes during peak hours, thereby optimizing the user experience. This logical progression of multi-dimensional analysis ensures comprehensive and adaptable node selection.

[0156] In step S16, a target cache node is determined according to the classification result of the data heat change and the target cache node set.

[0157] The determining of the target cache node according to the classification result of the data heat change and the target cache node set includes:

[0158] Obtain content feature vectors from data popularity and traffic distribution, and use clustering algorithms to group classification results;

[0159] If the heat of the grouped data exceeds a preset heat threshold, the initial candidate node set is obtained;

[0160] According to the topology structure of the initial candidate node set, link status and node performance parameters are obtained, and if the delay of the link status is lower than a preset delay threshold, the candidate node with high priority is determined by a sorting algorithm;

[0161] Iteratively optimizing the candidate nodes with high priority by using a reinforcement learning algorithm, inputting content feature vectors, link status, and storage capacity, to obtain an optimal cache location decision;

[0162] Through the optimal cache location decision and storage capacity, it is determined whether the cache node meets the traffic distribution requirements, and the updated target cache node is determined.

[0163] For example, when obtaining content feature vectors from data heat and traffic distribution, the frequency of user requests and content popularity in the distributed storage system may be analyzed.

[0164] For example, user logs from a video streaming platform show that requests for popular TV series increase significantly around 8:00 PM every Friday night, accounting for 40% of total traffic, while requests for standard content only account for 10%. Statistical analysis can generate content feature vectors to quantify the popularity of popular content. For example, a popular TV series might have a popularity value of 0.9, while standard content might have a value of 0.2. This feature vector reflects content popularity trends and helps accurately identify high-demand content.

[0165] In a possible implementation, when a clustering algorithm is used to group classification results, a K-means algorithm may be used to classify content feature vectors.

[0166] For example, content can be divided into three groups: high, medium, and low popularity, with a data popularity threshold of 0.7. If the average popularity of popular episodes in a group is 0.85, exceeding the threshold, the associated nodes of that group are included in the initial candidate node set. This grouping method ensures that the node set is highly relevant to the high-popularity content.

[0167] Specifically, when obtaining link status and node performance parameters, the link delay and computing power of nodes in the network can be monitored.

[0168] For example, a node with an average link latency of 8 milliseconds and a CPU utilization below 50% demonstrates excellent performance. If the preset latency threshold is 10 milliseconds, the node meets the criteria. Using a sorting algorithm, such as a weighted sort based on latency and performance, nodes with low latency and high performance can be prioritized, for example, being placed first in the candidate set.

[0169] Preferably, when using a reinforcement learning algorithm for iterative optimization, a reinforcement learning model can be constructed to input content feature vectors, link status, and storage capacity, and output an optimal cache location decision.

[0170] For example, through multiple iterations, the model discovered that nodes with 100TB of storage capacity and low link latency are more suitable for caching popular episodes. Each iteration adjusts the node selection strategy based on the cache hit rate, for example, prioritizing nodes with an 80% hit rate. This dynamic optimization approach adapts to traffic fluctuations.

[0171] It is understandable that when determining whether a cache node meets traffic distribution requirements, the storage capacity and actual traffic load of the node may be analyzed.

[0172] For example, a node with 120TB of storage capacity can support 40% of peak traffic demand, while another node with only 50TB capacity struggles to cope with peak loads. By comparing node capacity with traffic distribution, the node with sufficient capacity and high priority is selected as the final target cache node. This selection logic ensures that the node can efficiently respond to user requests.

[0173] For example, to address peak traffic for a popular TV series, the preferred node must feature low latency and large capacity. Testing revealed that one node had a latency of 7 milliseconds, an 85% cache hit rate, and a storage capacity of 150TB, perfectly matching traffic distribution requirements. In contrast, another node had a hit rate of only 60% and a capacity of 80TB, which was insufficient to meet the requirements. Therefore, the former was selected as the target cache node. This logical progression of multi-dimensional analysis ensures comprehensive and adaptable node selection.

[0174] In step S17, the storage space of the node is checked according to the target cache node. If the remaining space is lower than a preset space threshold, the storage space is released to obtain updated cache content.

[0175] The storage space data of the target cache node is obtained, and the remaining space value is extracted from the storage space. This is compared with a preset threshold. If the remaining space is lower than the preset threshold, the data replacement process is triggered to obtain a determination result that replacement is required. Based on the determination result, a least recently used algorithm is used to extract data access records from the cache content. The access frequency of each data item is counted, and the data with the lowest access frequency is determined to be unpopular data, thereby obtaining a list of unpopular data. Based on the list of unpopular data, a data replacement operation is performed to remove the unpopular data from the cache content. At the same time, new data is written to update the cache content to obtain updated cache content.

[0176] For example, when obtaining the storage space data of the target cache node, key indicators such as total capacity, used space, and remaining space need to be extracted from the storage management system of the cache node. Assuming that the total capacity of a cache node is 1000GB, the used space is 800GB, and the remaining space is 200GB. This process is usually implemented through the node's built-in monitoring tool to collect storage status in real time. The remaining space value is compared with the preset threshold. For example, if the threshold is set to 300GB, 200GB is less than 300GB, triggering the data replacement process. This mechanism ensures that the cache node has enough space to meet the needs of new data writing.

[0177] In one possible implementation, after the data replacement process is initiated, it is necessary to determine which data can be replaced. A common approach is to use the least recently used (LRU) algorithm. Its core principle is to prioritize the removal of the least recently accessed data based on the chronological order of data access.

[0178] For example, a cache node stores video files A, B, and C. In the last 24 hours, A was accessed 10 times, B was accessed 5 times, and C was not accessed. The LRU algorithm would mark C as unpopular data. This method accurately identifies low-value data through statistical analysis of access records.

[0179] Specifically, when analyzing data access frequency, log files in the cache management system can be used to extract access records for each piece of data. Suppose the logs show that file D has only been accessed twice in the past seven days, while file E has been accessed 50 times. D is classified as unpopular data and placed on a list of unpopular data. This process requires ensuring that the statistical period is reasonable, such as seven or 30 days, to reflect the true popularity of the data. Generating a list of unpopular data provides a clear target for subsequent replacement operations.

[0180] In one embodiment, when performing a data replacement operation, data is selected from a list of unpopular data and removed.

[0181] For example, the list contains files C and D, totaling 50GB. New data F is 40GB in size. After removing C and D, the remaining space increases to 250GB, sufficient for writing F. After writing the new data, the cache content is updated, and the management system records the new storage status, such as the used space becoming 790GB. This ensures that the cache content remains efficient, prioritizing the storage of hot data.

[0182] Preferably, after the cache content is updated, the data integrity can be verified through a verification mechanism.

[0183] For example, after writing file F, the system compares F's hash value to ensure no data loss. This step is particularly important in high-concurrency scenarios, as it prevents data errors caused by frequent replacements. The entire process ensures high cache node availability by monitoring free space, identifying unpopular data, and efficiently replacing it.

[0184] It is understandable that the above method needs to take node load into consideration in practical applications.

[0185] For example, if a node is processing multiple replacement tasks simultaneously, a priority queue must be set to ensure that high-demand data is written first. This design dynamically adjusts cache content, optimizes storage resource utilization, and provides stable support for the system.

[0186] In step S18, a cache status table is generated according to the updated cache content to obtain a global cache status sharing result.

[0187] Generating a cache status table according to the updated cache content to obtain a global cache status sharing result includes:

[0188] Get cache content from the target node, query the storage location through data identification, and use distributed hash tables to locate edge nodes;

[0189] If the updated cache content is updated, then the updated data is extracted to obtain the latest cache content;

[0190] Generate a status table based on the latest cache content, organize the data identifier, storage location and popularity value in a structured manner, and store them in key-value pairs;

[0191] If the heat value exceeds a preset heat threshold, it is marked as high-frequency data and a state table is obtained;

[0192] Synchronize the state table using a distributed consensus algorithm and broadcast the state table among edge nodes;

[0193] If the data between nodes is inconsistent, the conflict resolution mechanism is triggered to obtain the synchronized state table;

[0194] The edge node data is merged through the synchronized state table to obtain the global state, and the sharing result is calculated using an aggregation function. If there is a data conflict, the latest data is selected according to the timestamp to obtain the global cache state sharing result.

[0195] For example, when retrieving cached content from a target node, the data identifier can be used to quickly locate the storage location. The data identifier is typically a unique key, such as a file hash or user ID. In a distributed system, assuming there are 1,000 cache nodes, content with the data identifier "user_123" needs to be located at a specific node. A distributed hash table (DHT) can be used to map identifiers to node addresses.

[0196] For example, the hash function maps “user_123” to node N5, ensuring efficient query.

[0197] Specifically, a distributed hash table (DHT) works by assigning data identifiers to a ring of nodes using a consistent hashing algorithm. Assuming there are 10 nodes in the ring, and the hash value range is 0 to 2^32, the hash value of "user_123" falls into the range of node N5. Compared to centralized indexing, DHT avoids single points of failure and keeps query latency to milliseconds. This location method is particularly suitable for edge computing scenarios, such as quickly locating video files in CDN networks.

[0198] In one possible implementation, if the cached content is updated, for example, a user uploads a new avatar, the updated data needs to be extracted and the latest cached content needs to be generated. Assume that the original avatar is stored in node N5, with a size of 2MB, and the new avatar is 3MB. After the edge node detects the update, it writes the new data to N5 and marks the version number as v2. The generated state table records the data identifier "user_123_avatar", the storage location "N5", and the popularity value (such as 100 visits in 24 hours). If the popularity value exceeds the threshold of 50, it is marked as high-frequency data to facilitate subsequent optimization of the storage strategy.

[0199] Preferably, the Raft algorithm is used to synchronize the state table to ensure consistency across edge nodes. Assume there are three edge nodes, N1, N2, and N3. N1, acting as the leader, broadcasts the state table. If N2 doesn't receive the update due to network latency, Raft detects inconsistencies through log replication and triggers conflict resolution. Upon resolution, N2 accepts N1's latest log and updates its local state table. This approach ensures that the system maintains consistency even if some nodes are temporarily offline.

[0200] For example, when merging edge node data, an aggregation function can be used to calculate the global state. Suppose N1 stores 100 views of "user_123" and N2 stores 80. The aggregation function takes the maximum value, resulting in a global popularity of 100. If the timestamps of N1 and N2 show data conflicts—for example, N1 records a time of 10:00 and N2 records a time of 10:01—the newer data from N2 is selected. This timestamp-based conflict resolution ensures accurate global state and is particularly suitable for real-time recommendation systems.

[0201] It can be appreciated that the global cache state sharing result can be used to optimize resource allocation.

[0202] For example, high-frequency data can be preferentially stored on edge nodes close to users, reducing access latency. For example, if a video file's popularity value is 200, far exceeding the threshold of 50, the system will replicate it to more edge nodes, such as N4 and N6, to reduce pressure on the backbone network. This mechanism is particularly effective in live streaming scenarios, ensuring smooth viewing for users.

[0203] It's important to note that the above solution, with its progressive approach from location and update to synchronization, ensures the efficiency and consistency of the distributed cache system. By using DHT for rapid location, Raft for state synchronization, and aggregation functions for data consolidation, the system remains stable in dynamic environments and meets high concurrency requirements.

[0204] In step S19, based on the global cache state sharing result, it is detected whether the data identifier is repeatedly stored in multiple nodes. If there is repeated storage, the node storage with the highest heat value is retained to obtain an optimized cache distribution.

[0205] The data identifier and its corresponding storage location and heat value are obtained from the global cache status sharing result, and the storage status of the data identifier on the edge node is queried using a distributed hash table. If the same data identifier is found to be stored in multiple nodes, a set of duplicate stored data identifiers is generated. For the set of duplicate stored data identifiers, the heat value of each data identifier on different nodes is obtained, and the heat values ​​are compared using a sorting algorithm. If the node with the highest heat value is unique, an optimized node allocation table is generated. According to the optimized node allocation table, a data deletion instruction is sent to the node to be deleted through a distributed message queue. If a deletion confirmation message is received, an updated cache distribution is generated. The storage location of the edge node is obtained from the updated cache distribution, and the storage location of the data identifier is reallocated using a consistent hashing algorithm. If a load imbalance is detected, the final optimized cache distribution is generated.

[0206] For example, when obtaining data identification, storage location, and heat value from the global cache state sharing result, specific information can be queried through the distributed database. Assume that the system manages 1,000 edge nodes, and the content with the data identification "video_456" is stored on nodes N3 and N7, with heat values ​​of 150 and 80, respectively. The query process uses a distributed hash table to map the hash value of "video_456" to the node address to ensure efficient query. The implementation of the distributed hash table is based on the consistent hashing algorithm, which distributes data identification to the node ring. The query latency is usually in the millisecond level, which is suitable for high-concurrency scenarios such as video distribution networks.

[0207] In one possible implementation, after discovering that "video_456" is stored repeatedly at N3 and N7, a set of duplicate data identifiers is generated. The system then traverses the global cache state and extracts all duplicate identifiers, such as "video_456" and "image_789." For each duplicate identifier in the set, the popularity value at each node is obtained.

[0208] For example, the popularity of "image_789" on N4 is 120, while on N8 it is 90. A quick sort algorithm is used to compare the popularity values ​​and determine the node with the highest popularity. For example, N4 is the optimal storage node for "image_789." This sorting method has low time complexity and is suitable for large-scale data processing.

[0209] Preferably, an optimized node allocation table is generated based on the heat value comparison results. If the heat value of "video_456" in N3 is 150, which is higher than that of N7 at 80, the allocation table record "video_456" is stored in N3. The allocation table is stored using key-value pairs, such as "video_456: N3" and "image_789: N4". A deletion instruction is sent to the nodes to be deleted, N7 ​​and N8, through a distributed message queue, such as Kafka. The message queue ensures reliable transmission of the instruction. After receiving the instruction, N7 deletes "video_456" and returns a confirmation message. The system updates the cache distribution based on the confirmation message, generates a new distribution record, and marks "video_456" as being stored only in N3.

[0210] Specifically, after obtaining the storage location of the edge node from the updated cache distribution, the consistent hashing algorithm is used to reallocate the data identifier.

[0211] For example, a newly added data identifier, "video_999," is assigned to N5 through a hash map. If N5 detects an overabundance of data and a load imbalance, the system adjusts the allocation and migrates some data to N6. The resulting optimized cache distribution records the latest storage locations of all data identifiers, such as "video_456: N3" and "video_999: N5." This approach ensures load balancing and improves system stability.

[0212] As you can see, the above solution ensures efficient operation of the distributed cache system through a layered approach, including fast querying in distributed hash tables, optimized storage through sorting algorithms, reliable message queue deletion, and dynamic allocation through consistent hashing. Each step supports the others, achieving optimized data storage and load balancing, making it particularly suitable for edge computing scenarios.

[0213] The above description is only a specific implementation method of this specification. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here. It should be understood that the scope of protection of this specification is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in this specification, and these modifications or replacements should be included in the scope of protection of this specification.

[0214] In summary, the present invention discloses a method for efficient caching of network data based on edge computing, which realizes dynamic identification of hotspot data and optimal cache location decision by real-time analysis of edge node data flow characteristics and network topology status. First, extract data packet metadata from edge nodes, generate content feature vectors, and evaluate data heat; second, analyze network topology and link status, and screen candidate cache nodes; then, integrate business priority, network reliability, topology stability and other factors, and use reinforcement learning algorithms to determine the optimal cache location; finally, update cache content and synchronize cache status between edge nodes to optimize cache distribution. The present invention can effectively improve data access efficiency and resource utilization of edge networks, and adapt to dynamically changing network environments and user needs.

[0215] Reference Figure 2 The second embodiment of the present invention provides a structure diagram of an efficient network data caching system based on edge computing, including:

[0216] The generation module 201 is used to obtain real-time data streams from edge nodes, generate content feature vectors, and obtain a quantitative representation of the dynamic characteristics of the data;

[0217] The marking module 202 is used to calculate the popularity value of the data based on the content feature vector. If the popularity value exceeds a preset popularity threshold, it is marked as hot data to obtain a classification result of the data popularity change;

[0218] The calculation module 203 is used to obtain the current network topology and the distribution information of the edge nodes from the network management system and calculate the link state parameters between the nodes;

[0219] Fusion module 204 is configured to obtain the data source identifier in the content feature vector and the node connectivity weight in the link state parameter. If the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, the corresponding node is selected as a candidate cache node, thereby obtaining a candidate node set that integrates service priority and network reliability.

[0220] An acquisition module 205 is configured to acquire the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain a target cache node set that combines topological stability with request regularity.

[0221] A determination module 206 is configured to determine a target cache node based on the classification result of the data heat change and the target cache node set;

[0222] An updating module 207 is configured to check the storage space of the target cache node and release the storage space if the remaining space is lower than a preset space threshold to obtain updated cache content.

[0223] A first cache module 208 is configured to generate a cache status table according to the updated cache content to obtain a global cache status sharing result;

[0224] The second cache module 209 is used to detect whether the data identifier is repeatedly stored in multiple nodes based on the global cache status sharing result. If there is repeated storage, the node with the highest heat value is retained to obtain an optimized cache distribution.

[0225] It should be noted that an embodiment of the present invention provides an efficient network data caching system based on edge computing, which is used to execute all the process steps of an efficient network data caching method based on edge computing in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0226] An embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an edge computing-based network data efficient caching program. When the processor executes the computer program, the steps in each of the above-mentioned edge computing-based network data efficient caching method embodiments are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are realized, such as the generation module.

[0227] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0228] The electronic device may be a computing device such as a desktop computer, notebook, PDA, or smart tablet. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will appreciate that the aforementioned components are merely examples of electronic devices and do not constitute a limitation of the electronic device. The electronic device may include more or fewer components than those described above, or a combination of certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, and the like.

[0229] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, connecting various parts of the entire electronic device using various interfaces and lines.

[0230] The memory can be used to store the computer programs and / or modules, and the processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0231] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0232] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0233] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. An efficient network data caching method based on edge computing, characterized in that: include: Obtain real-time data streams from edge nodes, generate content feature vectors, and obtain a quantitative representation of the data's dynamic characteristics; Calculating the popularity value of the data based on the content feature vector; if the popularity value exceeds a preset popularity threshold, marking it as hot data, and obtaining a classification result of the data popularity change; Obtaining the current network topology and the distribution information of the edge nodes from the network management system, and calculating the link state parameters between the nodes; Obtaining a data source identifier in the content feature vector and a node connectivity weight in the link state parameter; if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, selecting the corresponding node as a candidate cache node, thereby obtaining a candidate node set that integrates service priority and network reliability; Obtaining the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter; if the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, filtering out the corresponding stable nodes in the candidate node set to obtain a target cache node set that combines topological stability and request regularity; Determining a target cache node according to the classification result of the data heat change and the target cache node set; Checking the storage space of the node according to the target cache node, and if the remaining space is lower than a preset space threshold, releasing the storage space to obtain the updated cache content; Generate a cache status table according to the updated cache content to obtain a global cache status sharing result; According to the global cache state sharing result, it is detected whether the data identifier is repeatedly stored in multiple nodes. If repeated storage exists, the node storage with the highest heat value is retained to obtain an optimized cache distribution.

2. The efficient network data caching method based on edge computing according to claim 1 is characterized in that: The data popularity value is calculated based on the content feature vector. If the popularity value exceeds a preset popularity threshold, the data is marked as popular data. The classification result of the data popularity change is obtained, including: Obtain user interaction data from a data source, extract feature dimensions such as page views, likes, and comments, and perform vector normalization to generate the initial content feature vector; According to a pre-established popularity evaluation model, the initial content feature vector is input, combined with the weight coefficient, and the weighted sum of each feature dimension is calculated through a linear regression algorithm to obtain the popularity value of the data; If the popularity value exceeds a preset popularity threshold, it is marked as hot data, otherwise it is marked as non-hot data, and the classification result is generated based on the mark; The heat values ​​of consecutive time points corresponding to the classification results are obtained, and the initial content feature vector is adjusted through a dynamic update mechanism to generate a classification result of the data heat change.

3. The efficient network data caching method based on edge computing according to claim 1 or 2, characterized in that: The obtaining of the current network topology and the distribution information of the edge nodes from the network management system and the calculation of the link state parameters between the nodes include: Obtaining the current network topology and distribution information of edge nodes from the network management system, combining it with real-time link monitoring data, and storing it in a distributed database to obtain a structured network data set; Based on the structured network data set, a graph analysis algorithm is used to calculate the link state parameters between each node, including the delay distribution between nodes and the link bandwidth utilization, and the connectivity weight is determined to obtain a link state parameter set; Afterwards, the method further includes: if the inter-node delay in the link state parameter set exceeds a preset delay threshold, calculating the path dynamic change frequency to obtain a network state fluctuation data set; According to the network status fluctuation data set, a statistical analysis tool is used to calculate the fluctuation amplitude and frequency distribution, and a quantitative description report is generated through a visualization tool to obtain a quantitative description of the network status fluctuation.

4. The efficient network data caching method based on edge computing according to any one of claims 1 to 3, characterized in that: The step of obtaining the data source identifier in the content feature vector and the node connectivity weight in the link state parameter, and selecting the corresponding node as a candidate cache node if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, thereby obtaining a candidate node set that integrates service priority and network reliability, including: Extracting a data source identifier from the content feature vector, querying a pre-established service priority mapping table through a distributed database, and generating a service priority data set if the data source identifier corresponds to a high-priority service; According to the service priority data set and in combination with the node connectivity weight in the link state parameter, if the service priority is higher than a preset priority threshold and the node connectivity weight is higher than a preset weight threshold, determining a candidate cache node set by a path calculation tool; According to the candidate cache node set, cache node capacity and service traffic distribution data of each node are obtained and processed through a clustering tool to obtain an optimized node subset; According to the optimized node subset, a weighted scoring mechanism is adopted to comprehensively consider the service priority, the node connectivity weight and the cache node capacity to calculate the score of each node and obtain the final candidate node set.

5. The efficient network data caching method based on edge computing according to any one of claims 1 to 4, characterized in that: The time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter are obtained. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain a target cache node set that combines topological stability and request regularity, including: Obtaining time series characteristics from the data request log, extracting the periodic pattern of the data request using a Fourier transform algorithm, generating a periodic feature vector, and obtaining a quantized result of the periodic feature vector; Calculating a path dynamic change frequency according to the link state parameters, and if the dynamic frequency is lower than a preset change threshold, screening nodes that meet the conditions from the candidate node set to determine the initial stable node set; According to the initial stable node set, a topological stability index of each node is obtained, and the periodic eigenvector and the topological stability index are fused by a weighted average algorithm to obtain the fused eigenvector; If the value of the fused feature vector is higher than a preset vector threshold, the corresponding node is included in the target cache node set to generate the target cache node set that is a fusion of the topology stability and request regularity.

6. The efficient network data caching method based on edge computing according to any one of claims 1 to 5, characterized in that: The determining of the target cache node according to the classification result of the data heat change and the target cache node set includes: Obtain content feature vectors from data popularity and traffic distribution, and use clustering algorithms to group classification results; If the heat of the grouped data exceeds a preset heat threshold, the initial candidate node set is obtained; According to the topology structure of the initial candidate node set, link status and node performance parameters are obtained, and if the delay of the link status is lower than a preset delay threshold, the candidate node with high priority is determined by a sorting algorithm; Iteratively optimizing the candidate nodes with high priority by using a reinforcement learning algorithm, inputting content feature vectors, link status, and storage capacity, to obtain an optimal cache location decision; Through the optimal cache location decision and storage capacity, it is determined whether the cache node meets the traffic distribution requirements, and the updated target cache node is determined.

7. The efficient network data caching method based on edge computing according to any one of claims 1 to 6, characterized in that: Generating a cache status table according to the updated cache content to obtain a global cache status sharing result includes: Get cache content from the target node, query the storage location through data identification, and use distributed hash tables to locate edge nodes; If the updated cache content is updated, then the updated data is extracted to obtain the latest cache content; Generate a status table based on the latest cache content, organize the data identifier, storage location and popularity value in a structured manner, and store them in key-value pairs; If the heat value exceeds a preset heat threshold, it is marked as high-frequency data and a state table is obtained; Synchronize the state table using a distributed consensus algorithm and broadcast the state table among edge nodes; If the data between nodes is inconsistent, the conflict resolution mechanism is triggered to obtain the synchronized state table; The edge node data is merged through the synchronized state table to obtain the global state, and the sharing result is calculated using an aggregation function. If there is a data conflict, the latest data is selected according to the timestamp to obtain the global cache state sharing result.

8. An efficient network data caching system based on edge computing, characterized in that: include: The generation module is used to obtain real-time data streams from edge nodes, generate content feature vectors, and obtain a quantitative representation of the dynamic characteristics of the data; a marking module, configured to calculate the popularity value of the data based on the content feature vector, and mark the data as hot data if the popularity value exceeds a preset popularity threshold, thereby obtaining a classification result of the data popularity change; A calculation module is used to obtain the current network topology and the distribution information of the edge nodes from the network management system and calculate the link state parameters between the nodes; a fusion module, configured to obtain a data source identifier in the content feature vector and a node connectivity weight in the link state parameter; if the data source identifier corresponds to a high-priority service and the node connectivity weight is higher than a preset weight threshold, select the corresponding node as a candidate cache node, and obtain a candidate node set that integrates service priority and network reliability; An acquisition module is configured to obtain the time series characteristics in the content feature vector and the path dynamic change frequency in the link state parameter. If the data request is periodic and the path dynamic change frequency is lower than a preset change threshold, the corresponding stable nodes in the candidate node set are screened out to obtain a target cache node set that combines topological stability with request regularity. a determination module, configured to determine a target cache node according to the classification result of the data heat change and the target cache node set; An update module is configured to check the storage space of a node according to the target cache node, and if the remaining space is lower than a preset space threshold, release the storage space to obtain updated cache content; A first cache module, configured to generate a cache status table according to the updated cache content to obtain a global cache status sharing result; The second cache module is used to detect whether the data identifier is repeatedly stored in multiple nodes based on the global cache state sharing result. If there is repeated storage, the node storage with the highest heat value is retained to obtain an optimized cache distribution.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for efficient caching of network data based on edge computing as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the efficient network data caching method based on edge computing as described in any one of claims 1 to 7.

Images