Data hierarchical caching and intelligent resuming method and device in off-network environment

Abstract

The application relates to a data grading cache and intelligent resuming transmission method and device in a network interruption environment, which comprises the following steps: obtaining to-be-transmitted data, grading evaluation based on data importance and timeliness to obtain corresponding priority data groups; real-time monitoring of network bandwidth state, recording the interruption time and triggering the grading cache mechanism when detecting network interruption, and setting the corresponding storage area; allocating the priority data groups to the corresponding storage area for caching to obtain the final cache area; when detecting network recovery, intelligently resuming transmission of the priority data groups according to the cache area, and generating a resuming transmission report; real-time detection of the network bandwidth state, dynamic resuming transmission adjustment of the priority data groups, marking of the priority data groups after resuming transmission, and obtaining the marked data groups; integrity check of the marked data groups, and generation of a transmission state report in combination with the resuming transmission report. The application can adapt to different network environments and data transmission requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data hierarchical caching and intelligent resuming, in particular to a data hierarchical caching and intelligent resuming method and device in a network interruption environment. BACKGROUND

[0002] With the rapid development of information technology and the increasing demand for data transmission, data transmission in network environment has become an important part of modern communication systems. However, in practical applications, the instability and interruption of network connection still exist, which poses a great challenge to the timely and complete transmission of data. Especially in some critical business scenarios, the reliability and timeliness of data transmission are directly related to the overall performance of the system and user experience. Therefore, how to ensure the effective transmission of data in a network interruption environment and realize intelligent resuming after network recovery has become one of the current research focuses. Traditional data transmission methods usually only consider the network connection state, ignoring the importance and timeliness of data itself, which may lead to the failure of timely transmission of critical data in the case of limited network resources. SUMMARY

[0003] The main purpose of the present application is to provide a data hierarchical caching and intelligent resuming method and device in a network interruption environment, which can adapt to different network environments and data transmission requirements.

[0004] To achieve the above purpose, the present application provides a data hierarchical caching and intelligent resuming method in a network interruption environment, comprising:

[0005] Obtaining the data to be transmitted, performing hierarchical evaluation based on the importance and timeliness of the data, and obtaining the corresponding priority data group;

[0006] Real-time monitoring of network bandwidth state, recording the interruption time and triggering the hierarchical caching mechanism when network interruption is detected, and setting the corresponding storage area;

[0007] Allocating the priority data group to the corresponding storage area for caching to obtain the final cache area;

[0008] When network recovery is detected, the priority data group is intelligently resumed according to the cache area, and a resuming report is generated;

[0009] Real-time detection of network bandwidth state, dynamic resuming adjustment of the priority data group, and marking of the priority data group completed by resuming to obtain a marked data group;

[0010] Integrity check of the marked data group, and generation of a transmission status report in combination with the resuming report.

[0011] Further, the obtaining the data to be transmitted, based on data importance and timeliness, is classified and evaluated to obtain a corresponding priority data group, comprising:

[0012] The attribute extraction is performed on the data to be transmitted to obtain a data creation timestamp and a data service type identifier;

[0013] According to the preset service classification rule library and the data service type identifier, the data to be transmitted is classified and encoded to obtain an identifier data record;

[0014] The current timestamp is obtained, and the difference between the current timestamp and the data creation timestamp is calculated to obtain a data existence duration, and the data existence duration is weighted according to a preset timeliness threshold to obtain a timeliness weight coefficient;

[0015] The identifier data record and the timeliness weight coefficient are standardized scored to obtain a comprehensive importance score;

[0016] According to a preset importance score interval threshold group, the comprehensive importance score is prioritized and grouped to obtain the priority data group including a high priority group, a medium priority group and a low priority group.

[0017] Further, the standardized scoring of the identifier data record and the timeliness weight coefficient to obtain the comprehensive importance score further comprises:

[0018] The importance of the identifier data record is quantified to obtain a data service importance score;

[0019] The timeliness weight coefficient is calculated to obtain a timeliness score;

[0020] According to a preset environmental factor evaluation rule, a multi-dimensional analysis of the current system environment is performed to obtain an environmental factor score;

[0021] The data service importance score, the timeliness score and the environmental factor score are weighted and fused to obtain an initial comprehensive score;

[0022] The initial comprehensive score is standardized and mapped to an interval between 0 and 1 to obtain a mapped initial value;

[0023] A preset mapping parameter is selected to perform mapping iteration calculation on the mapped initial value to obtain a dynamic adjustment factor;

[0024] According to the initial comprehensive score and the dynamic adjustment factor, the importance is calculated to obtain the comprehensive importance score.

[0025] Further, the network bandwidth state is monitored in real time, when network interruption is detected, the interruption time is recorded and the hierarchical cache mechanism is triggered, and the corresponding storage area is set, including:

[0026] The network bandwidth state is periodically detected to obtain network state parameters;

[0027] The network quality is calculated according to the network state parameters to obtain a network quality score;

[0028] The network quality score is compared with a threshold value to determine whether the network is interrupted;

[0029] When it is determined that the network is interrupted, the interruption time and timing data are recorded, and the network interruption duration is calculated;

[0030] The network interruption duration is classified based on a preset duration threshold to obtain an interruption level;

[0031] The interruption level is matched with a preset cache strategy library to obtain a corresponding hierarchical cache strategy;

[0032] The cache management module is initialized and set based on the hierarchical cache strategy, and corresponding data access and storage rules are set;

[0033] A cache index structure is established based on the data access and the storage rules;

[0034] The corresponding storage area is set according to the cache index structure, and the triggering and initialization of the hierarchical cache mechanism are completed.

[0035] Further, the priority data group is allocated to the corresponding storage area for caching to obtain a final cache area, including:

[0036] The priority data group is calculated to obtain the corresponding storage requirement;

[0037] The storage area is partitioned according to the storage requirement to obtain a plurality of storage partitions;

[0038] The access speed of the storage partition is tested to obtain an access speed value;

[0039] The storage partition is sorted according to the access speed value to obtain a speed level sequence;

[0040] The priority data group is matched with the speed level sequence to obtain a priority storage partition correspondence relationship;

[0041] The priority data group is written into the corresponding storage partition according to the priority storage partition correspondence relationship to obtain an initial cache area;

[0042] performing storage space usage calculation on the initial cache area to obtain usage rates of the storage partitions;

[0043] performing space reallocation on the initial cache area according to a preset cyclic coverage strategy and the usage rates to obtain an optimized cache area;

[0044] performing real-time monitoring on the optimized cache area, and when a storage capacity of the optimized cache area reaches a preset capacity threshold, performing data compression and encoding processing on the optimized cache area to obtain compressed cache data;

[0045] re-writing the compressed cache data into the optimized cache area to obtain the final cache area.

[0046] Further, the performing storage space usage calculation on the initial cache area to obtain usage rates of the storage partitions comprises:

[0047] performing multi-scale space division on the initial cache area to obtain scale grids of different scales;

[0048] performing occupancy calculation on each of the scale grids to obtain numbers of occupied grids of each scale;

[0049] performing fractal dimension calculation on the numbers of occupied grids by a preset box-counting method to obtain a space complexity index;

[0050] dividing the initial cache area into storage blocks of a fixed size, and performing data distribution probability calculation on each of the storage blocks to obtain block-level data distribution probabilities;

[0051] performing entropy value calculation on the block-level data distribution probabilities by a preset entropy formula to obtain data distribution entropy;

[0052] performing nonlinear mapping on the storage partitions according to the space complexity index to obtain preliminary usage rates;

[0053] correcting the preliminary usage rates according to the data distribution entropy to obtain corrected usage rates;

[0054] performing access frequency statistics on the storage partitions to obtain an access heat map;

[0055] performing comprehensive statistics on the corrected usage rates according to the access heat map to obtain the usage rates of the storage partitions.

[0056] Further, the when detecting network recovery, intelligently resuming transmission of the priority data groups according to the cache area, and generating a resuming transmission report, comprises:

[0057] Periodically detecting the network bandwidth state, and triggering the intelligent resuming program when detecting the network recovery;

[0058] Reading the storage location information of the priority data set in the cache area, and performing read sorting to generate a resuming queue;

[0059] Performing packet numbering and fragmentation processing on the resuming queue to obtain a data fragment set to be transmitted;

[0060] Calculating the optimal transmission concurrency and single fragment size according to the data fragment set to obtain transmission control parameters;

[0061] Concurrently transmitting the data fragment set based on the transmission control parameters, and recording the transmission state of each data fragment in real time to obtain initial resuming information;

[0062] When detecting the data fragment transmission failure, rejoining the data fragment to the resuming queue and recording the retry number;

[0063] Integrating the initial resuming information and the retry number to obtain the resuming report.

[0064] Further, the network bandwidth state is detected in real time, and the priority data set is dynamically adjusted for resuming, and the priority data set completed by resuming is marked to obtain a marked data set, including:

[0065] Periodically sampling and measuring the network bandwidth state to obtain bandwidth sampling data;

[0066] Performing network bandwidth feature calculation according to the bandwidth sampling data to obtain bandwidth statistical features;

[0067] Matching the bandwidth statistical features with a preset adjustment strategy to obtain a current transmission adjustment strategy;

[0068] Adjusting the transmission rate and concurrency of the priority data set based on the current transmission adjustment strategy to obtain adjusted transmission parameters;

[0069] Adjusting the transmission of the priority data set in blocks according to the adjusted transmission parameters to obtain transmission data blocks;

[0070] Performing checksum calculation on each transmission data block to obtain data block checksum information;

[0071] Sending the data block checksum information and the transmission data block to the receiving end to obtain a sending result;

[0072] Receiving the transmission confirmation information returned by the receiving end to obtain a transmission state;

[0073] marking the priority data set which is successfully transmitted according to the transmission state, to obtain the marked data set;

[0074] removing the marked data set from the priority data set, to obtain an updated priority data set;

[0075] repeating the transmission process on the updated priority data set until the transmission is completed.

[0076] Further, the marked data set is integrity checked, and a transmission state report is generated in combination with the retransmission report, comprising:

[0077] checking and calculating the checksum of each marked data set to obtain a checksum value, and comparing the checksum value with a preset standard value to obtain an integrity check result;

[0078] analyzing the success rate of the integrity check result to obtain a corresponding overall transmission state;

[0079] marking the marked data set according to the overall transmission state to obtain a data set to be retransmitted;

[0080] retransmitting the data set to be retransmitted based on a preset upper limit of the number of retransmissions to obtain a retransmission result;

[0081] identifying the data set to be retransmitted that exceeds the upper limit of the number of retransmissions according to the retransmission result, to obtain a failed data set, processing the failed data set according to a preset degradation strategy, and updating the overall transmission state to obtain a corresponding updated transmission state;

[0082] transmitting the marked data set based on the updated transmission state to obtain corresponding transmission statistical information;

[0083] classifying and summarizing the transmission statistical information according to transmission anomalies to obtain corresponding transmission anomaly information;

[0084] integrating the retransmission report, the transmission statistical information and the transmission anomaly information to obtain the transmission state report.

[0085] The application also provides a data hierarchical caching and intelligent retransmission device in a network interruption environment, which is applied to any one of the data hierarchical caching and intelligent retransmission methods in a network interruption environment, and comprises:

[0086] a collection module, which is used to obtain data to be transmitted, and perform hierarchical evaluation based on data importance and timeliness to obtain a corresponding priority data set;

[0087] An analysis module is configured to monitor network bandwidth state in real time, record the time of network interruption when interruption is detected, trigger a hierarchical caching mechanism, and set a corresponding storage area;

[0088] An association module is configured to cache the priority data set in the corresponding storage area to obtain a final cache area;

[0089] A processing module is configured to intelligently resume transmission of the priority data set according to the cache area when network recovery is detected, and generate a resume transmission report;

[0090] A control module is configured to detect network bandwidth state in real time, dynamically adjust the dynamic resume transmission of the priority data set, and mark the priority data set that has completed resume transmission to obtain a marked data set;

[0091] An execution module is configured to perform integrity check on the marked data set, and generate a transmission status report in combination with the resume transmission report.

[0092] The application provides a data hierarchical caching and intelligent resume transmission method and device in a network interruption environment, which has the following beneficial effects:

[0093] By hierarchical evaluation based on data importance and timeliness, intelligent classification of data to be transmitted is realized, key data can be preferentially transmitted when network resources are limited, and the problem that traditional methods ignore data characteristics is effectively solved. By using a hierarchical caching mechanism, storage areas are allocated according to data priority, rational use of storage resources is realized, the risk of loss of key data and waste of storage space is avoided, and the efficiency of data management is improved. By real-time monitoring of network bandwidth state and dynamic adjustment of the priority data set, adaptive optimization of the transmission process is realized, the transmission efficiency after network recovery is improved, and the problem of lack of flexibility of traditional resume transmission methods is solved. The marked data set is subjected to integrity check, and a transmission status report is generated in combination with the resume transmission report, a comprehensive data transmission monitoring mechanism is established, and the reliability and traceability of the transmission process are effectively improved. Through intelligent resume transmission and dynamic adjustment strategy, the system can better adapt to different network environments and data transmission requirements, improve the overall transmission efficiency and system reliability. The application comprehensively considers data characteristics, network state and storage resources, realizes intelligent data management and transmission in a network interruption environment, and provides a more reliable and efficient solution for data transmission in various application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0094] Figure 1 is a data hierarchical caching and intelligent resume transmission method flowchart provided by the application in a network interruption environment;

[0095] Figure 2 is a data hierarchical caching and intelligent resume device structure in a network interruption environment provided by the application.

[0096] The implementation, functional features and advantages of the application will be further described with reference to the drawings and embodiments. DETAILED DESCRIPTION

[0097] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0098] The application will be further described below in combination with the drawings and specific embodiments.

[0099] Referring to Figure 1 The application provides a data hierarchical caching and intelligent resume method in a network interruption environment, characterized by comprising:

[0100] Step S1: obtaining the data to be transmitted, performing hierarchical evaluation based on the importance and timeliness of the data to obtain the corresponding priority data group;

[0101] Step S2: real-time monitoring of network bandwidth state, when network interruption is detected, recording the interruption time and triggering the hierarchical caching mechanism, and setting the corresponding storage area;

[0102] Step S3: allocating the priority data group to the corresponding storage area for caching to obtain the final cache area;

[0103] Step S4: when network recovery is detected, intelligently resuming the priority data group according to the cache area, and generating a resume report;

[0104] Step S5: real-time detection of network bandwidth state, dynamic resume adjustment of the priority data group, and marking of the priority data group completed by resume to obtain the marked data group;

[0105] Step S6: integrity check of the marked data group, and generation of a transmission status report in combination with the resume report.

[0106] Based on the above steps, the detailed step process is shown as follows:

[0107] Step S1: establishing a data interface to connect various data source systems. Set the data acquisition period to obtain the data to be transmitted at a fixed time or in real time. The obtained data is preliminarily cleaned and formatted to ensure the consistency of the data.

[0108] Establish data importance evaluation rules, including:

[0109] Data type weight (e.g., critical business data, general business data, log data, etc.), data source weight (e.g., core system, auxiliary system, etc.), and data impact range (e.g., global impact, local impact, etc.).

[0110] Apply evaluation rules to each piece of data and calculate importance scores.

[0111] Establish timeliness evaluation criteria, such as real-time requirements (data that needs to be processed immediately), quasi-real-time requirements (data that allows for short delays), and batch processing requirements (data that can be delayed). Assign a timeliness label to each piece of data.

[0112] Design a comprehensive scoring algorithm that combines importance scores and timeliness labels to calculate the comprehensive priority score of each piece of data.

[0113] Define priority levels, such as high, medium, and low, or more detailed multi-levels. According to the comprehensive priority score, divide the data into corresponding priority groups. Within each priority group, sort the data by score from high to low.

[0114] Assign a unique identifier to each priority group. Record the data volume, average priority score, and other statistical information of each group. Generate priority data group metadata, including group identification, data list, priority information, etc.

[0115] Step S2: Real-time monitoring of network bandwidth status, when network interruption is detected, record the interruption time and trigger the hierarchical caching mechanism, and set the corresponding storage area.

[0116] Deploy network monitoring probes to continuously collect network performance indicators. Set the sampling frequency, such as sampling once per second or per minute. Collect key indicators, including uplink and downlink bandwidth utilization, network delay, packet loss rate, and connection status.

[0117] Define network state evaluation rules, such as normal (bandwidth utilization <80%, delay <100ms, packet loss rate <1%), warning (bandwidth utilization 80%-95%, delay 100ms-500ms, packet loss rate 1%-5%), severe (bandwidth utilization >95%, delay >500ms, packet loss rate >5%), and interruption (unable to establish a connection or continuous multiple severe states).

[0118] Real-time comparison of current network state and interruption definition. When interruption is detected, immediately record the precise timestamp of interruption occurrence, the last successful transmission packet information, and the network state history data before interruption. Send a trigger signal to the cache management module. Pass the interruption time information and the last transmission state.

[0119] According to the preset storage strategy, the cache storage area is dynamically allocated. The factors to be considered include the size of the available storage space, the estimated interruption duration, and the data priority distribution. The storage space proportion corresponding to different priority data is allocated. A storage area mapping table is generated, which contains the starting address, size, and corresponding data priority of each area. The network state is continuously monitored during the caching process. The conditions for detecting network recovery are prepared to switch back to the normal transmission mode in time.

[0120] Step S3: Based on the priority data group, the preset caching strategy is adopted to write the data to be transmitted into the corresponding storage area, and the data caching is completed.

[0121] According to the characteristics of the priority data group, the caching strategy is designed, including high-priority data (priority caching, allocating the most reliable storage area), medium-priority data (balanced storage, balancing between reliability and capacity), and low-priority data (large-capacity storage, accepting lower reliability).

[0122] The storage area mapping table generated in step S2 is read. The corresponding storage area is allocated for each priority data group.

[0123] The data groups are processed in priority order from high to low. For each data group, the data group metadata is read, the single data is extracted according to the priority order within the group, the data is written into the allocated storage area, and the storage area usage is updated. Each write operation is checked to ensure data integrity. The caching location, timestamp, and verification information of each data are recorded. The data group metadata is updated to mark the cached data. The final cache area is obtained.

[0124] Step S4: The network state is continuously monitored, and the network recovery judgment standard (such as continuous multiple sampling showing that the network index has returned to normal) is set. Once the network recovery is detected, the resuming mechanism is triggered immediately. The cache area mapping table and data group metadata are read to obtain the priority and storage location information of the data to be resumed. The data groups are processed in priority order from high to low. For each data group, the single data is extracted according to the priority order within the group, and the resuming is performed. During the transmission process, the transmission status of each data is recorded in real time, including the start time, completion time, transmission speed, and retry count. For the successfully transmitted data, the state in the data group metadata is updated. If the transmission fails, the retry strategy is applied, and the failure reason is recorded. The transmission order is dynamically adjusted, the transmission strategy of subsequent data is optimized according to the real-time network status and transmission success rate, and a real-time resuming report is generated, containing the transmitted data volume, remaining data volume, average transmission speed, and estimated completion time. The resuming report is updated regularly to provide real-time transmission progress for management personnel.

[0125] Step S5: Deploy bandwidth monitoring module, continuously collect network bandwidth utilization, available bandwidth and other indicators. Set the sampling frequency, such as once per second or once per minute. Define bandwidth state evaluation rules, such as high speed (available bandwidth > 90%), medium speed (available bandwidth 50%-90%), low speed (available bandwidth < 50%). According to the real-time bandwidth state, dynamically adjust the data transmission strategy.

[0126] In high-speed state, increase the number of concurrent transmissions to improve transmission efficiency. In medium-speed state, maintain balanced transmission strategy. In low-speed state, reduce the number of concurrent transmissions, and give priority to the transmission of high-priority data. Update the transmission progress of each priority data group in real time.

[0127] When all data in a data group is transmitted, mark it as "completed" state. Update the data group metadata, record the completion time, total transmission data volume, average transmission speed and other information. Generate a marked data group list containing all completed transmission data groups and their related statistical information. Continuously monitor the transmission status of incomplete data groups, dynamically adjust the transmission order and resource allocation according to network conditions and remaining data volume.

[0128] Step S6: Perform integrity check on each data group marked as "completed". Read the original check information in the data group metadata. Recalculate the check value of the transmitted data and compare it with the original check information. If inconsistencies are found, mark the data group as "need to retransmit" and add it to the transmission queue again. For data groups that pass the check, update their status to "verification completed". Aggregate the transmission and verification results of all data groups. Combine the retransmission report generated in step S4 with the statistical data in the transmission process. Generate a detailed transmission status report, including the total number of transmissions.

[0129] The application provides a data hierarchical caching and intelligent resuming method in a network interruption environment, which realizes intelligent classification of data to be transmitted by hierarchical evaluation based on data importance and timeliness, ensures that critical data can be transmitted preferentially when network resources are limited, and effectively solves the problem that traditional methods ignore data characteristics. The hierarchical caching mechanism is adopted to allocate storage areas according to data priority, realizing rational use of storage resources, avoiding the risk of loss of critical data and waste of storage space, and improving the efficiency of data management. Through real-time monitoring of network bandwidth state and dynamic resuming adjustment of priority data groups, adaptive optimization of the transmission process is realized, the transmission efficiency after network recovery is improved, and the problem of lack of flexibility of traditional resuming methods is solved. The integrity of the marked data group is checked, and a transmission status report is generated in combination with the resuming report to establish a comprehensive data transmission monitoring mechanism, effectively improving the reliability and traceability of the transmission process. Through intelligent resuming and dynamic adjustment strategies, the system can better adapt to different network environments and data transmission requirements, improving the overall transmission efficiency and system reliability. The application comprehensively considers data characteristics, network state and storage resources, realizes intelligent data management and transmission in a network interruption environment, and provides a more reliable and efficient solution for data transmission in various application scenarios.

[0130] In one embodiment, the data to be transmitted is obtained, hierarchical evaluation is performed based on data importance and timeliness, and corresponding priority data groups are obtained, including:

[0131] The data to be transmitted is subjected to attribute extraction to obtain data creation timestamps and data service type identifiers. In the attribute extraction process, the metadata information of the data file is parsed to extract the creation time and service type of the data and other key attributes. These attributes are the basis for subsequent data hierarchical evaluation.

[0132] The data to be transmitted is classified and coded according to a preset service classification rule library and data service type identifier to obtain identified data records. The service classification rule library contains the importance weight and classification standard of different service types. According to the extracted service type identifier, the rule library is matched to assign each data a corresponding classification code. This process ensures the consistency and traceability of data classification.

[0133] The current timestamp is obtained, and the difference between the current timestamp and the data creation timestamp is calculated to obtain the data existence duration. The data existence duration is subjected to weight calculation according to a preset timeliness reference threshold to obtain a timeliness weight coefficient. The timeliness reference threshold is preset according to the timeliness requirements of different service types. By comparing the data existence duration with the corresponding timeliness reference threshold, the weight coefficient reflecting the timeliness of the data is calculated. This step considers the time sensitivity of the data and gives higher weight to data with high timeliness requirements.

[0134] The identification data record and the timeliness weight coefficient are standardized scored to obtain a comprehensive importance score. The standardized scoring process adopts a standardization algorithm to unify the scoring indicators of different dimensions to the same scale. The comprehensive importance score of each piece of data is obtained by weighted calculation considering the business importance and timeliness of the data. The score reflects the overall importance of the data at the current time point.

[0135] The comprehensive importance score is prioritized and grouped according to a preset importance score interval threshold group to obtain a priority data group including a high priority group, a medium priority group and a low priority group. The importance score interval threshold group is predefined according to historical data statistics and business requirements. The comprehensive importance score of each piece of data is compared with the thresholds to divide the data into the corresponding priority groups. This grouping method provides clear priority guidance for subsequent data transmission and caching strategies.

[0136] The embodiment realizes accurate classification of data by attribute extraction and classification coding of the data to be transmitted, laying a foundation for subsequent priority evaluation. Based on timeliness weight calculation and standardized scoring, the system can dynamically evaluate the importance of the data to ensure that key data with high timeliness is given priority. The preset importance score interval threshold group is used for priority sorting and grouping, making the priority division of data more scientific and reasonable, effectively improving the utilization efficiency of network resources. By optimizing the data transmission order, not only the transmission success rate of key data is improved, but also the data backlog after network recovery is reduced, greatly improving the overall performance and reliability of the system.

[0137] In one embodiment, the identification data record and the timeliness weight coefficient are standardized scored to obtain a comprehensive importance score, further comprising:

[0138] The importance of the identification data record is quantified to obtain a data business importance score. The quantification process is based on a preset business importance evaluation rule, considering factors such as the business type, usage frequency and correlation of the data, to assign an importance value to each data record.

[0139] The timeliness weight coefficient is calculated for timeliness to obtain a timeliness score. The timeliness calculation adopts a predefined timeliness function that considers time attributes such as data generation time, validity period and update cycle to convert the timeliness weight coefficient into a standardized timeliness score.

[0140] The current system environment is analyzed in multiple dimensions according to a preset environmental factor evaluation rule to obtain an environmental factor score. The evaluation rule covers multiple dimensions such as network status, storage capacity and processing capacity, and generates a quantitative score reflecting the current system environment through comprehensive evaluation of the indicators in each dimension.

[0141] The data service importance score, the timeliness score and the environmental factor score are weighted and fused to obtain an initial comprehensive score. The fusion process adopts a preset weight distribution scheme to ensure that the contribution of each factor to the final score matches its actual importance.

[0142] The initial comprehensive score is standardized and mapped to an interval between 0 and 1 to obtain a Logistic mapping initial value. The standardization process adopts a maximum and minimum value normalization method to ensure that scores of different scales can be compared in the same interval.

[0143] The Logistic mapping initial value is iteratively calculated by selecting a preset Logistic mapping parameter to obtain a dynamic adjustment factor. The Logistic mapping introduces a nonlinear characteristic, enhancing the sensitivity of the scoring system to boundary conditions and improving the discrimination of the score.

[0144] The initial comprehensive score and the dynamic adjustment factor are used to calculate the importance to obtain a comprehensive importance score. The importance calculation adopts a predefined calculation formula to nonlinearly combine the initial comprehensive score and the dynamic adjustment factor to generate the final comprehensive importance score. This score fully considers the business importance, timeliness and current system environment of the data, providing a reliable quantitative basis for subsequent data caching and resuming decisions.

[0145] This embodiment realizes comprehensive quantification of data value by scoring the identification data record in multiple dimensions, including business importance, timeliness and environmental factors, providing a reliable basis for caching decisions. The introduction of Logistic mapping iterative calculation enhances the sensitivity of the scoring system to boundary conditions and improves the discrimination of the score. The weighted fusion and nonlinear combination method ensures that the influence of each evaluation factor on the final score matches its actual importance. Standardization and 0-1 interval mapping enable scores of different scales to be compared under a unified standard. This comprehensive scoring mechanism can effectively identify and prioritize critical data, improving data management efficiency and resource utilization in a disconnected environment, while ensuring the intelligence and accuracy of data resuming, significantly enhancing the adaptability and performance of the system in complex network environments.

[0146] In one embodiment, the network bandwidth state is monitored in real time, and when a network interruption is detected, the interruption time is recorded and a hierarchical caching mechanism is triggered, and a corresponding storage area is set, including:

[0147] The network bandwidth state is periodically detected to obtain network state parameters. The network state parameters include current bandwidth, delay, packet loss rate and other indicators. The interval of periodic detection is set according to actual needs, and the time length is set to several seconds to several minutes.

[0148] The network quality score is obtained by calculating the network quality based on the network state parameters. The network quality score is a comprehensive evaluation of the network state parameters. A weighted calculation method is used to assign different weights to each parameter according to its importance, and finally a score between 0 and 100 is obtained.

[0149] The network quality score is compared with a threshold value to determine whether the network is interrupted. A network quality score threshold is set, and when the score is lower than the threshold, it is determined that the network is interrupted. The selection of the threshold needs to balance the sensitivity and stability, and it is set between 20-30 points.

[0150] When it is determined that the network is interrupted, the interruption time and timing data are recorded, and the duration of the network interruption is calculated. The interruption time is accurate to the millisecond level, and the timing data is used for subsequent duration calculation. The duration calculation uses a real-time updating mechanism, which updates every second.

[0151] Based on the preset duration threshold, the duration of the network interruption is classified to obtain the interruption level. The duration threshold can be set to multiple levels, such as 1 minute, 5 minutes, 30 minutes, etc., corresponding to different interruption levels. The interruption level reflects the severity of the network interruption, which is used for the selection of the subsequent cache strategy.

[0152] The interruption level is matched with the preset cache strategy library to obtain the corresponding hierarchical cache strategy. The cache strategy library contains multiple predefined cache strategies, each of which corresponds to different interruption levels and data types. The strategy matching process considers factors such as interruption level, data importance, and system resources.

[0153] Based on the hierarchical cache strategy, the cache management module is initialized and set, and the corresponding data access and storage rules are set. The initialization settings include cache space allocation, cache update frequency, etc. The data access and storage rules define the cache priority, retention time, and access rights of different types of data.

[0154] Based on the data access and storage rules, the cache index structure is established. The cache index structure is implemented using a multi-level hash table, which supports fast lookup and update. The index structure contains data identification, storage location, timestamp, etc. information, which is convenient for subsequent data management and resuming operation.

[0155] According to the cache index structure, the corresponding storage area is set, and the triggering and initialization of the hierarchical cache mechanism are completed. The storage area division considers data type, importance, and access frequency, and adopts a hierarchical storage strategy. High-priority data is stored in a fast access area, and low-priority data can be stored in a slower storage medium. After initialization, the hierarchical cache mechanism starts running, realizing intelligent management and resuming preparation of data in a disconnected network environment.

[0156] The embodiment can detect network interruption in a timely manner and improve the response speed and accuracy of the system to network changes by monitoring the network bandwidth state in real time and performing dynamic evaluation. The interruption grading mechanism based on a preset time threshold realizes fine processing of network interruptions of different severity, enhancing the adaptability of the system. The hierarchical cache strategy and adaptive data access storage rules effectively improve the data management efficiency and resource utilization rate in a network interruption environment. The cache index of the multi-level hash table structure realizes fast data searching and updating, optimizing the cache performance. The hierarchical storage strategy reasonably allocates storage resources according to the importance and access frequency of data, ensuring the availability of key data while improving the overall storage efficiency. These measures together ensure the reliability and availability of data in a network interruption environment, laying a foundation for subsequent intelligent resuming and significantly improving the running efficiency and user experience of the system in an unstable network environment.

[0157] In one embodiment, the priority data groups are allocated to corresponding storage areas for caching to obtain final cache areas, including:

[0158] The storage requirement of the priority data groups is calculated to obtain the corresponding storage requirement. The storage requirement calculation considers factors such as data size, growth rate, and reserved space, and uses a dynamic prediction algorithm to ensure accurate estimation of the storage requirements of different priority data.

[0159] The storage area is partitioned according to the storage requirement to obtain a plurality of corresponding storage partitions. The partitioning operation uses an adaptive partitioning strategy to dynamically adjust the partition size and number according to the ratio of storage requirement and available storage space, ensuring the rational allocation of storage resources.

[0160] The access speed of the storage partition is tested to obtain an access speed value. The access speed test uses a combination of random reading and writing and sequential reading and writing, and takes the average of multiple tests to eliminate the influence of accidental factors and obtain an accurate access speed evaluation.

[0161] The storage partitions are sorted according to the access speed value to obtain a speed level sequence. The sorting process uses a quicksort algorithm to arrange the storage partitions from high to low access speed, forming a clear speed level sequence to provide a basis for subsequent data allocation.

[0162] The priority data groups are matched with the speed level sequence to obtain the priority storage partition correspondence. The matching process takes into account data priority and access frequency, with high-priority and high-frequency access data allocated to high-speed storage partitions and low-priority data allocated to slower storage partitions, achieving efficient use of storage resources.

[0163] The priority data group is written into the corresponding storage partition according to the priority storage partition correspondence relationship, to obtain an initial cache area. The writing process adopts a parallel writing technology to improve the data writing efficiency. Meanwhile, a data index structure is established to facilitate subsequent rapid retrieval and access.

[0164] The storage space usage rate of the initial cache area is calculated to obtain the usage rate of each storage partition. The usage rate calculation adopts a real-time monitoring mechanism to periodically update the usage of each partition, providing a basis for subsequent space optimization.

[0165] According to the preset cyclic coverage strategy and the usage rate, the initial cache area is subjected to space reallocation to obtain an optimized cache area. The cyclic coverage strategy defines the coverage rules of data of different priorities, and the high-priority data is not covered, and the low-priority data can be dynamically adjusted in storage space according to the usage rate. During the space reallocation process, the data integrity and consistency are considered to ensure that important data will not be lost.

[0166] The optimized cache area is monitored in real time, and when the storage capacity of the optimized cache area reaches a preset capacity threshold, the optimized cache area is subjected to data compression and encoding processing to obtain compressed cache data. The data compression adopts an adaptive compression algorithm, and according to the data type, an appropriate compression method is selected to balance between the compression rate and the decompression speed. The preset capacity threshold is usually set to 80%-90% of the total storage space to reserve sufficient operation space.

[0167] The compressed cache data is rewritten into the optimized cache area to obtain a final cache area. During the rewriting process, an incremental updating strategy is adopted to update only the data blocks that have changed, thereby reducing the influence of the writing operation on the system performance.

[0168] The embodiment realizes the reasonable allocation of storage resources by calculating the storage demand of the priority data group and adaptive partitioning, and improves the storage space utilization efficiency. The access speed test and sorting mechanism are adopted to ensure that the high-priority data is stored in the high-speed partition, thereby optimizing the data access performance. The dynamic monitoring and space reallocation strategy ensure the flexible adjustment of the storage space to adapt to the storage demand of different data types. The real-time monitoring combined with the adaptive compression algorithm effectively improves the storage capacity while ensuring the data integrity. The incremental updating strategy reduces the influence of data rewriting on the system performance, thereby ensuring the stability of the cache system. The overall scheme realizes the efficient storage, rapid access and intelligent management of data in the offline environment, improves the reliability and data security of the system, and lays a foundation for the data continuation in the offline environment.

[0169] In one embodiment, the storage space usage rate of the initial cache area is calculated to obtain the usage rate of each storage partition, including:

[0170] The initial cache area is divided into multiple scales of space to obtain scale grids of different scales. The multi-scale space division is a grid processing of the cache area according to different granularities to capture the spatial characteristics of data distribution. This division method describes the spatial distribution of data from macro to micro levels, which helps to more comprehensively analyze the use of storage space.

[0171] The occupancy of each scale grid is calculated to obtain the number of occupancy grids of each scale. In the occupancy calculation process, the number of grids actually occupied by data under each scale is counted, which reflects the spatial distribution density of data under different scales.

[0172] The number of occupancy grids is calculated by the preset box counting method to obtain the spatial complexity index. Box counting is a commonly used method for calculating fractal dimension, which analyzes the change rule of the number of occupancy grids under different scales to obtain an index describing the complexity of data spatial distribution. This index can quantify the spatial complexity of data distribution, providing an important reference for subsequent storage space usage calculation.

[0173] The initial cache area is divided into fixed-size storage blocks, and the data distribution probability of each storage block is calculated to obtain the block-level data distribution probability. This step divides the cache area into finer-grained storage units and analyzes the data distribution in each storage block to obtain more accurate data distribution information.

[0174] The data distribution probability of the block level is calculated by the preset Shannon entropy formula to obtain the data distribution entropy. Shannon entropy is an important index in information theory to describe uncertainty, which is used here to quantify the uniformity of data distribution. The higher the data distribution entropy, the more uniform the data distribution; otherwise, the data distribution is more concentrated.

[0175] According to the spatial complexity index, the storage partition is nonlinearly mapped to obtain the preliminary usage rate. Nonlinear mapping considers the spatial complexity of data distribution and converts fractal dimension and other indicators into preliminary estimates of storage space usage rate. This mapping method can reflect the influence of non-uniformity of data distribution on the utilization efficiency of storage space.

[0176] According to the data distribution entropy, the preliminary usage rate is corrected to obtain the corrected usage rate. This step takes into account the uniformity of data distribution and adjusts the preliminary usage rate. The more uniform the data distribution, the higher the actual utilization rate of storage space; otherwise, the concentrated distribution of data may lead to excessive use of some storage areas.

[0177] The access frequency of the storage partition is counted to obtain the access heat map. The access heat map reflects the usage frequency of different storage partitions, which is essential for evaluating the actual utilization efficiency of storage space.

[0178] According to the access heat map, the corrected usage rate is comprehensively counted to obtain the usage rate of each storage partition. By combining the data distribution characteristics with the actual access mode, more accurate and meaningful storage space usage rate evaluation results are obtained.

[0179] The embodiment can comprehensively capture the spatial distribution characteristics of data in the cache area through multi-scale space division and box counting method fractal dimension calculation, describe the data distribution from multiple levels from macro to micro, and provide accurate spatial complexity indicators for storage space usage rate calculation. Combined with the data distribution entropy calculated by Shannon entropy, the uniformity of data distribution can be accurately quantified, further improving the accuracy of usage rate evaluation. Through nonlinear mapping and data distribution entropy correction, the influence of non-uniform data distribution on storage space utilization efficiency is effectively considered, making the usage rate evaluation more close to the actual situation. Finally, combined with the access heat map for comprehensive statistics, the data distribution characteristics are combined with the actual access mode, and the obtained storage partition usage rate evaluation results are more accurate and meaningful, providing a reliable basis for subsequent data management and storage optimization, and helping to improve the performance and efficiency of the overall storage system.

[0180] In one embodiment, when network recovery is detected, the priority data group is intelligently resumed according to the cache area, and a resume report is generated, including:

[0181] Periodically detect the network bandwidth state, using timing polling or event triggering. The detection method includes sending small probe packets and measuring the round-trip time, or directly querying the network interface state. When network recovery is detected, i.e. the network connection is re-established and the bandwidth reaches the preset threshold, the intelligent resume program is triggered. Periodic detection ensures timely response to changes in network state, creating conditions for data resume.

[0182] Read the storage location information of the priority data group in the cache area, including the physical address, size, priority, and other metadata of the data. According to the pre-defined priority strategy, these information is sorted to generate a resume queue. The sorting strategy considers the importance, timeliness, size, and other factors of the data to ensure that the most critical data is transmitted first.

[0183] The resume queue is numbered and fragmented. The numbering process assigns a unique identifier to each data packet for tracking and management during transmission. Fragmentation processes large data packets into small pieces suitable for network transmission, improving transmission efficiency and reducing the risk of network congestion. After processing, a set of data fragments to be transmitted is obtained, each fragment with numbering and metadata information.

[0184] The optimal transmission concurrency and single shard size are calculated based on the data shard set to obtain the transmission control parameters. This process involves complex algorithms that take into account current network bandwidth, delay, packet loss rate, and other factors. An adaptive algorithm is used to dynamically adjust these parameters based on network conditions to achieve optimal transmission efficiency. Accurate setting of transmission control parameters is crucial for optimizing the performance of the resume transmission.

[0185] Based on the calculated transmission control parameters, concurrent transmission of the data shard set is performed, with multiple data shards being sent simultaneously to fully utilize network bandwidth. During the transmission process, the transmission status of each data shard is recorded in real time, including start time, completion time, and retransmission number. These records form the initial resume transmission information, providing basic data for subsequent resume transmission reports.

[0186] The transmission status of each data shard is continuously monitored. When a transmission failure of a data shard is detected, such as a timeout or packet loss caused by network fluctuations, the failed data shard is re-added to the resume transmission queue, and the number of retries for the shard is recorded. This mechanism ensures the integrity and reliability of data transmission, and guarantees the final successful transmission of data in unstable network environments.

[0187] The initial resume transmission information and the number of retries are integrated to generate a complete resume transmission report. The report includes overall transmission status, success rate, average transmission speed, and detailed transmission records of each data shard. The resume transmission report provides a comprehensive overview of this data resume transmission, providing important data support for optimization and problem diagnosis. By analyzing the indicators in the resume transmission report, the efficiency and reliability of the resume transmission process are evaluated, providing a basis for subsequent optimization.

[0188] This embodiment realizes efficient and reliable data transmission in a disconnected environment by establishing an intelligent data resume transmission mechanism. Through periodic network state detection and the triggering mechanism of the intelligent resume transmission program, the system can respond to network recovery in a timely manner, avoiding the blindness of data transmission and improving the timeliness of transmission. By sorting and fragmenting the storage location information of the priority data group and combining the predefined priority strategy, the priority transmission of critical data is ensured, improving the accuracy and rationality of data transmission. The adaptive algorithm is used to dynamically calculate the transmission control parameters, and the system can automatically adjust the transmission strategy according to the real-time network conditions, effectively improving the transmission efficiency. Through concurrent transmission and failure retry mechanism, not only the bandwidth utilization is improved, but also the integrity and reliability of data transmission are guaranteed. The final resume transmission report provides an important basis for system optimization, making the entire data transmission process more intelligent and efficient.

[0189] In one embodiment, the network bandwidth state is detected in real time, and the priority data group is dynamically adjusted for resume transmission. The priority data group that has completed resume transmission is marked to obtain a marked data group, including:

[0190] Periodically sample the network bandwidth state to obtain bandwidth sampling data. The time interval of the periodic sampling measurement is set to 100 milliseconds, and the sampling content includes the upload rate, download rate, delay time, and packet loss rate of the current network, and other network state parameters. Real-time values of these parameters are obtained through network probe packets to form a bandwidth sampling data sequence.

[0191] According to the bandwidth sampling data, calculate the network bandwidth characteristics to obtain bandwidth statistical characteristics. The bandwidth statistical characteristics include average transmission rate, fluctuation range, stability index, and congestion probability. In the calculation process, a sliding time window method is used, and the window size is set to 1 minute. The bandwidth sampling data in the window is statistically analyzed to generate a set of feature indicators reflecting the current network state.

[0192] According to the bandwidth statistical characteristics and the preset adjustment strategy, the current transmission adjustment strategy is obtained. The preset adjustment strategy contains multiple network state scenarios and their corresponding transmission parameter configurations. When the bandwidth is sufficient, use high-speed transmission mode; when the bandwidth is limited, enable conservative transmission mode; when the network is unstable, use fault-tolerant transmission mode. By calculating the similarity between the bandwidth statistical characteristics and the preset scenarios, the most matched transmission adjustment strategy is selected.

[0193] Based on the current transmission adjustment strategy, adjust the transmission rate and the number of concurrent priority data groups to obtain adjusted transmission parameters. Transmission parameters include single data block size, transmission rate upper limit, number of concurrent transmissions, and retransmission timeout. In the adjustment process, consider the priority level of the data. High-priority data gets more transmission resource quota to ensure priority transmission of critical data.

[0194] According to the adjusted transmission parameters, adjust the block transmission of the priority data group to obtain the transmission data block. The data block process uses an adaptive block algorithm to dynamically adjust the block size according to the network state. When the network condition is good, use a larger block size to improve transmission efficiency; when the network condition deteriorates, reduce the block size to improve transmission reliability.

[0195] Calculate the checksum for each transmission data block to obtain the data block checksum information. The checksum calculation uses the CRC32 algorithm to ensure the integrity of the transmission data. The checksum information is packaged together with the data block to form a complete transmission unit.

[0196] Send the data block checksum information and the transmission data block to the receiving end to obtain the sending result. In the sending process, a reliable transmission protocol is used to automatically retransmit the data blocks that fail to send. The sending result contains the transmission state identifier of each data block.

[0197] The transmission confirmation information returned by the receiving end is received to obtain the transmission status. The transmission confirmation information includes the reception status and the check result of the data block. For the data block with failed check, it is added to the retransmission queue; for the data block with successful transmission, the transmission completion flag is updated.

[0198] The priority data group with successful transmission is marked according to the transmission status to obtain a marked data group. The marking information includes the transmission completion time, the data integrity verification result and the transmission performance statistical data. These information are used for subsequent transmission quality analysis and optimization.

[0199] The marked data group is removed from the priority data group to obtain an updated priority data group. The priority order of the data group is kept unchanged during the removal process to ensure the continuity of the priority scheduling in the subsequent transmission process.

[0200] The transmission process is repeatedly performed on the updated priority data group until the transmission is completed. During the entire transmission process, the network state change is continuously monitored, and the transmission strategy is dynamically adjusted to realize reliable data transmission.

[0201] The embodiment can flexibly configure data transmission parameters according to the change of the current network environment by real-time monitoring of the network bandwidth state and dynamically adjusting the transmission strategy, and realizes the optimization of data transmission efficiency. The periodic sampling and sliding time window analysis method are adopted to accurately grasp the network state characteristics, so that the adjustment of the transmission strategy is more accurate and reliable. Through the priority grouping and adaptive blocking processing of the data, the priority transmission of important data is ensured, and the stability of the transmission is improved. The CRC32 check and reliable transmission protocol are adopted to effectively ensure the integrity and accuracy of the data transmission. Through the marking and removal mechanism of the transmission completed data, the subsequent transmission process is more efficient, and the resource waste caused by repeated transmission is avoided. The dynamic adjustment mechanism in the entire transmission process significantly improves the reliability and efficiency of data transmission in the offline environment, and provides a strong guarantee for intelligent subsequent transmission of data.

[0202] In one embodiment, the integrity check is performed on the marked data group, and a transmission status report is generated in combination with the subsequent transmission report, including:

[0203] The checksum value of each marked data group is calculated and compared with the preset standard value to obtain the integrity check result. The checksum calculation adopts the CRC32 algorithm to obtain a 32-bit check code through polynomial division operation on each byte in the data group. The preset standard value is stored in the check field of the data packet header as the reference for integrity comparison.

[0204] Success rate analysis is performed on the integrity verification results to obtain the overall transmission status. The success rate analysis is calculated based on a preset transmission quality assessment model, which divides the transmission success rate into a high-quality transmission range (95%-100%), a medium-quality transmission range (80%-95%), and a low-quality transmission range (below 80%). The assessment results reflect the stability of the current transmission environment.

[0205] Based on the overall transmission status, the marked data groups are marked for verification failure, resulting in data groups to be retransmitted. Verification failure marking is achieved by adding a retransmission flag to the data group metadata. This flag contains key information such as the number of retransmissions and the timestamp of the last transmission. The marking process uses atomic operations to ensure data consistency.

[0206] Data is retransmitted based on a preset maximum number of retransmissions for each data group to be retransmitted, resulting in a retransmission result. The maximum number of retransmissions is set differently based on data priority: 5 retransmissions for core business data groups and 3 retransmissions for non-core business data groups. The retransmission process uses a backoff algorithm, with the retransmission interval increasing progressively.

[0207] Based on the retransmission results, data groups exceeding the retransmission limit are identified as failed data groups. These failed data groups are then processed using a preset degradation strategy, including data compression and fragmented transmission. After degradation processing, the overall transmission status is updated, resulting in the updated transmission status. The selection of degradation strategies is based on multiple dimensions, including data type and network environment.

[0208] Based on the updated transmission status, transmission statistics are performed on the marked data groups to obtain transmission statistics information. The transmission statistics employ a sliding time window mechanism with a window size of 30 minutes, and the statistical content includes key indicators such as transmission success rate, average transmission latency, and retransmission rate. Outliers are smoothed during the statistical process to ensure the accuracy of the results.

[0209] Transmission statistics are categorized and summarized to obtain transmission anomaly information. Anomaly classification is based on a pre-defined anomaly feature library, and anomaly types include network timeouts, data corruption, and transmission interruptions. The anomaly feature library is continuously optimized and updated using machine learning methods to improve the accuracy of anomaly identification.

[0210] The transmission status report is generated by integrating the resume report, transmission statistics, and transmission anomaly information. The status report employs a hierarchical structure, including a summary layer, a details layer, and a diagnostic layer. The summary layer displays key performance indicators, the details layer contains specific transmission records, and the diagnostic layer provides anomaly analysis results. During report generation, various types of information are correlated and analyzed to establish a problem tracing mechanism.

[0211] The embodiment realizes the reliability and stability of data transmission in a broken network environment by checking the integrity of the marked data set and generating a transmission status report in combination with the resume report. The method uses CRC32 algorithm for checking and calculation, and combines a preset transmission quality evaluation model for success rate analysis, which can timely detect transmission abnormalities and trigger the retransmission mechanism. Through the differential retransmission strategy and the degradation processing mechanism, the transmission quality of the core business data is guaranteed, and the fault tolerance of the system is improved. Based on the transmission statistics mechanism of the sliding time window, in combination with the abnormal feature library optimized by machine learning, the accurate identification and classification of transmission abnormalities are realized. The state report designed in a hierarchical structure establishes a complete problem tracing mechanism, which is convenient for system operation and maintenance personnel to quickly locate and solve transmission problems. The overall scheme ensures the reliability of data transmission while effectively reducing system resource consumption and improving transmission efficiency through optimized retransmission and degradation strategies.

[0212] Referring to Figure 2 The application also provides a data hierarchical caching and intelligent resuming device in a broken network environment, which is applied to the data hierarchical caching and intelligent resuming method in the broken network environment.

[0213] The acquisition module is used to acquire the data to be transmitted, and perform hierarchical evaluation based on the importance and timeliness of the data to obtain corresponding priority data sets;

[0214] The analysis module is used to monitor the network bandwidth state in real time, record the interruption time and trigger the hierarchical caching mechanism when detecting network interruption, and set corresponding storage areas;

[0215] The association module is used to allocate the priority data sets to the corresponding storage areas for caching to obtain final cache areas;

[0216] The processing module is used to perform intelligent resuming of the priority data sets according to the cache areas when detecting network recovery, and generate a resuming report;

[0217] The control module is used to detect the network bandwidth state in real time, dynamically adjust the resuming of the priority data sets, and mark the priority data sets that have completed resuming to obtain marked data sets;

[0218] The execution module is used to check the integrity of the marked data sets, and generate a transmission status report in combination with the resuming report.

[0219] The application provides a data hierarchical caching and intelligent resuming device in a network interruption environment, which realizes intelligent classification of data to be transmitted by hierarchical evaluation based on data importance and timeliness, ensures that critical data can be transmitted preferentially when network resources are limited, and effectively solves the problem that traditional methods ignore data characteristics. The hierarchical caching mechanism is adopted, and storage areas are allocated according to data priorities, so that the storage resources are reasonably utilized, the risk of loss of critical data and waste of storage space is avoided, and the efficiency of data management is improved. Through real-time monitoring of the network bandwidth state and dynamic resuming adjustment of the priority data group, adaptive optimization of the transmission process is realized, the transmission efficiency after network recovery is improved, and the problem of lack of flexibility of traditional resuming methods is solved. The integrity of the marked data group is checked, and a transmission status report is generated in combination with the resuming report, so that a comprehensive data transmission monitoring mechanism is established, and the reliability and traceability of the transmission process are effectively improved. Through intelligent resuming and dynamic adjustment strategies, the system can better adapt to different network environments and data transmission requirements, and the overall transmission efficiency and system reliability are improved. The application comprehensively considers data characteristics, network status and storage resources, realizes intelligent data management and transmission in a network interruption environment, and provides a more reliable and efficient solution for data transmission in various application scenarios.

[0220] It should be noted that, for the convenience and brevity of description, the specific working processes of the system and each module described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0221] The above only describes the preferred embodiments of the application, and does not limit the patent scope of the application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields based on the content of the specification and drawings, is also included in the patent protection scope of the application.

Claims

1. A method for hierarchical data caching and intelligent resume transmission in a network outage environment, characterized in that, include: The data to be transmitted is acquired, and a hierarchical evaluation is performed based on the importance and timeliness of the data to obtain the corresponding priority data groups; Real-time monitoring of network bandwidth status; when a network interruption is detected, the interruption time is recorded and a tiered caching mechanism is triggered, and the corresponding storage area is set. The priority data group is allocated to the corresponding storage area for caching to obtain the final cache area; When network recovery is detected, the priority data group is intelligently resumed according to the cache area, and a resume report is generated. The network bandwidth status is detected in real time, and the priority data group is dynamically adjusted for continued transmission. The priority data group that has been resumed is marked to obtain the marked data group. The integrity of the marked data group is verified, and a transmission status report is generated in conjunction with the resume report; The step of allocating the priority data group to the corresponding storage area for caching to obtain the final cache area includes: The storage requirements of the priority data group are calculated to obtain the corresponding storage requirements; The storage area is partitioned according to the storage demand to obtain multiple corresponding storage partitions; The access speed of the storage partition was tested to obtain the access speed value; The storage partitions are sorted according to the access speed values ​​to obtain a speed level sequence; The priority data group is matched with the speed level sequence to obtain the priority storage partition correspondence; The priority data group is written into the corresponding storage partition according to the priority storage partition correspondence to obtain the initial cache area; The storage space utilization rate of the initial cache area is calculated to obtain the utilization rate of each storage partition. The initial cache region is reallocated according to the preset cyclic coverage strategy and the usage rate to obtain an optimized cache region; The optimized cache area is monitored in real time. When the storage capacity of the optimized cache area reaches a preset capacity threshold, the optimized cache area is subjected to data compression encoding to obtain compressed cache data. The compressed cache data is rewritten into the optimized cache area to obtain the final cache area; The calculation of storage space utilization for the initial cache region to obtain the utilization rate of each storage partition includes: The initial buffer region is divided into multi-scale spatial partitions to obtain scale grids of different scales; Occupancy calculations are performed for each scale grid to obtain the number of occupied grids for each scale; The space complexity index is obtained by calculating the fractal dimension of the occupied grid number using a preset box counting method. The initial cache region is divided into fixed-size storage blocks, and the data distribution probability is calculated for each storage block to obtain the block-level data distribution probability. The entropy of the block-level data distribution probability is calculated using a preset entropy formula to obtain the data distribution entropy. The storage partition is non-linearly mapped based on the space complexity index to obtain the preliminary utilization rate; The preliminary utilization rate is corrected based on the data distribution entropy to obtain the corrected utilization rate; Access frequency statistics are performed on the storage partition to obtain an access heat map; The usage rate of each storage partition is obtained by comprehensively statistically analyzing the modified usage rate based on the access heat map. When network recovery is detected, the priority data group is intelligently resumed based on the cache area, and a resumption report is generated, including: The network bandwidth status is periodically checked, and when network recovery is detected, an intelligent resume transmission program is triggered. Read the storage location information of the priority data group in the cache area, sort the reads, and generate a resume queue; The data packet numbering and fragmentation process is performed on the resume queue to obtain a set of data fragments to be transmitted; The optimal number of concurrent transmissions and the size of a single fragment are calculated based on the data fragment set to obtain the transmission control parameters; The data fragment set is transmitted concurrently based on the transmission control parameters, and the transmission status of each data fragment is recorded in real time to obtain the initial resume transmission information. When a data fragment transmission failure is detected, the data fragment is added back to the resume queue, and the number of retries is recorded. The initial resume information is combined with the number of retries to obtain the resume report.

2. The data hierarchical caching and intelligent resume method under network outage conditions according to claim 1, characterized in that, The process of acquiring data to be transmitted involves a tiered evaluation based on data importance and timeliness to obtain corresponding priority data groups, including: The attributes of the data to be transmitted are extracted to obtain the data creation timestamp and data service type identifier; The data to be transmitted is classified and encoded according to a preset business classification rule base and the data service type identifier to obtain an identifier data record; Obtain the current timestamp and calculate the difference between it and the timestamp created by the data to obtain the duration of data existence. Calculate the weight of the duration of data existence based on a preset timeliness benchmark threshold to obtain a timeliness weight coefficient. The identification data records and the timeliness weight coefficient are scored in a standardized manner to obtain a comprehensive importance score; The comprehensive importance score is prioritized and grouped according to a preset importance score range threshold group to obtain the priority data group, which includes a high priority group, a medium priority group, and a low priority group.

3. The data hierarchical caching and intelligent resume method under network outage conditions according to claim 2, characterized in that, The step of standardizing and scoring the identification data record and the timeliness weight coefficient to obtain a comprehensive importance score also includes: The importance of the identified data records is quantified to obtain a data service importance score; The timeliness weight coefficient is used to calculate the timeliness score. Based on the preset environmental factor assessment rules, a multi-dimensional analysis of the current system environment is performed to obtain an environmental factor score. The data service importance score, the timeliness score, and the environmental factor score are weighted and fused to obtain an initial comprehensive score; The initial comprehensive score is standardized and mapped to an interval between 0 and 1 to obtain the initial mapping value; Select preset mapping parameters and perform iterative mapping calculations on the initial mapping value to obtain a dynamic adjustment factor; The importance score is obtained by calculating the importance based on the initial comprehensive score and the dynamic adjustment factor.

4. The data hierarchical caching and intelligent resume method under network outage conditions according to claim 1, characterized in that, The real-time monitoring of network bandwidth status, when a network interruption is detected, records the interruption time and triggers a tiered caching mechanism, and sets up a corresponding storage area, including: The network bandwidth status is periodically detected to obtain network status parameters; Based on the network state parameters, network quality is calculated to obtain a network quality score; The network quality score is compared with a threshold to determine whether the network is interrupted; When a network interruption is detected, record the interruption time and timing data, and calculate the duration of the network interruption; The duration of the network interruption is classified based on a preset duration threshold to obtain the interruption level; The interruption level is matched with a preset cache policy library to obtain the corresponding hierarchical cache policy; The cache management module is initialized based on the tiered caching strategy, and corresponding data access and storage rules are set. Establish a cache index structure based on the data access and storage rules; The corresponding storage area is set according to the cache index structure to complete the triggering and initialization of the hierarchical caching mechanism.

5. The data hierarchical caching and intelligent resume transmission method under network outage conditions according to claim 1, characterized in that, The process involves real-time detection of network bandwidth status, dynamic adjustment of priority data groups for resume transmission, and marking of priority data groups that have completed resume transmission, resulting in marked data groups, including: The network bandwidth status is periodically sampled and measured to obtain bandwidth sampling data; Based on the bandwidth sampling data, network bandwidth characteristics are calculated to obtain bandwidth statistical characteristics; The current transmission adjustment strategy is obtained by matching the bandwidth statistical characteristics with the preset adjustment strategy. Based on the current transmission adjustment strategy, the transmission rate and concurrency of the priority data group are adjusted to obtain the adjusted transmission parameters; The priority data group is divided into blocks for transmission according to the adjusted transmission parameters to obtain transmission data blocks; Perform a checksum calculation on each of the transmitted data blocks to obtain data block checksum information; The data block verification information and the transmitted data block are sent to the receiving end to obtain the transmission result; Receive the transmission confirmation information returned by the receiving end to obtain the transmission status; The priority data group that has been successfully transmitted is marked according to the transmission status to obtain the marked data group; The marked data group is removed from the priority data group to obtain the updated priority data group; The transmission process is repeated for the updated priority data group until the transmission is complete.

6. The data hierarchical caching and intelligent resume method under network outage conditions according to claim 1, characterized in that, The step of performing integrity verification on the marked data group and generating a transmission status report in conjunction with the resume report includes: For each of the marked data groups, a checksum is calculated to obtain a checksum value, which is then compared with a preset standard value to obtain an integrity check result. The integrity verification results are analyzed for success rate to obtain the corresponding overall transmission status; Based on the overall transmission status, the marked data group is marked as having failed verification to obtain a data group to be retransmitted. The data group to be retransmitted is retransmitted based on a preset maximum number of retransmissions to obtain the retransmission result. Based on the retransmission results, identify the data group to be retransmitted that exceeds the upper limit of the number of retransmissions, obtain the failed data group, process the failed data group according to the preset degradation strategy, and update the overall transmission status to obtain the corresponding updated transmission status. Based on the updated transmission status, transmission statistics are performed on the marked data group to obtain the corresponding transmission statistics information; The transmission statistics are categorized and summarized to obtain the corresponding transmission anomaly information; The transmission status report is obtained by integrating the resume report, the transmission statistics, and the transmission anomaly information.

7. A data hierarchical caching and intelligent resume transmission device for offline environments, characterized in that, The data hierarchical caching and intelligent resume method applied to any one of claims 1-6 in a network outage environment includes: The acquisition module is used to acquire the data to be transmitted, and to perform hierarchical evaluation based on the importance and timeliness of the data to obtain corresponding priority data groups; The analysis module is used to monitor the network bandwidth status in real time. When a network interruption is detected, it records the interruption time and triggers a hierarchical caching mechanism, and sets the corresponding storage area. An association module is used to allocate the priority data group to the corresponding storage area for caching, thereby obtaining the final cache area; The processing module is used to intelligently resume the transmission of the priority data group according to the cache area and generate a resumption report when the network recovery is detected. The control module is used to detect the network bandwidth status in real time, dynamically adjust the priority data group for continued transmission, and mark the priority data group that has completed the continued transmission to obtain the marked data group. An execution module is used to perform integrity verification on the marked data group and generate a transmission status report in conjunction with the resume report; The step of allocating the priority data group to the corresponding storage area for caching to obtain the final cache area includes: The storage requirements of the priority data group are calculated to obtain the corresponding storage requirements; The storage area is partitioned according to the storage demand to obtain multiple corresponding storage partitions; The access speed of the storage partition was tested to obtain the access speed value; The storage partitions are sorted according to the access speed values ​​to obtain a speed level sequence; The priority data group is matched with the speed level sequence to obtain the priority storage partition correspondence; The priority data group is written into the corresponding storage partition according to the priority storage partition correspondence to obtain the initial cache area; The storage space utilization rate of the initial cache area is calculated to obtain the utilization rate of each storage partition. The initial cache region is reallocated according to the preset cyclic coverage strategy and the usage rate to obtain an optimized cache region; The optimized cache area is monitored in real time. When the storage capacity of the optimized cache area reaches a preset capacity threshold, the optimized cache area is subjected to data compression encoding to obtain compressed cache data. The compressed cache data is rewritten into the optimized cache area to obtain the final cache area; The calculation of storage space utilization for the initial cache region to obtain the utilization rate of each storage partition includes: The initial buffer region is divided into multi-scale spatial partitions to obtain scale grids of different scales; Occupancy calculations are performed for each scale grid to obtain the number of occupied grids for each scale; The space complexity index is obtained by calculating the fractal dimension of the occupied grid number using a preset box counting method. The initial cache region is divided into fixed-size storage blocks, and the data distribution probability is calculated for each storage block to obtain the block-level data distribution probability. The entropy of the block-level data distribution probability is calculated using a preset entropy formula to obtain the data distribution entropy. The storage partition is non-linearly mapped based on the space complexity index to obtain the preliminary utilization rate; The preliminary utilization rate is corrected based on the data distribution entropy to obtain the corrected utilization rate; Access frequency statistics are performed on the storage partition to obtain an access heat map; The usage rate of each storage partition is obtained by comprehensively statistically analyzing the modified usage rate based on the access heat map. When network recovery is detected, the priority data group is intelligently resumed based on the cache area, and a resumption report is generated, including: The network bandwidth status is periodically checked, and when network recovery is detected, an intelligent resume transmission program is triggered. Read the storage location information of the priority data group in the cache area, sort the reads, and generate a resume queue; The data packet numbering and fragmentation process is performed on the resume queue to obtain a set of data fragments to be transmitted; The optimal number of concurrent transmissions and the size of a single fragment are calculated based on the data fragment set to obtain the transmission control parameters; The data fragment set is transmitted concurrently based on the transmission control parameters, and the transmission status of each data fragment is recorded in real time to obtain the initial resume transmission information. When a data fragment transmission failure is detected, the data fragment is added back to the resume queue, and the number of retries is recorded. The initial resume information is combined with the number of retries to obtain the resume report.

Images