Intelligent storage management method and device for telecommunication network data and storage medium

Abstract

The application relates to the technical field of big data storage, in particular to an intelligent storage management method and device for telecommunication network data and a storage medium, the method comprising the following steps: in response to regional business flow data meeting preset abnormal event conditions, determining the reference heat value of each grid unit in the region; based on the reference heat value of the grid unit and the actual accessed network data volume, constructing a comprehensive heat index of the grid unit and performing classification; determining the data traceability value of the network data based on the comprehensive deviation and a topological influence factor; based on the comprehensive heat index of the grid unit and the business priority weight, performing dynamic hierarchical storage on the network data of the grid unit; screening the network data of the grid unit in the hot storage layer and the warm storage layer, determining an object to be evaluated, transferring the network data of the grid unit based on the object to be evaluated; and executing a cleaning strategy based on the capacity of the cold storage layer. The application effectively improves the accuracy of data hotness determination.

Description

Technical Field

[0001] This invention relates to the field of big data storage technology, and in particular to an intelligent storage management method, apparatus and storage medium for telecommunications network data. Background Technology

[0002] With the large-scale deployment of 5G mobile communication technology and the continued explosive growth in the number of IoT terminals, telecommunications networks are facing an unprecedented surge in data volume. Under the dual pressure of exponential data growth and increasingly longer retention periods, traditional storage management methods are facing increasingly severe challenges. In traditional telecommunications network data storage architectures, various types of network data are often stored indiscriminately on high-performance, high-cost storage media. While this "one-size-fits-all" storage strategy ensures convenient data access, it brings significant cost and efficiency problems. A large amount of historical data is rarely accessed again after its generation, yet it continues to occupy expensive high-performance storage resources, resulting in a serious waste of storage costs.

[0003] However, existing tiered storage solutions still have significant shortcomings in the specific scenario of telecommunications network data. First, most solutions rely on a single dimension for evaluating data "hotness," primarily depending on basic indicators such as access frequency, failing to fully consider the priority characteristics of telecommunications services themselves (such as the differentiated weights of different service types like VoLTE voice and video streaming in the 3GPP standard) and the decay law of data activity over time. Second, existing solutions lack a refined dynamic evaluation and migration mechanism for the transition layer management between hot and cold data, often employing static thresholds or fixed time window strategies.

[0004] Therefore, there is an urgent need for an intelligent storage management method tailored to the characteristics of telecommunications network data, which can achieve precise, dynamic, and compliant hierarchical management and storage of network data among hot, warm, and cold storage layers. This would minimize storage costs and improve the timeliness and accuracy of data hot / cold determination while ensuring data availability and compliance. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent storage management method, device, and storage medium for telecommunications network data, so as to solve the problem that the threshold in the prior art cannot be automatically adjusted with the periodic fluctuations of the service load.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for intelligent storage management of telecommunications network data, comprising:

[0008] In response to regional business traffic data meeting preset abnormal event conditions, network data within a preset time window is collected synchronously, and the baseline heat value of each grid unit in the region is determined.

[0009] Based on the baseline popularity value of the grid cell and the actual amount of network data accessed, a comprehensive popularity index of the grid cell is constructed and the network data of the grid cell is classified.

[0010] The data traceability value score is determined based on the comprehensive deviation degree and topological influence factor, thereby determining the data traceability value;

[0011] Based on the comprehensive heat index and business priority weight of the grid cell, the network data of the grid cell is dynamically and hierarchically stored, and the data activity decay coefficient is constructed based on the proportion of each business type in the network data of the grid cell and the scale parameter of the business type.

[0012] The network data of the grid cells in the hot storage layer and the warm storage layer are filtered to determine the objects to be evaluated, and the network data of the grid cells are transferred based on the objects to be evaluated;

[0013] A cleanup strategy is executed based on the remaining capacity of the cold storage layer.

[0014] Optionally, the regional service traffic data includes the region's downlink physical resource block utilization and the number of users connected to the radio resource control system.

[0015] When any region is detected to meet the triggering conditions for a business traffic anomaly event, a synchronous data collection is triggered to collect network data within a preset time window at that moment.

[0016] The synchronously collected network data includes:

[0017] Within the time window, the network configuration management interface, performance management interface, and signaling tracing interface are invoked to collect network data of the area and neighboring areas with the same coverage, and each collection record is attached with a unified timestamp and area identifier.

[0018] The process for determining the baseline heat value of each grid cell within the region is as follows:

[0019] For each grid cell, the historical query records and data volume falling into it are counted, where the data volume is the amount of network data actually read in one historical query record;

[0020] The weight of the historical query records is negatively correlated with the number of days since the current time;

[0021] The baseline heat value of the grid cell is obtained by summing the weights and data volumes of historical query records.

[0022] Optionally, the reference heat value of the grid cell with the largest reference heat value among all grid cells is used as the reference reference heat value, and the ratio of the reference heat value of the grid cell to the reference reference heat value is used as the reference heat value factor.

[0023] Collect the actual network data volume accessed within one hour after the generation of each grid cell, take the largest actual network data volume among all the actual network data volumes as the reference network data volume, and take the ratio of the actual network data volume to the reference network data volume as the actual access factor.

[0024] The comprehensive popularity index of a grid cell is determined by weighting and fusing the baseline popularity value of the grid cell with the actual access record factor.

[0025] The process of constructing the hot threshold and cold threshold is as follows:

[0026] The heat threshold and cold threshold are determined based on the comprehensive heat index of the grid cells.

[0027] Sort the grid cells by their overall heat index over the past month in descending order;

[0028] The heat threshold is defined as the P80 quantile of the comprehensive heat index of the grid cell over the past month.

[0029] The cold threshold is defined as the P20 quantile of the comprehensive heat index of the grid cell over the past month.

[0030] If the overall heat index of a grid cell is greater than the heat threshold, the network data of that grid cell is marked as hot data and stored in the hot storage layer.

[0031] If the comprehensive heat index of a grid cell is less than or equal to the heat threshold and greater than or equal to the cold threshold, then the network data of that grid cell is marked as warm data and stored in the warm storage layer.

[0032] If the overall heat index of a grid cell is less than the cold threshold, the network data of that grid cell is marked as cold data and stored in the cold storage layer.

[0033] Optionally, the process of constructing the data traceability value score is as follows:

[0034] The peak value of downlink physical resource block utilization and the maximum value of the number of wireless resource control connected users are collected within a preset time window in the grid cell.

[0035] The deviation of the downlink physical resource block utilization rate is determined based on the peak value of the downlink physical resource block utilization rate, and the deviation of the downlink physical resource block utilization rate is positively correlated with the peak value of the downlink physical resource block utilization rate.

[0036] The deviation of the number of wireless resource control (RRR) connected users is determined based on the maximum value of the number of RRR connected users, and the deviation of the number of RRR connected users is positively correlated with the maximum value of the number of RRR connected users.

[0037] The overall deviation is determined by weighted fusion of the deviation of downlink physical resource block utilization and the deviation of the number of connected users under radio resource control.

[0038] Based on the network elements in the grid cells of the region that begin to meet the triggering conditions of the anomaly event, the number of network elements connected to the network element in the network element topology snapshot and the total number of network elements in the grid cell are counted from the network element topology snapshot; the number of network elements is normalized to obtain the topology influence factor, which is positively correlated with the number of connected network elements;

[0039] The data traceability value score is determined by weighted fusion of comprehensive deviation and topological influence factor.

[0040] The process for determining the traceability value of the data is as follows:

[0041] If the data traceability value score is greater than or equal to the traceability threshold, the network data of the grid cell is marked as having high traceability value.

[0042] If the data traceability value score is less than the traceability threshold, the network data of the grid cell will be marked as having low traceability value.

[0043] Optionally, the process for constructing the business priority weights is as follows:

[0044] According to the 3GPP standard, 5QI values ​​are defined for 5G QoS flows of different service types;

[0045] Business priority weights are determined based on the 5QI values ​​of different business types.

[0046] If the 5QI value of a business type is greater than the first preset threshold, the business priority weight is 2;

[0047] If the 5QI value of a business type is greater than or equal to the second preset threshold and less than or equal to the first preset threshold, then the business priority weight is 1.

[0048] If the 5QI value of a business type is less than the second preset threshold, the business priority weight is 0.5.

[0049] If there are multiple service types in the network data, the average weight is taken as the service priority weight; the average weight is the ratio of the service priority weight of the multiple service types to the number of service types.

[0050] The process for constructing the data activity decay coefficient is as follows:

[0051] Based on the time interval between the generation and last access of network data in the grid cell, the time interval is divided according to the service type, and then the Weibull distribution is used to perform curve fitting to obtain the activity decay curve and extract the scale parameter.

[0052] For the network data of the collected grid cells, the data activity decay coefficient is obtained by summing the proportion of each service type and the scale parameter of the service type within the network data of the grid cells.

[0053] Optionally, the decision-making process for the dynamic hierarchical storage is as follows:

[0054] When the overall heat index of a grid cell is higher than the heat threshold, or the business priority weight is greater than or equal to the first weight, the network data of the grid cell will be allocated to the hot storage layer.

[0055] When the overall heat index of a grid cell is lower than the cold threshold, the priority weight is less than or equal to the second weight, and the data activity decay coefficient is less than the preset decay threshold, the network data of the grid cell will be allocated to the cold storage layer.

[0056] For network data allocated to grid cells in the warm storage layer, an expiration migration timestamp is additionally written when allocated to the warm storage layer. The migration timestamp is the sum of the product of the current time and the data activity decay coefficient and the expansion factor. After the migration timestamp is reached, the network data will be re-evaluated.

[0057] If the network data of a grid cell is marked as having high traceability value, it will be retained in the original storage layer even if the overall popularity index of the grid cell is lower than the cold threshold; if the network data of a grid cell is marked as having low traceability value, it will be processed normally.

[0058] Optionally, the comprehensive heat index is updated hourly for the network data of the grid cells that have been stored in the hot storage layer and the warm storage layer.

[0059] If the comprehensive heat index of the grid cell in the hot storage layer is lower than the heat threshold within a continuous four-hour update cycle, it is marked as an object to be evaluated.

[0060] If the overall heat index of the grid cell in the warm storage layer is lower than the cold threshold within a continuous four-hour update cycle, it is marked as an object to be evaluated.

[0061] For all grid cells marked as objects to be evaluated, the network data is analyzed based on the network data's survival time and temporal locality index; the survival time is the difference between the current evaluation time and the network data's entry time. Using the Weibull survival function, the conditional probability of the data continuing to survive beyond the prediction window, given the survival time, is calculated.

[0062] The average ratio of the number of times each user is accessed in the first hour to the total number of records within the network data of that grid cell is used as an indicator of temporal locality.

[0063] The query probability is determined by a weighted fusion of temporal locality indicators and conditional probability.

[0064] The transfer of network data is determined by comparing the query probability with a preset minimum probability threshold;

[0065] If the query probability is less than the preset minimum probability threshold, the network data will be transferred to the cold storage layer.

[0066] If the query probability is greater than or equal to the preset minimum probability threshold, the network data will be retained in the original storage layer.

[0067] From the collected network element topology snapshots, the topological distance between each network element in the service process is obtained. The topological distance between network elements is the number of hops on the logical interface path between two network elements. The network elements are arranged from smallest to largest topological distance, and the network data is stored from nearest to farthest on the cold storage layer according to the topological distance between network elements from smallest to largest.

[0068] Optionally, the current usage capacity of the cold storage layer and the average daily growth rate over the past month are statistically analyzed to obtain the predicted remaining capacity of the cold storage layer; the telecommunications service compliance requirement configuration table is read, which defines the minimum retention period for network data of different service types;

[0069] If the predicted remaining capacity of the cold storage layer is lower than the minimum capacity, a cleanup strategy will be executed.

[0070] The process of constructing the cleanup strategy is as follows:

[0071] Traverse the network data of the grid cells in the cold storage layer. If the network data of the grid cell was generated earlier than the current date minus the statutory minimum retention period for the corresponding business type, and the number of queries in the last six months is zero, then mark it as clearable and generate a list of candidates to be cleared.

[0072] If the network data of the grid cell in the candidate list to be removed is of high traceability value, it will be removed from the candidate list to be removed and an observation window will be reset. The traceability value of the data will be re-evaluated after the observation window expires.

[0073] Before cleaning up the candidate list, manual approval is required, and an unchangeable audit log must be generated. The audit log includes information about the network data to be cleaned and the person performing the cleanup operation.

[0074] According to another aspect of this application, an intelligent storage management device for telecommunications network data is provided, comprising:

[0075] The baseline heat determination unit is used to respond to the regional business traffic data meeting the preset abnormal event conditions, synchronously collect network data within a preset time window, and determine the baseline heat value of each grid unit in the region.

[0076] The data classification unit is used to construct a comprehensive popularity index for the grid unit and classify the network data of the grid unit based on the baseline popularity value and the actual amount of network data accessed by the grid unit.

[0077] The source tracing scoring unit is used to determine the source tracing value score of network data based on the comprehensive deviation degree and topological influence factor, thereby determining the source tracing value of the data;

[0078] The dynamic hierarchical storage unit is used to dynamically hierarchically store the network data of the grid unit based on the comprehensive heat index and business priority weight of the grid unit. The data activity decay coefficient is constructed based on the proportion of each business type in the network data of the grid unit and the scale parameter of the business type.

[0079] The evaluation unit is used to filter the network data of the grid cells in the hot storage layer and the warm storage layer, determine the objects to be evaluated, and transfer the network data of the grid cells based on the objects to be evaluated.

[0080] The execution unit is used to execute cleanup strategies based on the remaining capacity of the cold storage layer.

[0081] According to one aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, wherein the computer program is used to control the electronic device in which the computer-readable storage medium is located to perform the intelligent storage management method for telecommunications network data during runtime.

[0082] The beneficial effects of this invention are as follows:

[0083] By introducing dynamic quantile thresholds for hot and cold data classification, the classification criteria can be automatically adjusted according to the overall load and data value distribution, avoiding misjudgments caused by fixed thresholds during business fluctuations. High-value business data is guaranteed mandatory residency in the hot layer, while low-value, rapidly decaying data is quickly cooled, achieving a precise match between storage resources and business value. By reading the telecommunications business compliance requirement configuration table, a statutory minimum retention period is set for different types of data, and future storage capacity is predicted based on the data activity decay coefficient. When insufficient available space in the cold storage layer is predicted, low-value data that has exceeded its retention period and has not been accessed for a long time can be prioritized for cleanup, while ensuring compliance. Each cleanup requires manual approval and generates an immutable audit log, achieving a balance between storage capacity management, business traceability needs, and regulatory compliance. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0085] Figure 1 This is a flowchart illustrating the intelligent storage management method for telecommunications network data in this embodiment.

[0086] Figure 2 This is a schematic diagram of the intelligent storage management device for telecommunications network data in this embodiment.

[0087] Figure 3 A schematic diagram of the structure of a computer system for implementing the apparatus provided in the embodiments of this disclosure. Detailed Implementation

[0088] To more clearly illustrate the present invention, the following description, in conjunction with preferred embodiments and accompanying drawings, further explains the invention. Similar components in the drawings are indicated by the same reference numerals. Those skilled in the art should understand that the specific description below is illustrative rather than restrictive and should not be construed as limiting the scope of protection of the present invention.

[0089] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0090] Specifically, the method described in this embodiment is applied to the scenario of a telecom-grade full-domain operation and maintenance data lake in which telecom operators face the contradiction between massive data growth and storage costs, and which carries multiple generations of network services and whose data value dynamically decays with the life cycle. Existing technologies lack the ability to perceive the semantics of data services and predict future access probabilities, and rely solely on static rules for indiscriminate storage stratification and elimination. This results in high-value traceability data being archived prematurely and low-value data occupying high-performance resources for a long time. This method avoids the waste of resources caused by storing all historical data in high-performance storage.

[0091] Applied to the above application scenarios, this embodiment provides an intelligent storage management method for telecommunications network data, see below. Figure 1 As shown, the method includes:

[0092] Step S1: In response to the regional service traffic data meeting the preset abnormal event conditions, network data within the preset time window is collected synchronously, and the baseline heat value of each grid cell in the region is determined. The regional service traffic data includes the downlink physical resource block utilization rate and the number of wireless resource control connected users in the region. The region is a minimum wireless signal coverage area with a clear boundary and independent identification in the mobile communication network.

[0093] In this embodiment of the invention, the preset abnormal event condition is defined as follows: within three consecutive statistical periods with a granularity of 15 minutes, the downlink physical resource block utilization rate of the area exceeds 70%, and the number of wireless resource control connection users in the area exceeds 80% of the rated licensed users in the area; the rated licensed users are the maximum number of wireless resource control connection users in the area, which is directly read from the network configuration management system (OMC / CM) of the minimum wireless signal coverage area;

[0094] When any region is detected to meet the triggering conditions for a business traffic anomaly event, a synchronous data collection is triggered to collect network data within a preset time window at that moment. The preset time window is a 30-minute time window that is based on the triggering moment, going back 15 minutes and extending forward 15 minutes.

[0095] In this embodiment of the invention, the synchronous acquisition of network data includes:

[0096] Within the time window, the network configuration management interface, performance management interface, and signaling tracing interface are invoked to collect network data of the region and its neighboring areas with the same coverage. The neighboring areas with the same coverage are areas covered by the same wireless signal as the region. The network data includes at least: user control plane signaling records, user plane service details, and network element topology snapshots obtained from the operation and maintenance center. Each collection record is accompanied by a unified timestamp and area identifier.

[0097] In this embodiment of the invention, the construction process of the mesh cell is as follows:

[0098] The area is divided into multiple non-overlapping grid cells using a fixed 50m x 50m grid, and each grid cell has a unique identifier.

[0099] The process for determining the baseline heat value of each grid cell within the region is as follows:

[0100] For each grid cell, the historical query records and data volume falling into it are counted, where the data volume is the amount of network data actually read in one historical query record;

[0101] Furthermore, the weight of the historical query records is determined as: m = β × (1 - β) t-1 Where β is the initial weight of the historical query records, and t is the number of days from the current time;

[0102] The baseline heat value of a grid cell is obtained based on the weight and data volume of historical query records. The expression for the baseline heat value of the grid cell is: Q = Σ(q=1, e)m×r q Where e represents the total number of historical query records within the grid cell, and r q Let q be the amount of data recorded in the q-th historical query.

[0103] Preferably, in this embodiment, the initial weight of historical query records is 0.4, so that the calculation of the baseline popularity value is mainly focused on the query behavior in the past 1-2 weeks, which can reflect this periodic pattern.

[0104] Step S2: Based on the baseline heat value of the grid cell and the actual amount of network data accessed, construct a comprehensive heat index for the grid cell and classify the network data of the grid cell.

[0105] Specifically, the reference heat value of the grid cell with the largest reference heat value among all grid cells is taken as the reference reference heat value, and the ratio of the reference heat value of the grid cell to the reference reference heat value is taken as the reference heat value factor H1.

[0106] Collect the actual network data volume accessed within one hour after the generation of each grid cell, take the largest actual network data volume among all the actual network data volumes as the reference network data volume, and take the ratio of the actual network data volume to the reference network data volume as the actual access factor H2.

[0107] Furthermore, a comprehensive heat index for the grid cell is constructed based on the baseline heat value and the actual access record factor. The expression for the comprehensive heat index of the grid cell is: H = h × H1 + (1 - h) × H2; where h is the balance coefficient.

[0108] Preferably, in this embodiment, the balance coefficient is 0.5. A balance coefficient of 0.5 means that the comprehensive heat index of the grid cell is determined by the base heat value of the grid cell and the actual access record factor, each accounting for half, so as to avoid one of them being too high and affecting the determination of the comprehensive heat index of the grid cell.

[0109] The process of constructing the hot threshold and cold threshold is as follows:

[0110] The heat threshold and cold threshold are determined based on the comprehensive heat index of the grid cells.

[0111] Sort the grid cells by their overall heat index over the past month in descending order;

[0112] The heat threshold is defined as the P80 quantile of the comprehensive heat index of the grid cell over the past month.

[0113] The cold threshold is defined as the P20 quantile of the comprehensive heat index of the grid cell over the past month.

[0114] If the overall heat index of a grid cell is greater than the heat threshold, the network data of that grid cell is marked as hot data and stored in the hot storage layer.

[0115] If the comprehensive heat index of a grid cell is less than or equal to the heat threshold and greater than or equal to the cold threshold, then the network data of that grid cell is marked as warm data and stored in the warm storage layer.

[0116] If the overall heat index of a grid cell is less than the cold threshold, the network data of that grid cell is marked as cold data and stored in the cold storage layer.

[0117] Step S3: Determine the data traceability value score of network data based on comprehensive deviation and topological influence factor, and then determine the data traceability value.

[0118] The process of constructing the data traceability value score is as follows:

[0119] The peak value x of downlink physical resource block utilization and the maximum value c of the number of wireless resource control connected users are collected within a preset time window in the grid cell.

[0120] The deviation X of the downlink physical resource block utilization rate is determined based on the peak value of the downlink physical resource block utilization rate. The expression for the deviation of the downlink physical resource block utilization rate is: X = (x - X1) / X1; where X1 is the baseline value of the downlink physical resource block utilization rate.

[0121] The deviation C of the number of wireless resource control (RRR) connected users is determined based on the maximum value of the number of RRR connected users. The expression for the deviation of the number of RRR connected users is: C = (c - Cmax) / Cmax; where Cmax is the rated number of licensed users.

[0122] Preferably, in this embodiment, the baseline value of downlink physical resource block utilization is 70%. This 70% is the abnormal starting line of the entire scheme. The data is collected precisely because its downlink physical resource block utilization has crossed this line, and 70% is used as the baseline value for deviation calculation.

[0123] The overall deviation is determined based on the deviation of downlink physical resource block utilization and the deviation of the number of wireless resource control connected users: The expression for the overall deviation is: N=n1×X+n2×C; where n1 is the deviation weight of downlink physical resource block utilization, n2 is the deviation weight of the number of wireless resource control connected users, and n1+n2=1.

[0124] Preferably, in this embodiment, the deviation weight of downlink physical resource block utilization is 0.6, and the deviation weight of the number of connected users under radio resource control is 0.4. The fundamental purpose of the network is to provide high-quality resources for users. The scarcity of resources is a more direct reflection of service quality than the number of connections. The deviation weight of downlink physical resource block utilization being 0.6 and the deviation weight of the number of connected users under radio resource control being 0.4 are in line with this industry practice.

[0125] Furthermore, based on the network elements corresponding to the grid cells in the region that begin to meet the triggering conditions of the anomaly event, the number of network elements u connected to the network element in the network element topology snapshot and the total number of network elements U1 in the grid cell are counted from the network element topology snapshot; the number of network elements is normalized to obtain the topology influence factor, and the expression of the topology influence factor is: U=u / (U1-1).

[0126] The data traceability value score is determined based on the comprehensive deviation and the topological influence factor. The expression for the data traceability value score G is: G=g1×N+g2×U; where g1 is the comprehensive deviation weight, g2 is the topological influence factor weight, and g1+g2=1.

[0127] Preferably, in this embodiment, the weight of the overall deviation is 0.69 and the weight of the topology influence factor is 0.31, so that the weight allocation of the entire data traceability value score presents a consistent tilt: performance first, topology auxiliary.

[0128] The process for determining the traceability value of the data is as follows:

[0129] If the data traceability value score is greater than or equal to the traceability threshold, the network data of the grid cell is marked as having high traceability value.

[0130] If the data traceability value score is less than the traceability threshold, the network data of the grid cell will be marked as having low traceability value.

[0131] Step S4: Based on the comprehensive heat index and business priority weight of the grid cell, the network data of the grid cell is dynamically and hierarchically stored, and the data activity decay coefficient is constructed based on the proportion of each business type in the network data of the grid cell and the scale parameter of the business type.

[0132] Specifically, the process of constructing the business priority weights is as follows:

[0133] According to the 3GPP standard, 5QI values ​​are defined for 5G QoS flows of different service types;

[0134] Business priority weights are determined based on the 5QI values ​​of different business types.

[0135] If the 5QI value of a business type is greater than the first preset threshold, the business priority weight is 2;

[0136] If the 5QI value of a business type is greater than or equal to the second preset threshold and less than or equal to the first preset threshold, then the business priority weight is 1.

[0137] If the 5QI value of a business type is less than the second preset threshold, the business priority weight is 0.5.

[0138] If there are multiple service types in the network data, the average weight is taken as the service priority weight; the average weight is the ratio of the service priority weight of the multiple service types to the number of service types.

[0139] The service types include, but are not limited to: VoLTE voice, video streaming, web browsing, and IoT short messages;

[0140] Specifically, the process for constructing the data activity decay coefficient is as follows:

[0141] Based on the time interval between the generation and last access of network data in the grid cell, the time interval is divided according to the service type, and then the Weibull distribution is used to perform curve fitting to obtain the activity decay curve, and the scale parameter η is extracted.

[0142] For the network data of the collected grid cells, the data activity decay coefficient λ is obtained according to the proportion w(i) of each service type in the network data of the grid cell and the scale parameter η(i) of the service type. The expression of the data activity decay coefficient is: λ=Σ(i=1,Y)w(i)×η(i), where i is the service type and Y is the number of service types.

[0143] The decision-making process for the dynamic hierarchical storage is as follows:

[0144] When the overall heat index of a grid cell is higher than the heat threshold, or the business priority weight is greater than or equal to the first weight, the network data of the grid cell will be allocated to the hot storage layer.

[0145] When the overall heat index of a grid cell is lower than the cold threshold, the priority weight is less than or equal to the second weight, and the data activity decay coefficient is less than the preset decay threshold, the network data of the grid cell will be allocated to the cold storage layer.

[0146] In other cases, no network data is processed;

[0147] Preferably, in this embodiment, the preset attenuation threshold is 72 hours. Setting it to 72 hours means that only data that has basically lost all access value within 72 hours will be allowed to skip the buffer period of the warm storage layer and directly enter the cold storage layer. This corresponds to the business characteristics of data such as IoT short messages, that is, if they are not accessed within 72 hours after being collected, the probability of them being accessed thereafter is close to zero.

[0148] For network data allocated to grid cells in the warm storage layer, an expiration migration timestamp is additionally written when the data is allocated to the warm storage layer. The migration timestamp is the sum of the product of the current time and the data activity decay coefficient and the expansion factor. After the migration timestamp is reached, the network data will be re-evaluated.

[0149] In this embodiment, if the network data of a grid cell is marked as having high traceability value, it will be retained in the original storage layer even if the overall heat index of the grid cell is lower than the cold threshold; if the network data of a grid cell is marked as having low traceability value, it will be processed normally.

[0150] Preferably, in this embodiment, the first weight is 1.5, the second weight is 0.6, and the expansion factor is 1.75, with values ​​ranging from approximately 1.5 to 2.0. This provides a moderate additional buffer period for most business data, preventing rapid cooling that could lead to potential tracing difficulties, and also preventing prolonged occupation of thermosphere resources.

[0151] Step S5: Filter the network data of the grid cells in the hot storage layer and the warm storage layer to determine the object to be evaluated, and transfer the network data of the grid cells based on the object to be evaluated.

[0152] For the network data of the grid cells that have been stored in the hot storage layer and the warm storage layer, the comprehensive heat index is updated on an hourly basis.

[0153] If the comprehensive heat index of the grid cell in the hot storage layer is lower than the heat threshold within a continuous four-hour update cycle, it is marked as an object to be evaluated.

[0154] If the overall heat index of the grid cell in the warm storage layer is lower than the cold threshold within a continuous four-hour update cycle, it is marked as an object to be evaluated.

[0155] For all grid cells marked as objects to be evaluated, the network data is analyzed based on the survival time and temporal locality index of the grid cell's network data; the survival time is the difference between the current evaluation time and the network data entry time; using the Weibull survival function, the conditional probability b1 of continuing to survive beyond the prediction window period is calculated, given the survival time.

[0156] Furthermore, the average ratio of the number of times each user is accessed within the network data of this grid cell in the first hour to the total number of records is used as the temporal locality index b2;

[0157] The query probability is determined based on the temporal locality index and conditional probability. The expression for the query probability is: B = b1 × (1 + b × b2), where b is the correction intensity coefficient.

[0158] Preferably, in this embodiment, the prediction window is 30 days, with the vast majority of occurrences within one month of data generation. Data that has not been accessed for more than 30 days has an extremely low probability of being actively traced and queried again, and the 30-day window can cover most of the effective tracing needs; the correction strength coefficient is 1, and the common value range of b is 0.5 to 2. b=0: completely ignores the temporal locality feature, and the query probability is determined only by the basic conditional probability based on the Weibull survival function. If the correction strength coefficient is too large, the correction is stronger, and its final query probability may be amplified, making it more difficult to migrate. If the correction strength coefficient is too large, the correction is weaker, and it is more dependent on the general decay law of the business type.

[0159] The transfer of network data is determined by comparing the query probability with a preset minimum probability threshold;

[0160] If the query probability is less than the preset minimum probability threshold, the network data will be transferred to the cold storage layer.

[0161] If the query probability is greater than or equal to the preset minimum probability threshold, the network data will be retained in the original storage layer.

[0162] Preferably, in this embodiment, the preset minimum probability threshold is 0.05. In hypothesis testing, 0.05 is widely accepted as the critical value for low-probability events. Applying this statistical thinking to data lifecycle management means that if the model predicts that the probability of data being accessed is less than 5%, it is considered that it is almost impossible for it to be queried again, and the traceability risk brought about by migrating it is acceptable.

[0163] Furthermore, from the collected network element topology snapshots, the topological distance between each network element in the service process is obtained. The topological distance between network elements is the number of hops on the logical interface path between two network elements. The topological distances between network elements are arranged from smallest to largest, and the network data is stored in the cold storage layer from nearest to farthest according to the topological distances between network elements.

[0164] Step S6: Execute a cleanup strategy based on the remaining capacity of the cold storage layer.

[0165] Specifically, the current usage capacity of the cold storage layer and the average daily growth rate over the past month are statistically analyzed to obtain the predicted remaining capacity of the cold storage layer; the telecommunications service compliance requirement configuration table is read, which defines the minimum retention period for network data of different service types;

[0166] If the predicted remaining capacity of the cold storage layer is lower than the minimum capacity, a cleanup strategy will be implemented; the minimum capacity is 15% of the total capacity.

[0167] The process of constructing the cleanup strategy is as follows:

[0168] Traverse the network data of the grid cells in the cold storage layer. If the network data of the grid cell was generated earlier than the current date minus the statutory minimum retention period for the corresponding business type, and the number of queries in the last six months is zero, then mark it as clearable and generate a list of candidates to be cleared.

[0169] If the network data of the grid cell in the candidate list to be removed is of high traceability value, it will be removed from the candidate list to be removed and an observation window will be reset. The traceability value of the data will be re-evaluated after the observation window expires.

[0170] Preferably, in this embodiment, the observation window is 90 days. Using 90 days as the observation window can fully cover the entire lifecycle from the occurrence of the fault to the accumulation of knowledge. If a batch of high traceability value data is not found by any query within the 90-day in-depth analysis window, it means that the event did not actually trigger an organizational-level review action, and its expected traceability value was not realized. In this case, it is reasonable to re-include it in the removal candidate list.

[0171] Furthermore, before cleaning up the candidate list, manual approval must be obtained, and an unchangeable audit log must be generated. The audit log includes information about the network data to be cleaned and the person performing the cleanup operation.

[0172] Please see Figure 2 As shown, this is a schematic diagram of the structure of the intelligent storage management device for telecommunications network data according to this embodiment, including,

[0173] The baseline heat determination unit is used to respond to the regional business traffic data meeting the preset abnormal event conditions, synchronously collect network data within a preset time window, and determine the baseline heat value of each grid unit in the region.

[0174] The data classification unit is used to construct a comprehensive popularity index for the grid unit and classify the network data of the grid unit based on the baseline popularity value and the actual amount of network data accessed by the grid unit.

[0175] The source tracing scoring unit is used to determine the source tracing value score of network data based on the comprehensive deviation degree and topological influence factor, thereby determining the source tracing value of the data;

[0176] The dynamic hierarchical storage unit is used to dynamically hierarchically store the network data of the grid unit based on the comprehensive heat index and business priority weight of the grid unit. The data activity decay coefficient is constructed based on the proportion of each business type in the network data of the grid unit and the scale parameter of the business type.

[0177] The evaluation unit is used to filter the network data of the grid cells in the hot storage layer and the warm storage layer, determine the objects to be evaluated, and transfer the network data of the grid cells based on the objects to be evaluated.

[0178] The execution unit is used to execute cleanup strategies based on the remaining capacity of the cold storage layer.

[0179] The intelligent storage management device for telecommunications network data provided in this application embodiment can execute the intelligent storage management method for telecommunications network data provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0180] like Figure 3As shown, a computer system suitable for implementing the intelligent storage management method for telecommunications network data provided in the above embodiments includes a central processing module (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer system. The CPU, ROM, and RAM are connected via a bus. An input / output (I / O) interface is also connected to the bus.

[0181] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including liquid crystal displays (LCDs) and speakers, etc.; storage sections including hard disks, etc.; and communication sections including network interface cards such as LAN cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0182] Specifically, according to this embodiment, the process described in the flowchart above can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program tangibly embodied on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0183] The flowcharts and schematic diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of the system, method, and computer program product of this embodiment. In this regard, each block in the flowchart or schematic diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the schematic diagram and / or flowchart, and combinations of blocks in the schematic diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0184] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is impossible to exhaustively list all the implementation methods here. All obvious variations or modifications derived from the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A method for intelligent storage management of data in telecommunications networks, characterized in that, include: In response to regional business traffic data meeting preset abnormal event conditions, network data within a preset time window is collected synchronously, and the baseline heat value of each grid unit in the region is determined. Based on the baseline popularity value of the grid cell and the actual amount of network data accessed, a comprehensive popularity index of the grid cell is constructed and the network data of the grid cell is classified. The data traceability value score is determined based on the comprehensive deviation degree and topological influence factor, thereby determining the data traceability value; Based on the comprehensive heat index and business priority weight of the grid cell, the network data of the grid cell is dynamically and hierarchically stored, and the data activity decay coefficient is constructed based on the proportion of each business type in the network data of the grid cell and the scale parameter of the business type. The network data of the grid cells in the hot storage layer and the warm storage layer are filtered to determine the objects to be evaluated, and the network data of the grid cells are transferred based on the objects to be evaluated; A cleanup strategy is executed based on the remaining capacity of the cold storage layer.

2. The intelligent storage management method for telecommunications network data according to claim 1, characterized in that, The regional service traffic data includes the region's downlink physical resource block utilization rate and the number of wireless resource control connection users. When any region is detected to meet the triggering conditions for a business traffic anomaly event, a synchronous data collection is triggered to collect network data within a preset time window at that moment. The synchronously collected network data includes: Within the time window, the network configuration management interface, performance management interface, and signaling tracing interface are invoked to collect network data of the area and neighboring areas with the same coverage, and each collection record is attached with a unified timestamp and area identifier. The process for determining the baseline heat value of each grid cell within the region is as follows: For each grid cell, the historical query records and data volume falling into it are counted, where the data volume is the amount of network data actually read in one historical query record; The weight of the historical query records is negatively correlated with the number of days since the current time; The baseline heat value of the grid cell is obtained by summing the weights and data volumes of historical query records.

3. The intelligent storage management method for telecommunications network data according to claim 2, characterized in that, The reference heat value of the grid cell with the largest reference heat value among all grid cells is taken as the reference reference heat value, and the ratio of the reference heat value of the grid cell to the reference reference heat value is taken as the reference heat value factor. Collect the actual network data volume accessed within one hour after the generation of each grid cell, take the largest actual network data volume among all the actual network data volumes as the reference network data volume, and take the ratio of the actual network data volume to the reference network data volume as the actual access factor. The comprehensive popularity index of a grid cell is determined by weighting and fusing the baseline popularity value of the grid cell with the actual access record factor. The process of constructing the hot threshold and cold threshold is as follows: The heat threshold and cold threshold are determined based on the comprehensive heat index of the grid cells. Sort the grid cells by their overall heat index over the past month in descending order; The heat threshold is defined as the P80 quantile of the comprehensive heat index of the grid cell over the past month. The cold threshold is defined as the P20 quantile of the comprehensive heat index of the grid cell over the past month. If the overall heat index of a grid cell is greater than the heat threshold, the network data of that grid cell is marked as hot data and stored in the hot storage layer. If the comprehensive heat index of a grid cell is less than or equal to the heat threshold and greater than or equal to the cold threshold, then the network data of that grid cell is marked as warm data and stored in the warm storage layer. If the overall heat index of a grid cell is less than the cold threshold, the network data of that grid cell is marked as cold data and stored in the cold storage layer.

4. The intelligent storage management method for telecommunications network data according to claim 3, characterized in that, The process of constructing the data traceability value score is as follows: The peak value of downlink physical resource block utilization and the maximum value of the number of wireless resource control connected users are collected within a preset time window in the grid cell. The deviation of the downlink physical resource block utilization rate is determined based on the peak value of the downlink physical resource block utilization rate, and the deviation of the downlink physical resource block utilization rate is positively correlated with the peak value of the downlink physical resource block utilization rate. The deviation of the number of wireless resource control (RRR) connected users is determined based on the maximum value of the number of RRR connected users, and the deviation of the number of RRR connected users is positively correlated with the maximum value of the number of RRR connected users. The overall deviation is determined by weighted fusion of the deviation of downlink physical resource block utilization and the deviation of the number of connected users under radio resource control. Based on the network elements in the grid cells of the region that begin to meet the triggering conditions of the anomaly event, the number of network elements connected to the network element in the network element topology snapshot and the total number of network elements in the grid cell are counted from the network element topology snapshot; the number of network elements is normalized to obtain the topology influence factor, which is positively correlated with the number of connected network elements; The data traceability value score is determined by weighted fusion of comprehensive deviation and topological influence factor. The process for determining the traceability value of the data is as follows: If the data traceability value score is greater than or equal to the traceability threshold, the network data of the grid cell is marked as having high traceability value. If the data traceability value score is less than the traceability threshold, the network data of the grid cell will be marked as having low traceability value.

5. The intelligent storage management method for telecommunications network data according to claim 4, characterized in that, The process of constructing the business priority weights is as follows: According to the 3GPP standard, 5QI values ​​are defined for 5G QoS flows of different service types; Business priority weights are determined based on the 5QI values ​​of different business types. If the 5QI value of a business type is greater than the first preset threshold, the business priority weight is 2; If the 5QI value of a business type is greater than or equal to the second preset threshold and less than or equal to the first preset threshold, then the business priority weight is 1. If the 5QI value of a business type is less than the second preset threshold, the business priority weight is 0.

5. If there are multiple service types in the network data, the average weight is taken as the service priority weight; the average weight is the ratio of the service priority weight of the multiple service types to the number of service types. The process for constructing the data activity decay coefficient is as follows: Based on the time interval between the generation and last access of network data in the grid cell, the time interval is divided according to the service type, and then the Weibull distribution is used to perform curve fitting to obtain the activity decay curve and extract the scale parameter. For the network data of the collected grid cells, the data activity decay coefficient is obtained by summing the proportion of each service type and the scale parameter of the service type within the network data of the grid cells.

6. The intelligent storage management method for telecommunications network data according to claim 5, characterized in that, The decision-making process for the dynamic hierarchical storage is as follows: When the overall heat index of a grid cell is higher than the heat threshold, or the business priority weight is greater than or equal to the first weight, the network data of the grid cell will be allocated to the hot storage layer. When the overall heat index of a grid cell is lower than the cold threshold, the priority weight is less than or equal to the second weight, and the data activity decay coefficient is less than the preset decay threshold, the network data of the grid cell will be allocated to the cold storage layer. For network data allocated to grid cells in the warm storage layer, an expiration migration timestamp is additionally written when allocated to the warm storage layer. The migration timestamp is the sum of the product of the current time and the data activity decay coefficient and the expansion factor. After the migration timestamp is reached, the network data will be re-evaluated. If the network data of a grid cell is marked as having high traceability value, it will be retained in the original storage layer even if the overall popularity index of the grid cell is lower than the cold threshold; if the network data of a grid cell is marked as having low traceability value, it will be processed normally.

7. The intelligent storage management method for telecommunications network data according to claim 6, characterized in that, For the network data of the grid cells that have been stored in the hot storage layer and the warm storage layer, the comprehensive heat index is updated on an hourly basis. If the comprehensive heat index of the grid cell in the hot storage layer is lower than the heat threshold within a continuous four-hour update cycle, it is marked as an object to be evaluated. If the overall heat index of the grid cell in the warm storage layer is lower than the cold threshold within a continuous four-hour update cycle, it is marked as an object to be evaluated. For all grid cells marked as objects to be evaluated, the network data is analyzed based on the network data's survival time and temporal locality index; the survival time is the difference between the current evaluation time and the network data's entry time. Using the Weibull survival function, the conditional probability of the data continuing to survive beyond the prediction window, given the survival time, is calculated. The average ratio of the number of times each user is accessed in the first hour to the total number of records within the network data of that grid cell is used as an indicator of temporal locality. The query probability is determined by a weighted fusion of temporal locality indicators and conditional probability. The transfer of network data is determined by comparing the query probability with a preset minimum probability threshold; If the query probability is less than the preset minimum probability threshold, the network data will be transferred to the cold storage layer. If the query probability is greater than or equal to the preset minimum probability threshold, the network data will be retained in the original storage layer. From the collected network element topology snapshots, the topological distance between each network element in the service process is obtained. The topological distance between network elements is the number of hops on the logical interface path between two network elements. The network elements are arranged from smallest to largest topological distance, and the network data is stored from nearest to farthest on the cold storage layer according to the topological distance between network elements from smallest to largest.

8. The intelligent storage management method for telecommunications network data according to claim 7, characterized in that, The current usage capacity of the cold storage layer and the average daily growth rate over the past month are statistically analyzed to obtain the predicted remaining capacity of the cold storage layer; the telecommunications service compliance requirement configuration table is read, which defines the minimum retention period for network data of different service types; If the predicted remaining capacity of the cold storage layer is lower than the minimum capacity, a cleanup strategy will be executed. The process of constructing the cleanup strategy is as follows: Traverse the network data of the grid cells in the cold storage layer. If the network data of the grid cell was generated earlier than the current date minus the statutory minimum retention period for the corresponding business type, and the number of queries in the last six months is zero, then mark it as clearable and generate a list of candidates to be cleared. If the network data of the grid cell in the candidate list to be removed is of high traceability value, it will be removed from the candidate list to be removed and an observation window will be reset. The traceability value of the data will be re-evaluated after the observation window expires. Before cleaning up the candidate list, manual approval is required, and an unchangeable audit log must be generated. The audit log includes information about the network data to be cleaned and the person performing the cleanup operation.

9. An intelligent storage management device for telecommunications network data, applied to the intelligent storage management method for telecommunications network data as described in any one of claims 1-8, characterized in that, include: The baseline heat determination unit is used to respond to the regional business traffic data meeting the preset abnormal event conditions, synchronously collect network data within a preset time window, and determine the baseline heat value of each grid unit in the region. The data classification unit is used to construct a comprehensive popularity index for the grid unit and classify the network data of the grid unit based on the baseline popularity value and the actual amount of network data accessed by the grid unit. The source tracing scoring unit is used to determine the source tracing value score of network data based on the comprehensive deviation degree and topological influence factor, thereby determining the source tracing value of the data; The dynamic hierarchical storage unit is used to dynamically hierarchically store the network data of the grid unit based on the comprehensive heat index and business priority weight of the grid unit. The data activity decay coefficient is constructed based on the proportion of each business type in the network data of the grid unit and the scale parameter of the business type. The evaluation unit is used to filter the network data of the grid cells in the hot storage layer and the warm storage layer, determine the objects to be evaluated, and transfer the network data of the grid cells based on the objects to be evaluated. The execution unit is used to execute cleanup strategies based on the remaining capacity of the cold storage layer.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is used to control the electronic device on which the computer-readable storage medium resides to perform the intelligent storage management method for telecommunications network data as described in any one of claims 1-8 during runtime.

Images