RPKI data processing dynamic storage method

Abstract

According to the dynamic storage scheme for the RPKI data storage system, system parameters are optimized by closely surrounding the real-time load condition of the RPKI data storage system, on one hand, the processing level of an RPKI analysis algorithm is dynamically adjusted according to the CPU occupancy rate and the disk write-in rate of the system, so that CPU resources are reasonably configured, the operation efficiency is improved, and on the other hand, the operation efficiency is improved; on the other hand, the IO operation data size is automatically adjusted according to the fine regulation and control of the data disk falling rate and the change trend of the processing level of an analysis algorithm, the adaptation of the data disk falling rhythm and the system running state is achieved, the efficient cooperation of RPKI data analysis and RPKI data storage is guaranteed, and in addition, based on the collected CPU occupancy rate and the disk write-in rate, the RPKI data storage efficiency is improved. The CPU utilization rate threshold value and the disk write-in rate threshold value are dynamically calibrated so as to meet large-scale RPKI data processing requirements, and a solid guarantee is provided for efficient and stable operation of a system under multiple working conditions.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, specifically to the field of RPKI data management, and more specifically, to a method for dynamically storing RPKI data in an RPKI storage system. Background Art

[0002] In today's explosive and rapid development of the Internet, the effective management and security guarantee of network resources have become crucial and urgent key issues. Against this background, the Resource Public Key Infrastructure (RPKI) emerged as the times require. It aims to provide a trust system based on public key infrastructure for Internet number resources. The three core components of RPKI, namely CA, RP, and repository, play an indispensable role in building a secure and trustworthy network resource allocation and verification environment. Among them, CA is responsible for authenticating and issuing digital certificates for information related to network resources, and these certificates carry key information such as authorization of network resources; RP performs operations such as resource verification based on the certificates issued by CA to ensure that the use of network resources complies with established rules; as a key link in data storage, the repository is responsible for storing various digital objects, including information such as certificates and Route Origin Authorizations (ROAs), providing basic support for the data flow of the entire system.

[0003] Since 2012, the five RIRs have actively promoted the deployment of RPKI in their affiliated Autonomous Systems (ASs), which reflects the attention and demand of the global Internet industry for the security management of network resources. During this process, its influence has continued to spread throughout the Internet field.

[0004] In the development process of RPKI, the deployment scale of RPKI has been continuously expanding, and more network resources have been incorporated into the management scope of RPKI. A series of problems that need to be solved urgently have gradually emerged. First, over time, the amount of ROA data has shown a significant annual growth trend, and the ROA fields have become more and more diverse. This directly leads to huge pressure on the RP during calculation and processing. A large number of complex data calculation tasks exceed the load of the traditional RP processing ability, greatly reducing the timeliness of data processing, which may lead to delays in network resource verification, and then affect the normal provision and quality guarantee of network services. Secondly, for the storage system, the huge storage demand for RPKI data poses a severe challenge. When processing this data, the input / output (IO) performance of the storage system becomes a bottleneck. Facing large-scale and high-frequency data read and write operations, the traditional storage architecture often proves inadequate. The data storage speed is difficult to meet the real-time requirements of the system, and the delay in reading data will also affect the cooperation efficiency among various components in the entire RPKI system, which may lead to problems such as excessive waiting time due to slow data acquisition during the RP verification process, thereby reducing the operation efficiency and reliability of the entire RPKI system and threatening the security and stability of Internet number resource management.

[0005] The problems of the traditional RPKI storage system are prominent. In terms of RPKI data parsing, the existing technology adopts a fixed parsing strategy of full-scale parsing, without considering the CPU load of RPKI storage and the scale of RPKI data. When the system faces a high CPU load situation, it still performs full-scale parsing operations on RPKI data, which will occupy a large amount of computing resources and cause a sharp decline in the overall performance of the system. In terms of RPKI data storage, the traditional storage mode is inefficient and difficult to meet the large-scale data storage demand. It can neither effectively guarantee the data storage efficiency nor easily endanger the data integrity and reliability due to storage problems, and thus it is difficult to meet the stringent requirements of modern Internet for RPKI data management. Therefore, there is an urgent need for an RPKI data storage method that can flexibly and intelligently adjust the parsing and storage strategies closely based on the real-time load status of the RPKI storage system.

[0006] It should be noted that: This background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily prior art. Without evidence showing that the relevant information has been made public before the filing date of the present invention, the relevant information should not be regarded as prior art. Summary of the Invention

[0007] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an RPKI data dynamic storage method for the RPKI storage system.

[0008] The object of the present invention is achieved by the following technical solutions:

[0009] According to a first aspect of the present invention, a method for dynamically storing RPKI data in an RPKI storage system is proposed. The RPKI storage system includes a CPU, a disk, and an IO, and is used to collect RPKI data and perform operations of pre-processing and then storing the collected RPKI data. The method includes: initializing the IO operation data volume, the processing level of the RPKI parsing algorithm, the disk write rate threshold, and the CPU usage threshold; collecting RPKI data in real time; pre-processing the RPKI data collected in real time and then parsing it according to the current processing level of the RPKI parsing algorithm in the system to obtain the parsed RPKI data; storing the parsed RPKI data on the disk according to the current IO operation data volume; and collecting the CPU occupancy rate and the disk write rate of the RPKI storage system at a preset period, and adjusting the processing level of the RPKI parsing algorithm and the IO operation data volume in a preset manner based on the collected CPU usage rate and disk write rate.

[0010] Preferably, the processing level of the RPKI parsing algorithm can be configured into multiple levels. Among them, the higher the processing level of the RPKI parsing algorithm, the higher the parsing accuracy of the RPKI data.

[0011] Preferably, the processing level of the RPKI parsing algorithm is initialized to: the middle level of all configurable processing levels of the RPKI parsing algorithm.

[0012] Preferably, the preset manner is: when the CPU usage rate is greater than or equal to the current CPU usage threshold, lower the RPKI parsing algorithm by one processing level; when the CPU usage rate is less than the current CPU usage threshold and the disk write rate is less than the current disk write rate threshold, raise the RPKI parsing algorithm by one processing level.

[0013] Preferably, the configurable processing levels of the RPKI parsing algorithm include level 1, level 2, and level 3. Among them, the parsing accuracy of level 1 is the lowest. The RPKI parsing algorithm configured as level 1 only parses the necessary field information in the RPKI data. The parsing accuracy of level 3 is the highest. The RPKI parsing algorithm configured as level 3 parses all the field information in the RPKI data. The parsing accuracy of level 2 is between level 1 and level 3. The RPKI parsing algorithm configured as level 2 parses the necessary field information and some other field information in the RPKI data.

[0014] Preferably, if in multiple consecutive preset periods, the CPU usage rate is greater than the current CPU usage threshold, or the CPU usage rate is less than the initial CPU usage threshold, adjust the processing level of the RPKI parsing algorithm to the initial processing level.

[0015] Preferably, the amount of IO operation data is adjusted as follows: whenever the RPKI parsing algorithm increases a processing level, the amount of IO operation data is increased by a first preset amount; whenever the RPKI parsing algorithm decreases a processing level, the amount of IO operation data is decreased by the first preset amount.

[0016] Preferably, the first preset amount is any value within the range of 0.5GB - 1GB.

[0017] Preferably, if the CPU usage rate is greater than the current CPU usage rate threshold in multiple consecutive preset periods, the CPU usage rate threshold is increased by a second preset amount; if the CPU usage rate is less than the initial CPU usage rate threshold, the CPU usage rate threshold is adjusted to the initial CPU usage rate threshold.

[0018] Preferably, if the disk write rate is greater than the current disk write rate threshold in multiple consecutive preset periods, the disk write rate threshold is increased by a second preset amount; if the disk write rate is less than the initial disk write rate threshold, the disk write rate threshold is adjusted to the initial disk write rate threshold.

[0019] Preferably, the second preset amount is any value within the range of 10% - 20%.

[0020] Preferably, the multiple consecutive preset periods are any one of three consecutive preset periods, four consecutive preset periods, and five consecutive preset periods.

[0021] Compared with the prior art, the advantages of the present invention are as follows:

[0022] The present invention proposes a dynamic storage scheme for an RPKI data storage system. This scheme closely relies on the real-time status of the RPKI data storage system, flexibly adjusts the accuracy level of RPKI data parsing and synchronously adjusts the data disk writing rate, so as to achieve a scientific and reasonable allocation of internal resources in the RPKI data storage system, and also meets the strict standards of large-scale RPKI data parsing operations and storage tasks at the present stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The following further describes the embodiments of the present invention with reference to the drawings, where:

[0024] Figure 1 is a schematic diagram of the steps of a dynamic storage scheme for an RPKI data storage system according to an embodiment of the present invention;

[0025] Figure 2 is a schematic diagram of the steps of an RPKI data storage method according to an embodiment of the present invention;

[0026] Figure 3 Schematic diagram of an RPKI data storage system according to an embodiment of the present invention. Detailed implementation manners

[0027] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0028] As mentioned in the background art section, the existing RPKI data storage solutions have the following problems: In terms of RPKI data parsing, the existing technologies adopt a fixed parsing strategy of full-scale parsing without considering the CPU load of RPKI storage and the scale of RPKI data. When the system faces a high CPU load situation, full-scale parsing operations are still performed on RPKI data, which will occupy a large amount of computing resources, resulting in a sharp decline in the overall performance of the system. In terms of RPKI data storage, the traditional storage mode is inefficient and difficult to meet the large-scale data storage requirements. It can neither effectively ensure the storage efficiency of data nor easily endanger the integrity and reliability of data due to storage problems, and thus it is difficult to meet the stringent requirements of modern Internet for RPKI data management.

[0029] To effectively solve the above problems, the present invention proposes a dynamic storage solution for an RPKI data storage system. This solution closely adjusts the accuracy of RPKI data parsing and synchronously adjusts the data writing rate according to the real-time load status of the RPKI data storage system, so as to realize the reasonable allocation of resources of the RPKI data storage system, and then fully meet the stringent requirements of large-scale RPKI data parsing and storage.

[0030] To better understand the present invention, the present invention will be described in detail below in combination with specific embodiments.

[0031] According to an embodiment of the present invention, the present invention proposes a dynamic storage solution for an RPKI data storage system, and the process of this solution is as Figure 1As shown in the figure, this figure shows the specific steps for implementing RPKI data storage in this solution, including: Step S1, initialize parameters, initialize the amount of IO operation data, the processing level of the RPKI parsing algorithm, the disk write rate threshold, and the CPU usage threshold; Step S2, collect RPKI data, collect RPKI data in real time; Step S3, system monitoring, collect the system CPU usage and disk write rate according to a preset period. Preferably, the preset period is configured as 10s; Step S4, adjust the processing level of the RPKI parsing algorithm and the CPU usage threshold based on the collected CPU usage and disk write rate; Step S5, adjust the amount of IO operation data based on the change in the processing level of the RPKI parsing algorithm, and adjust the disk write rate threshold based on the collected disk write rate. Among them, Steps S3-S5 are continuously looped according to a preset period. And when executing Steps S3-S5, the system also continuously operates on the collected RPKI data and the operation of parsing and then storing the collected RPKI data. Specifically, the system preprocesses the real-time collected RPKI data and then parses it according to the current processing level of the RPKI parsing algorithm in the system to obtain the parsed RPKI data, and then stores the parsed RPKI data on the disk according to the current amount of IO operation data.

[0032] It should be noted that the RPKI parsing algorithm is a parsing algorithm used to parse RPKI data to obtain the field information to be stored. This algorithm can perform different-precision parsing of RPKI data by configuring different processing levels. Specifically, the higher the configured processing level, the higher the parsing precision of the RPKI data, and the more detailed the field information that can be captured. Correspondingly, the higher the requirement for the data storage rate of the RPKI data storage system. In addition, the parsing precision of the RPKI parsing algorithm is also positively correlated with its algorithm complexity. The higher the parsing precision of the PKI parsing algorithm, the higher the corresponding algorithm complexity. Correspondingly, the more CPU resources required to execute the algorithm. Therefore, reasonably adjusting the processing level of the RPKI parsing algorithm is crucial for ensuring the efficient and stable operation of the system.

[0033] According to an embodiment of the present invention, the RPKI parsing algorithm in the RPKI data storage system can be configured into multiple processing levels. Exemplarily, the configurable levels of the RPKI parsing algorithm include Level 1, Level 2, and Level 3. Among them, Level 1 is the processing level with the lowest parsing accuracy for RPKI data, and the data obtained after parsing is the least. Correspondingly, the data to be stored is also the least. Level 1 only parses the necessary field information and stores it as database data; Level 3 is the processing level with the highest parsing accuracy for RPKI, and the data obtained after parsing is the most. Correspondingly, the data to be stored is also the most. Level 3 parses RPKI data most comprehensively, including database data, the original file data collected, the file content re-translated after parsing, and the memory cache update content; Level 2 has a parsing accuracy for RPKI data between Level 1 and Level 2. The information stored in Level 2 is the database data of the RPKI parsing data and the original file storage. It should be understood that the three configurable processing levels of the above RPKI parsing algorithm are only exemplary and non-exhaustive. Users can pre-configure the number of processing levels of the RPKI parsing algorithm and the parsing accuracy and parsed field information of each processing level according to their needs. How to configure multiple processing levels of the RPKI parsing algorithm is well-known technology to those skilled in the art and will not be elaborated here again.

[0034] According to an embodiment of the present invention, in this solution, the RPKI parsing algorithm is initialized to the middle level of all configurable processing levels. Exemplarily, if all configurable processing levels of the RPKI parsing algorithm include Level 1, Level 2, and Level 3, then the processing level of the initial RPKI parsing algorithm is configured to Level 2. If all configurable processing levels of the RPKI parsing algorithm include Level 1, Level 2, Level 3, and Level 4, then the processing level of the initial RPKI parsing algorithm is configured to Level 2 or Level 3. If all configurable processing levels of the RPKI parsing algorithm include Level 1, Level 2, Level 3, Level 4, and Level 5, then the processing level of the initial RPKI parsing algorithm is configured to Level 3.

[0035] According to an embodiment of the present invention, in this solution, there are two ways to adjust the processing level of the RPKI parsing algorithm based on the collected CPU usage rate, including: Way 1, in each cycle, according to the numerical relationship between the collected CPU usage rate and the current CPU usage rate threshold of the system, and the numerical relationship between the collected disk write rate and the current disk write rate threshold of the system, to increase or decrease the processing level of the RPKI parsing algorithm. This way only adjusts the processing level of the RPKI parsing algorithm; Way 2, adjusts the processing level of the RPKI algorithm according to the relationship between the CPU usage rate and the CPU usage rate threshold in multiple consecutive cycles.

[0036] According to an embodiment of the present invention, if the processing level of the RPKI parsing algorithm is adjusted by Method 1, it can be subdivided into three specific cases as follows: (1) When the CPU usage rate is greater than or equal to the current CPU usage rate threshold, the CPU occupancy rate of the system is too high. To avoid excessive CPU resources being occupied by parsing RPKI data, the RPKI parsing algorithm is lowered by one processing level to reduce the algorithm complexity of RPKI, thereby reducing the CPU occupancy rate and ensuring the stability of the system. (2) When the CPU usage rate is less than the current CPU usage rate threshold and the disk write rate is less than the current disk write rate threshold, this indicates that the CPU resources occupied by the current RPKI algorithm processing level are still within the system's bearable range and have not reached the current CPU usage rate threshold. At the same time, since the disk write rate is less than the current disk write rate threshold, when the current system stores the parsed data, it has not reached the current disk write rate threshold. Therefore, even if the RPKI algorithm processing level is moderately increased to cause a corresponding increase in the data generation volume, considering the system resource bearing capacity and the current running situation comprehensively, it is very likely that the existing disk write rate threshold will not be exceeded, and thus no additional impact will be caused to the system storage performance. Therefore, to ensure the accuracy of RPKI data parsing as much as possible, the RPKI parsing algorithm is increased by one processing level. (3) When the CPU usage rate is less than the current CPU usage rate threshold and the disk write rate is greater than or equal to the current disk write rate threshold, although the CPU resources occupied by the current RPKI algorithm processing level are still within the system's bearable range, when storing the data parsed by the current RPKI algorithm processing level, it has exceeded the current disk write rate threshold. Therefore, if the RPKI algorithm processing level is increased, although the CPU resources may be within the system's bearable range, it will affect the data storage operation, resulting in a sudden increase in disk write pressure, thereby causing disk I / O blockage, further exacerbating the system response delay, and even possibly causing partial data loss. In view of this, to ensure the stability and reliability of the system, the RPKI algorithm processing level is not changed in this case.

[0037] According to an embodiment of the present invention, the adjustment of the processing level of the RPKI algorithm by Method 2 is specifically as follows: When, in multiple consecutive cycles, there is a situation where the CPU usage rate is greater than the current CPU usage rate threshold or the CPU usage rate is less than the initial CPU usage rate threshold, the processing level of the RPKI algorithm is adjusted to the initial processing level of the RPKI algorithm.

[0038] According to an embodiment of the present invention, when adjusting the processing level of the RPKI algorithm by Method 2, it is also necessary to adjust the CPU usage threshold. Specifically, if the CPU usage rate is greater than the current CPU usage threshold in multiple consecutive cycles, it indicates that the processing level of the RPKI algorithm has been decreasing in these consecutive cycles, but it still exceeds the CPU usage threshold. Therefore, it can be inferred that the current CPU usage threshold is too low, and the CPU usage threshold needs to be increased. Preferably, it is increased by 10%-20% based on the original CPU usage threshold. Exemplarily, if the CPU usage threshold was originally 50%, the increased CPU usage threshold is any value between 55% and 60%; if the CPU usage rate is less than the initial CPU usage threshold in multiple consecutive cycles, it indicates that the CPU resources occupied by the current RPKI parsing data are small, and the CPU resource allocation can be reduced, so the CPU usage threshold is adjusted to the initial CPU usage threshold. This solution provides a solid guarantee for the efficient and stable operation of the system under diverse operating conditions by dynamically calibrating the CPU usage threshold to avoid frequent adjustment of the algorithm processing level.

[0039] When the processing level of the RPKI parsing algorithm changes, it will cause fluctuations in the amount of parsed data, which in turn leads to changes in the data storage rate. According to an embodiment of the present invention, in this solution, the amount of IO operation data is adjusted according to the change in the processing level of the RPKI parsing algorithm to control the data storage rate. Specifically, whenever the RPKI parsing algorithm decreases one processing level, the amount of IO operation data is decreased. Preferably, the amount of IO operation data is decreased by any amount between 0.5GB and 1GB; whenever the RPKI parsing algorithm increases one processing level, the amount of IO operation data is increased. Preferably, the amount of IO operation data is increased by any amount between 0.5GB and 1GB. This solution accurately adjusts the amount of IO operation data involved in data storage according to the dynamic change trend of the algorithm processing level, can effectively provide a solid guarantee for the data storage performance, ensure that the data can be efficiently and reliably stored at different processing stages, and at the same time, helps to break the limitations of traditional resource allocation, and ultimately achieves the goal of maximizing the system resource utilization efficiency, laying a solid foundation for the stable and efficient operation of the entire RPKI data storage system.

[0040] According to an embodiment of the present invention, in this solution, the disk write rate threshold is adjusted according to the numerical relationship between the disk write rate collected in multiple consecutive preset periods and the current disk write rate threshold of the system. Specifically, if the disk write rate is greater than the current disk write rate threshold in multiple consecutive preset periods, the disk write rate threshold is increased by 10%-20% on the original basis. Exemplarily, if the disk write rate threshold was originally 100M / s, the increased disk write rate threshold is any value between 110M / s and 120M / s; if the disk write rate is less than the initial disk write rate threshold in multiple consecutive preset periods, the disk write rate threshold is reset to the initial disk write rate threshold.

[0041] According to an embodiment of the present invention, the present invention implements a method for storing RPKI data according to the above-mentioned dynamic storage solution for the RPKI data storage system. The flow of this method is as Figure 2As shown in the figure, the key steps of implementing RPKI data storage by the RPKI data storage method are shown in the figure. Specifically, they include: Step A1, load the initial RPKI parsing algorithm processing level 2 (initial level 2), IO block size (32k), monitoring period (every 10s), threshold adjustment period (5 monitoring periods), CPU usage rate of 80% and disk write rate of 100MB / s. These default values can be configured and adjusted according to actual requirements. Set appropriate CPU usage rate thresholds (such as exceeding 80%) and disk write rate thresholds (such as less than 100MB / s) according to historical data and business requirements; Step A2, start the scanning thread, collect the CPU usage rate and disk write rate at preset time intervals. The scanning thread can start one or more at the same time. The preset period (set to every 10s in this embodiment) periodically collects the CPU usage rate and disk write rate. Step A3, judge whether the CPU usage rate collected in each period exceeds the CPU usage rate threshold. If the collected CPU usage rate is higher than or equal to the CPU load threshold, reduce the current RPKI parsing algorithm processing level. If the collected CPU usage rate is lower than the CPU usage rate threshold, it further includes judging whether the disk write rate exceeds the disk write rate threshold (disk write rate of 100MB / s). According to the judgment result, adjust the RPKI parsing algorithm processing level again. Specifically, if the collected disk write rate is lower than the threshold (100MB / s), increase the RPKI parsing algorithm processing level; if the collected disk write rate is equal to or higher than the disk write rate threshold, maintain the current RPKI parsing algorithm processing level. When the CPU usage rate exceeds 80%, reduce the algorithm level each time from the currently used RPKI parsing algorithm processing level until the CPU usage rate is lower than the threshold or drops to the lowest processing level of the RPKI parsing algorithm to reduce the burden on the CPU. When the CPU load is normal (that is, the collected CPU usage rate is less than the CPU usage rate threshold), then judge whether the IO performance (disk write rate) is lower than the initial threshold of 100MB / s. If it is lower, increase the algorithm level step by step from the currently used RPKI parsing algorithm processing level until the IO write rate threshold is restored or the highest processing level of the RPKI parsing algorithm is increased to reduce the CPU load and improve the IO performance.When the collected CPU usage rate is higher than or equal to the CPU usage rate threshold, the processing level is preferentially reduced to relieve the CPU burden. When the CPU load drops below the threshold, the level is adjusted according to the disk write rate (IO performance). This can avoid a sharp decline in system performance caused by high-intensity storage while the CPU load is extremely high. Step A4: Data storage. According to the RPKI parsing algorithm processing level reduced in step A3, the parameters of the IO operation are adjusted. Specifically, if the RPKI parsing algorithm processing level is reduced, the amount of data per IO operation is increased; if the RPKI parsing algorithm processing level is increased, the amount of data per IO operation is decreased. Step A5: Determine whether the CPU usage rate or the disk write rate exceeds the current CPU usage rate threshold or the current disk write rate threshold in consecutive monitoring cycles. If so, the current CPU usage rate threshold or the current disk write rate threshold is increased; if the CPU usage rate / disk write rate is lower than the initial CPU usage rate threshold / initial disk write rate threshold in consecutive monitoring cycles, the initial CPU usage rate threshold / initial disk write rate threshold is restored; if the CPU usage rate / disk write rate is lower than or equal to the current CPU usage rate threshold / current disk write rate threshold in any one of the consecutive monitoring cycles, the current CPU usage rate threshold / initial disk write rate threshold is maintained. When the CPU usage rate / disk write rate exceeds the CPU usage rate threshold / disk write rate threshold in 3 consecutive monitoring cycles, the CPU usage rate threshold / disk write rate threshold is adjusted to avoid frequent parsing strategy adjustments. Specifically, it is increased by 10%-20% based on the original CPU usage rate threshold / disk write rate threshold.

[0042] According to an embodiment of the present invention, based on the above RPKI data storage method, the present invention proposes an RPKI data storage system, as Figure 3As shown in the figure, it shows the composition structure of the RPKI data storage system, which includes an initialization module, an RKPI data acquisition module, an RKPI data preprocessing module, an RKPI data parsing and processing module, an IO operation module, and a monitoring module. Specifically, the functions implemented by each module are as follows: The initialization module is used to initialize the parameters of the RPKI storage system, initialize the RKPI acquisition algorithm, the RKPI data parsing and processing level, the data volume of IO operations, the CPU usage threshold, the disk write rate threshold, the monitoring period (the period for collecting the CPU usage and disk write rate of the RPKI storage system, preferably, the monitoring period is initialized to 10s), and the threshold adjustment period (that is, several consecutive monitoring periods, preferably, the threshold adjustment period is initialized to 3 - 5 monitoring periods); The RKPI data acquisition module is used to collect various RKPI update resource libraries from the network resource library to obtain the RKPI data to be stored; The RKPI data preprocessing module is used to preprocess the RKPI data collected by the RKPI data acquisition module, and the preprocessing includes RKPI data decoding, classification, and index creation; The RKPI data parsing and processing module is used to parse and translate the RKPI data, that is, to parse the RKPI data using the processing level of the RKPI parsing algorithm of the current RPKI storage system. The RKPI data parsing and processing module can perform data parsing and storage according to different parameters. The data storage adopts storage methods such as database, source file storage, copy file storage, and memory cache storage, and selects an appropriate parsing algorithm for storage according to the processing complexity level of the system; The IO operation module is used to store the RKPI data parsed by the RKPI data parsing and processing module according to the current IO operation data stream of the RPKI data storage system, and dynamically adjust the parameters of the IO operation (the data volume of the IO operation) through the output of the data parsing and processing module; The monitoring module is used to collect the CPU usage and disk write rate according to the monitoring period, compare them with the current CPU usage threshold and disk write rate threshold of the system, and transfer the judgment result to the data parsing and processing module to enable the data parsing and processing module to adjust the processing level of the RPKI parsing algorithm to complete the parsing. The monitoring module is also used to judge according to the numerical relationship between the CPU usage or the current disk write rate according to the threshold adjustment period, and transfer the judgment result to the data parsing and processing module.

[0043] The solution of the present invention adjusts the parameters of the system based on the real-time load status of the RPKI data storage system. On the one hand, it dynamically adjusts the processing level of the RPKI parsing algorithm according to the CPU occupancy rate and disk write rate of the system, so as to ensure the efficient and reasonable allocation of CPU resources and greatly improve the computing efficiency. On the other hand, it focuses on the fine control of the data disk writing rate, and automatically adjusts the amount of IO operation data according to the dynamic change of the parsing algorithm processing level to achieve the adaptation of the data disk writing rate to the system operation state, so as to ensure the smoothness of the data storage process. In addition, the RPKI data storage system also dynamically calibrates the CPU usage threshold and disk write rate threshold based on the collected CPU occupancy rate and disk write rate to meet the requirements of large-scale RPKI data processing, providing a solid guarantee for the efficient and stable operation of the system under various working conditions.

[0044] It should be noted that although the above steps are described in a specific order, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order, as long as the required functions can be achieved.

[0045] The present invention can be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement various aspects of the present invention.

[0046] The computer-readable storage medium can be a tangible device that retains and stores instructions for use by an instruction execution device. The computer-readable storage medium may include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing.

[0047] The embodiments of the present invention have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled persons in the technical field to understand the embodiments disclosed herein.

Claims

1. A method for dynamically storing RPKI data in an RPKI storage system, wherein the RPKI storage system includes a CPU, a disk, and an IO, and is used to collect RPKI data and process the collected RPKI data before storing it, wherein: The method comprises: Initialize the IO operation data volume, the processing level of the RPKI parsing algorithm, the disk write rate threshold, and the CPU usage threshold; Collect RPKI data in real time; After preprocessing the real-time collected RPKI data, the data is parsed according to the processing level of the current RPKI parsing algorithm of the system to obtain the parsed RPKI data; Store the parsed RPKI data to disk based on the current IO operation data volume; Furthermore, the CPU usage and disk write rate of the RPKI storage system are collected according to a preset period, and the processing level and IO operation data volume of the RPKI parsing algorithm are adjusted according to a preset method based on the collected CPU usage and disk write rate.

2. The method according to claim 1, characterized in that The processing level of the RPKI parsing algorithm may be configured as multiple levels, wherein the higher the processing level of the RPKI parsing algorithm, the higher the accuracy of the RPKI data parsing.

3. The method according to claim 2, characterized in that The processing level of the RPKI resolution algorithm is initialized to the middle level of all configurable processing levels of the RPKI resolution algorithm.

4. The method according to claim 3, characterized in that: The preset method is: When the CPU usage is greater than or equal to the current CPU usage threshold, the RPKI parsing algorithm is reduced by one processing level; when the CPU usage is less than the current CPU usage threshold and the disk write rate is less than the current disk write rate threshold, the RPKI parsing algorithm is increased by one processing level.

5. The method according to claim 2, characterized in that: The configurable processing levels of the RPKI parsing algorithm include level 1, level 2 and level 3, wherein level 1 has the lowest parsing accuracy, and the RPKI parsing algorithm configured as level 1 only parses necessary field information in RPKI data; level 3 has the highest parsing accuracy, and the RPKI parsing algorithm configured as level 3 parses all field information in RPKI data; and level 2 has a parsing accuracy between level 1 and level 3, and the RPKI parsing algorithm configured as level 2 parses necessary field information and part of other field information in RPKI data.

6. The method according to claim 1, characterized in that If the CPU usage is greater than the current CPU usage threshold in multiple consecutive preset cycles, or the CPU usage is less than the initial CPU usage threshold, the processing level of the RPKI resolution algorithm is adjusted to the initial processing level.

7. The method according to claim 1, characterized in that Adjust the IO operation data volume in the following ways: Whenever the RPKI parsing algorithm increases a processing level, the IO operation data volume is increased by a first preset amount; Whenever the RPKI parsing algorithm decreases a processing level, the IO operation data volume is reduced by a first preset amount.

8. The method according to claim 7, characterized in that The first preset amount is any value in the range of 0.5 GB-1 GB.

9. The method according to claim 1, characterized in that: If the CPU usage is greater than the current CPU usage threshold in multiple consecutive preset periods, the CPU usage threshold is increased by a second preset amount; if the CPU usage is less than the initial CPU usage threshold, the CPU usage threshold is adjusted to the initial CPU usage threshold.

10. The method according to claim 1, characterized in that If the disk write rate is greater than the current disk write rate threshold in multiple consecutive preset cycles, the disk write rate threshold is increased by a second preset amount; if the disk write rate is less than the initial disk write rate threshold, the disk write rate threshold is adjusted to the initial disk write rate threshold.

11. The method according to any one of claims 9 and 10, characterized in that: The second preset number is any value within the range of 10%-20%.

12. The method according to any one of claims 6, 9 and 10, characterized in that: The multiple consecutive preset periods are any one of three consecutive preset periods, four consecutive preset periods, and five consecutive preset periods.

Images