Methods and apparatus for identifying residual files, storage media and computer program products

Abstract

This application discloses a method, apparatus, storage medium, and computer program product for determining residual files, relating to the computer field. The method includes: identifying the first target file as a potential residual file when a first target file stops updating among multiple target files in a target file set; performing a residual rate score on a first monitored object based on the first target file to obtain a residual rate evaluation score corresponding to the first monitored object, wherein the multiple target files correspond one-to-one with multiple monitored objects, and the multiple monitored objects include the first monitored object; and determining the first target file as a residual file if the residual rate evaluation score is greater than a residual rate threshold, wherein the residual file is a file to be cleaned.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a method and apparatus for determining residual files, a storage medium, and a computer program product. Background Technology

[0002] With the widespread application of virtualization technology, resource monitoring of virtualization platforms has become increasingly important. Currently, many virtualization platforms use the Ganglia framework to monitor the operational status of various resources, including virtual machines, hosts, and network devices. The Ganglia framework periodically collects monitoring data and stores it in RRD (Round-Robin Database) files (equivalent to the target file mentioned above). RRD files can store a large amount of time-series data, including CPU utilization, memory utilization, and network traffic. However, in virtualization platforms, the operational status of monitored objects changes frequently; for example, virtual machines may be deleted or temporarily shut down, causing RRD files to stop updating. While the cessation of RRD file updates does not necessarily mean that data is residual, failure to promptly clean up this residual data leads to wasted disk storage space.

[0003] Currently, many virtualization platforms use simple timestamps to determine whether RRD files are residual data, but this method has shortcomings. For example, if a virtual machine is temporarily shut down, the RRD file will stop updating, but this does not mean the data is residual. Meanwhile, historical data also has significant value, such as being used to investigate the causes of recent fault alarms from monitored objects.

[0004] Furthermore, virtualization platforms generate a massive amount of resource monitoring data, requiring an efficient algorithm to process it. Traditional data processing algorithms may not meet the needs of virtualization platforms, necessitating a more optimized algorithm for data identification and cleansing.

[0005] Currently, there is a lack of effective technical solutions to identify residual data in the time-series monitoring of virtualization platform resources, which leads to the problem of low performance of virtualization platforms due to the inability to release hard disk storage space in a timely manner. Summary of the Invention

[0006] This application provides a method and apparatus for determining residual files, a storage medium, and a computer program product, to at least solve the problem in the related art that there is a lack of a technical solution that can effectively identify residual data in the time-series monitoring of virtualization platform resources, and that the inability to release hard disk storage space in a timely manner leads to low performance of the virtualization platform.

[0007] This application provides a method for determining residual files, comprising: identifying the first target file as a potential residual file when a first target file stops updating among multiple target files in a target file set is detected; performing a residual rate score on a first monitored object based on the first target file to obtain a residual rate evaluation score corresponding to the first monitored object, wherein the multiple target files correspond one-to-one with multiple monitored objects, and the multiple monitored objects include the first monitored object; and determining the first target file as a residual file when the residual rate evaluation score is greater than a residual rate threshold, wherein the residual file is a file to be cleaned up.

[0008] This application also provides a device for determining residual files, comprising: a first determining module, configured to determine the first target file as a potential residual file when a first target file stops updating among multiple target files in a set of target files is detected; a scoring module, configured to score the residual rate of a first monitored object based on the first target file to obtain a residual rate evaluation score corresponding to the first monitored object, wherein the multiple target files correspond one-to-one with multiple monitored objects, and the multiple monitored objects include the first monitored object; and a second determining module, configured to determine the first target file as a residual file when the residual rate evaluation score is greater than a residual rate threshold, wherein the residual file is a file to be cleaned up.

[0009] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described methods for determining residual files.

[0010] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described methods for determining residual files.

[0011] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described methods for determining residual files.

[0012] This application addresses the issue that when a first target file stops updating among multiple target files in a target file set, it identifies this first target file as a potential residual file, i.e., a potential file to be cleaned up. Then, based on this first target file, a residual rate score is calculated for the first monitored object, resulting in a residual rate evaluation score. Multiple target files correspond one-to-one with multiple monitored objects, including the first monitored object. If the residual rate evaluation score exceeds a preset residual rate threshold, the first target file is identified as a residual file to be cleaned up. Using this scheme, after detecting that an RRD file has stopped updating, the running status, query frequency, and historical fault alarms of the monitored object are used to comprehensively determine whether the RRD file is a residual file, thereby promptly cleaning up residual data and freeing up storage space. This solves the problem in related technologies where there is a lack of a technical solution that can effectively identify residual data in the time-series monitoring of virtualization platform resources, leading to the inability to release hard disk storage space in a timely manner and resulting in low virtualization platform performance. Attached Figure Description

[0013] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a hardware structure block diagram of a computer terminal for a method of determining residual files according to an embodiment of this application;

[0015] Figure 2 This is a flowchart of a method for determining residual files according to an embodiment of this application;

[0016] Figure 3 This is a system architecture diagram of a virtualization platform time-series monitoring residual data identification and cleaning system based on dynamic perception, according to an embodiment of this application.

[0017] Figure 4 This is a structural block diagram of a residual document determination device according to an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] The specific application environment architecture or specific hardware architecture on which the method for determining residual files depends is described here.

[0022] The methods and embodiments provided in this application can be executed on a computer terminal or similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware block diagram of a computer terminal for a method of determining residual files according to an embodiment of this application. (See diagram below.) Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0023] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the operating system startup method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer terminal. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0025] This embodiment provides a method for determining residual files, including but not limited to those applied to computer terminals. Figure 2 This is a flowchart of a method for determining residual files according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps S202-S206:

[0026] Step S202: If it is detected that a first target file has stopped updating among multiple target files in the target file set, the first target file is identified as a potential residual file.

[0027] Step S204: The residual rate of the first monitoring object is scored according to the first target file to obtain the residual rate evaluation score corresponding to the first monitoring object. The multiple target files correspond one-to-one with the multiple monitoring objects, and the multiple monitoring objects include the first monitoring object.

[0028] Step S206: If the residual rate assessment score is greater than the residual rate threshold, the first target file is determined to be a residual file, wherein the residual file is a file to be cleaned.

[0029] Through the above steps, when a first target file stops updating among multiple target files in the target file set, this first target file is identified as a potential residual file, i.e., a potential file to be cleaned up. Then, a residual rate score is calculated for the first monitored object based on this first target file, resulting in a residual rate evaluation score. Multiple target files correspond one-to-one with multiple monitored objects, including the first monitored object. If the residual rate evaluation score is greater than a preset residual rate threshold, the first target file is identified as a residual file to be cleaned up. Using this scheme, after detecting that an RRD file has stopped updating, the running status, query frequency, and historical fault alarms of the monitored object are used to comprehensively determine whether the RRD file is a residual file, thereby promptly cleaning up residual data and freeing up storage space. This solves the problem in related technologies where there is a lack of a technical solution that can effectively identify residual data in the time-series monitoring of virtualization platform resources, leading to the inability to release hard disk storage space in a timely manner and resulting in low virtualization platform performance.

[0030] In an exemplary embodiment, before determining the first target file as a potential residual file when a first target file stops updating among multiple target files in a target file set is detected, the method further includes: obtaining the last modification timestamp of a second target file among the multiple target files at preset time intervals; determining whether the difference between the last modification timestamp and the current timestamp is greater than a first time threshold; if it is determined to be greater than the first time threshold, querying the current status of a second monitoring object corresponding to the second target file; and if it is determined that the current status of the second monitoring object is an inactive state, determining that the second target file has stopped updating.

[0031] In one embodiment, before the monitoring target file set discovers that the first target file has stopped updating, it is necessary to additionally perform the following: obtain the last modification timestamp of the second target file in the set at a preset time interval, verify whether the difference between this timestamp and the current time exceeds a first time threshold, and if it is confirmed that it exceeds the threshold, then investigate the status of the second monitoring object associated with the second target file. If the object is inactive, it is determined that the second target file has stopped updating.

[0032] The target file set refers to a collection of RRD files used to record time-series data of different monitored object metrics in virtualization platform resource monitoring. The preset time interval is for periodically checking the file update status to ensure the real-time performance and accuracy of the monitoring system. The last modification timestamp represents the time when the file was last modified and is a key reference for determining the file update status.

[0033] The first time threshold is set to a reasonable time period to determine whether a file has not been updated for an extended period. This serves as the initial signal to infer possible changes in the monitored object's state. The second monitored object is the entity corresponding to the second target file, such as a virtual machine or network device. Its current state needs to be confirmed through system queries to distinguish between normal inactive states (e.g., shutdown, pause) and abnormal residual states.

[0034] The definition of an inactive state covers situations where the monitored object no longer performs routine operations, such as virtual machines being shut down or network devices going offline. This means that the associated monitoring data is no longer being updated. The determination of whether a second target file has stopped updating is based on the above process, ensuring that the actual situation of the monitored object is fully considered when making decisions to clean up residual files. This avoids accidental data deletion due to temporary changes in the monitored object's state, and ensures efficient use of system storage space and the integrity of monitoring data.

[0035] This approach allows for more accurate identification of truly residual files, rather than simply relying on update timestamps. It effectively prevents conflicts between valid residual monitoring data and excessive cleanup, thereby improving the reliability of the entire monitoring system and the level of intelligence in resource management.

[0036] Optionally, the residual rate score of the first monitored object is obtained by scoring the residual rate of the first monitored object based on the first target file, including: parsing the first target file to determine the operating status of the first monitored object based on the parsing result; determining the first residual rate assessment score of the first monitored object based on the operating status; and determining the first residual rate assessment score as the residual rate assessment score if the first residual rate assessment score is a first value.

[0037] The process of obtaining the residual rate assessment score related to the first monitored object by scoring the residual rate of the first target file involves: parsing the file to understand the operating status of the monitored object, establishing the first residual rate assessment score of the monitored object based on this operating status, and when the first residual rate assessment score falls within a specific first value range, the score is directly used as the residual rate assessment score.

[0038] Parsing refers to reading and understanding the information in the first target file. By analyzing the file content, the system can identify the real-time status of the corresponding monitored objects, such as virtual machines and servers, including whether they are running, paused, under maintenance, or removed. The determination of the running status is based on the current activity level of the monitored object; the lower the activity level, the higher the probability of residue, and vice versa. The determination of the first residue rate assessment score depends on the assessment of the monitored object's running status. Different statuses are assigned different residue rate scores. For example, running scores 0, under maintenance scores 10, paused scores 20, and removed scores 100. The lower the score, the higher the object's activity and the lower the likelihood of residue.

[0039] When the first residual rate assessment score is equal to or lower than a preset first value, it indicates that the residual risk of the monitored object is controllable. In this case, the first residual rate assessment score is directly used as the residual rate assessment score, without the need to consider other factors, simplifying the scoring process and improving system processing efficiency. This process ensures that the virtualization platform resource monitoring system can more accurately and efficiently assess the residual characteristics of data when processing historical data, providing a solid data foundation for subsequent data cleaning strategies.

[0040] Optionally, after determining the first residual rate assessment score of the first monitored object based on the operating status, the method further includes: if the first residual rate assessment score is not the first value, determining the second residual rate assessment score of the first monitored object based on the fault alarm history of the first monitored object, and determining the third residual rate assessment score of the first monitored object based on the query frequency of the first monitored object; and determining the residual rate assessment score based on the first residual rate assessment score, the second residual rate assessment score, and the third residual rate assessment score.

[0041] After the first residual rate assessment score determined based on the operating status exceeds the preset first value range, the fault alarm history and query frequency of the first monitored object are additionally considered to derive the second and third residual rate assessment scores respectively; then, the first, second and third residual rate assessment scores are integrated to make a comprehensive determination of the final residual rate assessment score.

[0042] The primary monitoring object refers to a specific resource on the virtualization platform, such as a virtual machine or network device. Changes in its status directly affect the value and effectiveness of the monitoring data. The primary residual rate assessment score is derived from the assessment of the operational status of the primary monitoring object and reflects the fundamental relationship between object activity and residual risk; decreased activity corresponds to increased residual risk.

[0043] The fault alarm history details past abnormal events occurring on the primary monitored object, including time, type, and severity level. This is a crucial basis for determining data value and the likelihood of residual events. The residual rate assessment score depends on the analysis of the alarm records; frequent or severe alarm events make the data more valuable, thus lowering the residual probability assessment.

[0044] Query frequency reflects the usage density of the data of the primary monitored object. High-frequency queries indicate active and valuable data, while low-frequency queries have the opposite effect. The third residual rate assessment score, based on the analysis results of query frequency, is closely related to the determination of data residual. Active data has a low risk of residual, while inactive data has a higher risk of residual.

[0045] Based on the above scores, the residual dynamic sensing and judgment device uses a weighted average method or other comprehensive scoring strategies to generate the final residual rate assessment score, ensuring that the assessment results are comprehensive and accurate, and providing strong support for subsequent data cleaning decisions. This refined process enhances the accuracy of residual data identification, avoids misjudgments that may occur based solely on operating status, and improves the data management efficiency and storage space utilization of the virtualization platform resource monitoring system.

[0046] Optionally, determining the second residual rate assessment score of the first monitored object based on its fault alarm history includes: obtaining the fault alarm history of the first monitored object within a first time period, wherein the first time period is before the current time and the time difference between the first time period and the current time is less than a second time threshold; classifying multiple alarm records in the fault alarm history to obtain a set of alarm records for multiple alarm levels; determining the alarm deduction score for alarm records of different alarm levels according to a first scoring strategy, wherein the alarm deduction score is proportional to the alarm level; and determining the second residual rate assessment score corresponding to the fault alarm history based on the alarm deduction score for different alarm levels.

[0047] The second residual rate assessment score of the first monitored object is determined based on its fault alarm history. This involves: extracting the fault alarm history of the first monitored object within a second time threshold before the current moment; classifying these alarm records to form alarm sets of different levels; calculating alarm deductions for each level of alarm according to the first scoring strategy, with the deductions being positively correlated with the alarm severity; and finally, summarizing all alarm deductions to obtain the second residual rate assessment score.

[0048] The fault alarm history record refers to the summary of all abnormal events experienced by the first monitored object within a specific time period in the past. These events include, but are not limited to, performance degradation, hardware failure, or software anomalies. Each record includes the alarm time, type, and severity level. The setting of the first time period ensures that the system only considers recent alarm events, avoiding interference from outdated information in the current assessment.

[0049] Alarm classification and severity levels: Alarm records are categorized into different levels based on their severity, such as minor, moderate, and severe. Each level of alarm record has a different impact on the system and reflects the potential residual risk of the monitored data. Alarm deduction is a quantitative indicator used to describe the negative impact of records of a specific alarm level on residual risk assessment. Severe alarms have a higher deduction than minor alarms, reflecting the inherent link between data preservation and residual judgment.

[0050] The calculation of the second residual rate assessment score integrates the sum of alarm deductions. Based on historical fault alarm data, the system can more comprehensively assess the residual risk of data, ensuring the accuracy and rationality of data cleaning decisions. This process enhances the complexity and accuracy of residual data identification, fully considers the historical value of the data, avoids data loss that may result from simple rules, and improves the data management efficiency and storage space optimization level of the virtualization platform resource monitoring system.

[0051] Optionally, determining the third residual rate assessment score of the first monitored object based on the query frequency of the first monitored object includes: querying the total number of query operations of the first monitored object within a second time period, and determining the query frequency based on the total number of operations and the second time period, wherein the second time period is before the current time and the time difference between the second time period and the current time is less than a third time threshold; determining the query deduction corresponding to the query frequency based on a second scoring strategy, wherein the query deduction is inversely proportional to the query frequency; and determining the third residual rate assessment score corresponding to the query frequency based on the query deduction.

[0052] The query frequency is calculated by counting the total number of queries made to the first monitored object within the second time period immediately preceding the current time. A second scoring strategy is then applied to convert the query frequency into query deduction points, with the deduction points being inversely proportional to the frequency. Finally, based on the query deduction points, a third residual rate score related to the query frequency is determined.

[0053] Query frequency refers to the frequency with which monitoring data for the primary monitored object is retrieved within a given time period, reflecting the activity and importance of the data in upper-level business applications. The selection of the second time period is usually a recent time window, ensuring that the query frequency statistics reflect the latest data usage and avoiding misleading residual assessments due to outdated queries.

[0054] Query deduction, as a calculation mechanism, is used to convert query frequency into an integral score for assessing residual risk. Monitoring data with high-frequency queries has a lower deduction score due to its activity and importance, while data that has not been queried for a long time has a higher deduction score, indicating an increased residual risk. Query deduction is inversely proportional to query frequency; that is, the more queries, the lower the deduction score and the smaller the residual risk.

[0055] The third residual rate assessment score, derived from query errors, reflects the residual risk level of monitoring data under recent usage conditions. This assessment process provides a more detailed judgment on data residuals, avoiding misjudgments of data as residual due to its high value. This ensures that the virtualization platform resource monitoring system makes more accurate decisions when cleaning up time-series monitoring data, thereby optimizing storage space usage and improving overall system performance.

[0056] Based on the above steps, the residue rate assessment score is determined according to the first residue rate assessment score, the second residue rate assessment score, and the third residue rate assessment score, including: using the formula P=aP A +bP B +cP C Calculate the residual rate assessment score, where P is the residual rate assessment score. A P is the score for the first residual rate assessment. B P is the score for the second residual rate assessment. C The score for the third residual rate assessment is given, where a, b, and c are weighting constants.

[0057] The process of determining the total score for the residue rate assessment includes: averaging the scores from the first, second, and third residue rate assessments using the weighted summation formula P=aP. A +bP B +cP C The overall residual rate score P is calculated, where a, b, and c are weighting constants for each evaluation dimension, representing the importance of different scoring factors in the overall evaluation.

[0058] The residual rate assessment score is an evaluation result that comprehensively considers three dimensions: the operating status of the monitored object, the history of fault alarms, and the query frequency, forming a comprehensive judgment on the possibility of residual monitoring data.

[0059] The first residual rate assessment score, based on the operational status assessment of the monitored object, reflects the direct correlation between the object's activity level and potential residual risk. Higher activity level results in a lower score and a lower residual risk.

[0060] The second residual rate assessment score is derived from the assessment results of the fault alarm history. Taking into account the impact of historical fault alarms on the value of monitoring data, the score reflects the residual risk under abnormal data conditions. Frequent alarms or important alarms will reduce the score and lower the assessment of residual risk.

[0061] The third residual rate assessment score is derived from the query frequency assessment, reflecting the indirect relationship between the frequency of use of monitoring data in upper-level business and residual risk. High-frequency query data has a lower score due to its activity and importance, and the residual risk is relatively small.

[0062] Weighting constants a, b, and c are used to balance the weight of each residual rate assessment score in the overall evaluation. These constants can be adjusted according to different application scenarios and needs to ensure that the final P score accurately reflects the residual risk of the data. This comprehensive scoring mechanism improves the accuracy and flexibility of residual data identification, helps optimize the virtualization platform resource monitoring system in storage space management and monitoring data cleanup, and enables more intelligent data residual assessment and cleanup strategies.

[0063] Optionally, before determining the first target file as a residual file when the residual rate assessment score is greater than the residual rate threshold, the method further includes: obtaining real-time system parameters of the target system, wherein the target file set is stored in the target system, and the real-time system parameters include: system load, data query frequency, and hard disk occupancy; predicting the optimal residual rate threshold based on the real-time system parameters by dynamically adjusting the model to obtain the optimal residual rate threshold corresponding to the current moment; and updating the residual rate threshold to the optimal residual rate threshold.

[0064] Before determining whether the first target file is a residual file, it is necessary to collect real-time parameters of the target system, including system load, data query frequency, and hard disk utilization. These parameters reflect the real-time operating status of the system. Using a dynamic adjustment model, the current optimal residual rate threshold is predicted based on the obtained real-time parameters to adapt to changes in the system state. The original residual rate threshold is then updated to the predicted optimal value to ensure dynamic accuracy in residual file determination.

[0065] The target system, specifically referring to the virtualization platform or its resource monitoring system, includes real-time system parameters such as system load, data query frequency, and hard disk utilization. These parameters collectively depict the current operating pressure, data activity, and storage capacity of the system, and are key indicators that determine system efficiency and performance.

[0066] System load refers to the amount of tasks that a target system is processing at a certain moment, including CPU utilization, memory usage ratio, and number of threads. High system load means more resource consumption and will also affect the strategy for cleaning up residual data.

[0067] Data query frequency reflects the density of monitoring data usage in upper-level business processes. Frequently queried data may represent high value or criticality, requiring more cautious residual data assessment strategies. Disk utilization rate is the proportion of currently stored data in the target system's total storage space. A high disk utilization rate indicates high system storage pressure, potentially requiring more aggressive residual data cleanup strategies.

[0068] The model is dynamically adjusted, employing mathematical modeling and data analysis techniques to dynamically predict the optimal residual rate threshold based on real-time system parameters. This ensures that the determination of residual files is neither too conservative, leading to wasted storage space, nor too aggressive, resulting in the loss of important data. This optimization process enhances the flexibility and accuracy of residual data identification, helps improve the data management capabilities of the virtualization platform resource monitoring system, and ensures that the system remains highly efficient and stable under different operating conditions.

[0069] Optionally, before predicting the optimal residual rate threshold based on the real-time system parameters by dynamically adjusting the model, the method further includes: acquiring historical storage data of the target system, wherein the historical storage data includes historical residual rate thresholds under different system states, as well as the corresponding storage space utilization rate of the target system and the data preservation integrity results of the multiple target files; and training the dynamic adjustment model using the historical storage data.

[0070] Before using a dynamic adjustment model to predict the optimal residual rate threshold, it is necessary to collect historical storage information of the target system, including historical results of residual rate thresholds under different operating conditions, system storage space utilization efficiency, and target file preservation integrity. With the help of historical data, a dynamic adjustment model is trained to achieve accurate prediction of the current optimal threshold.

[0071] Historical storage data is a record of the target system's operational history, including historical residual rate thresholds, storage space utilization, and data preservation integrity under different system states. This data reveals the correlation between system operating conditions and residual data management strategies, providing a foundation for dynamically adjusting model training.

[0072] The historical residual rate threshold represents the sensitivity of the target system to residual data at different time points. It is an important reference for dynamically adjusting model training and helps the model learn the optimal strategy for threshold setting under different system states.

[0073] Storage space utilization rate, which is the proportion of data previously stored in the target system to the total storage capacity, reflects the efficiency of the system's storage space utilization and is a key indicator for evaluating the system's operating status and the rationality of its storage strategy.

[0074] The data preservation integrity result of the target file refers to the preservation status of the target file during the historical residual data cleaning process, including whether it was accidentally deleted, whether it remained intact, and the stability of the system after cleaning. These results provide empirical evidence for the model to optimize the residual file judgment strategy.

[0075] By analyzing the correlations in historical stored data, the dynamically adjusted model learns the optimal strategy for setting residual rate thresholds under different system states and storage pressures. This ensures that sufficient space is reserved for important data when real-time parameters change, while effectively cleaning up truly residual data, thereby achieving efficient storage space management and complete data preservation. This dynamically adjusted model, trained on historical data, significantly improves the intelligence of residual data identification and is an important component of the virtualization platform resource monitoring system for achieving accurate data management.

[0076] Optionally, the method further includes: statistically analyzing the access frequency and storage period of a third target file among the plurality of target files, and determining a target compression algorithm corresponding to the third target file based on the access frequency and the storage period; when the storage time of the third target file reaches a fourth time threshold, or the file size of the third target file reaches a preset compression threshold, compressing the third target file using the target compression algorithm, and storing the compressed third target file in the target system, wherein the set of target files is stored in the target system.

[0077] The access frequency and storage time limit of the third target file in multiple target files are statistically analyzed to select a matching compression algorithm. When the storage time of the third target file reaches the fourth time limit or the file size exceeds the preset compression start point, the selected compression algorithm is used to compress the file, and then the compressed file is stored back to the target system to optimize storage space.

[0078] It should be noted that the target file set specifically refers to all monitoring data files stored in the target system, including all data of objects to be monitored or already monitored, including third-party target files, which constitute the main content of the system storage.

[0079] The third target file is a member of the target file set that receives special attention. Its access frequency and storage period are key factors in determining the compression strategy. Access frequency refers to the number of times the third target file is read or accessed within a certain period, reflecting the file's activity level. Storage period refers to the duration a file has been stored in the system since its creation. A longer storage period may indicate a reduced need for file access, making it a candidate for compression.

[0080] The target compression algorithm is the most suitable compression method selected based on the access frequency and storage cycle characteristics of the target file, aiming to minimize storage space usage while maintaining file data integrity. The fourth time threshold sets the storage duration limit for the file to be compressed; once the file storage time reaches or exceeds this limit, the system automatically triggers compression processing.

[0081] A preset compression threshold is a standard set for initiating compression based on file size. When a file size reaches or exceeds this threshold, it will also be included in the compression process. This additional optimization step not only specifically reduces storage usage but also intelligently adjusts the compression strategy based on file access frequency and storage cycle, ensuring efficient storage and convenient access to monitoring data. It is an effective means for virtualization platform resource monitoring systems to improve data management efficiency.

[0082] Optionally, after determining that the first target file is a residual file when the residual rate assessment score is greater than the residual rate threshold, the method further includes: removing the first target file from the target system, wherein the set of target files is stored in the target system.

[0083] For the first target file whose residual rate assessment score exceeds the residual rate threshold, after being determined to be a residual file, the following actions are immediately taken: remove the first target file from the target system, free up storage space, and streamline the monitoring database. The target file set, i.e., the system environment in which all monitoring data files are stored, is the execution location of this operation.

[0084] The residual rate assessment score is a quantitative indicator for assessing the likelihood of file residuals. It is derived by comprehensively considering the operating status of the monitored object, fault alarm records, and query frequency. When the score is higher than the preset residual rate threshold, the file is considered a residual file and should be cleaned up.

[0085] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. To better understand the above method, the following description, in conjunction with embodiments, illustrates the process, but is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0086] In an optional embodiment, this application provides a dynamic-aware virtualization platform time-series monitoring residual data identification and cleanup system, such as... Figure 3 As shown, the system includes: a time-series database storage medium, a monitoring object status listening device, a query frequency calculation device, a fault alarm collection module, a residual dynamic sensing and analysis device, and a cleaning device. The functions and roles of each device are described below in conjunction with the system's processing flow:

[0087] (1) Time-series database storage medium, which contains a collection of time-series database RRD files corresponding to various monitoring indicators of all monitored objects in the system. When an RRD file that has stopped updating is found, it is initially judged to be a potential residual file.

[0088] (2) The monitoring object status listening device receives RRD files that have stopped updating from the RRD file library of the time-series database storage medium, finds the running status of the corresponding monitoring object by parsing the RRD file, and evaluates different residual rate scores P for different running statuses. A (i.e., the first residual rate assessment score mentioned above), for example, with a full score of 100, the running status is 0 points, the maintenance status is 10 points, the paused status is 20 points, the shutdown status is 50 points, and the object does not exist status is 100 points. When the score is lower than 100 points (i.e., the first value mentioned above), the RRD file will continue to be sent to the remaining module query frequency calculation device and the fault alarm collection module for further analysis.

[0089] (3) The fault alarm collection module finds the historical fault alarm records of the associated monitored objects in the system in a recent period based on the received RRD file, and evaluates the residual rate score P based on the alarm importance level and the number of alarm occurrences. B (i.e., the second residual rate assessment score mentioned above), with a full score of 100 points. A deduction strategy is adopted. If there are 2 major level alarms and 3 minor level alarms, the final score is calculated as 100-2*20-3*10=30 points.

[0090] (4) The query frequency calculation device checks the relevant interface call logs based on the received RRD file, analyzes the number of times the RRD file has been queried in a recent period, and evaluates the residual rate score P. C (i.e., the third residual rate assessment score mentioned above), with a full score of 100 points. A deduction strategy is adopted. For example, if the query is conducted 200 times within the past week, 0.1 points will be deducted each time. The final assessment score will be 100-200*0.1=80 points.

[0091] (5) After steps (2)-(4), the residual rate assessment scores of the three modules are transmitted to the residual dynamic sensing and judgment device for calculation. The final residual rate calculation formula is P=aP A +bP B +cP C Where a, b, and c are weighting constants, a+b+c=1, and different values ​​can be set according to different judgment scenarios to adapt to different judgment needs. The final residual rate assessment score P is calculated. When P is greater than the residual rate threshold, it is finally determined as a monitoring residual RRD (i.e. the above residual file).

[0092] (6) When the final residual rate score (i.e. the residual rate assessment score) evaluated by the residual dynamic sensing and judgment device in step (5) exceeds the residual rate threshold, the cleaning device starts to delete the RRD file and release the hard disk storage space in time.

[0093] This application designs a reasonable monitoring residual file evaluation strategy, which analyzes from multiple dimensions such as the operating status of the monitored object, fault alarms, and query frequency. It can effectively identify and clean up residual data of virtualization platform resource time-series monitoring, free up hard disk storage space, and improve the performance of the virtualization platform.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0095] Embodiments of this application also provide a device for determining residual files. Figure 4 This is a residual document determination device according to an embodiment of this application, such as... Figure 4 As shown, the device includes:

[0096] The first determining module 42 is used to determine the first target file as a potential residual file when it is detected that the first target file has stopped updating among multiple target files in the target file set;

[0097] The scoring module 44 is used to score the residual rate of the first monitoring object based on the first target file to obtain the residual rate evaluation score corresponding to the first monitoring object. The plurality of target files correspond one-to-one with the plurality of monitoring objects, and the plurality of monitoring objects include the first monitoring object.

[0098] The second determining module 46 is used to determine the first target file as a residual file when the residual rate assessment score is greater than the residual rate threshold, wherein the residual file is a file to be cleaned.

[0099] Using the above device, when a first target file stops updating among multiple target files in a target file set, this first target file is identified as a potential residual file, i.e., a potential file to be cleaned up. Then, a residual rate score is calculated for the first monitored object based on this first target file to obtain a residual rate evaluation score. Here, multiple target files correspond one-to-one with multiple monitored objects, including the first monitored object. If the residual rate evaluation score is greater than a preset residual rate threshold, the first target file is identified as a residual file to be cleaned up. By adopting the above scheme, after detecting that an RRD file has stopped updating, the running status, query frequency, historical fault alarms, and other indicators of the monitored object are used to comprehensively determine whether the RRD file is a residual file, thereby cleaning up residual data and storage space in a timely manner. This solves the problem in related technologies where there is a lack of a technical solution that can effectively identify residual data in the time-series monitoring of virtualization platform resources, resulting in low performance of the virtualization platform due to the inability to release hard disk storage space in a timely manner.

[0100] Optionally, the first determining module 42 is further configured to obtain the last modification timestamp of the second target file in the plurality of target files according to a preset time interval; determine whether the difference between the last modification timestamp and the current timestamp is greater than a first time threshold; if it is determined to be greater than the first time threshold, query the current status of the second monitoring object corresponding to the second target file; if it is determined that the current status of the second monitoring object is an inactive state, determine that the second target file stops updating.

[0101] Optionally, the scoring module 44 is further configured to parse the first target file to determine the operating status of the first monitored object based on the parsing result; determine the first residual rate evaluation score of the first monitored object based on the operating status; and determine the first residual rate evaluation score as the residual rate evaluation score if the first residual rate evaluation score is a first value.

[0102] Optionally, the scoring module 44 is further configured to determine a second residual rate assessment score of the first monitored object based on the fault alarm history of the first monitored object when the first residual rate assessment score is not the first value, and to determine a third residual rate assessment score of the first monitored object based on the query frequency of the first monitored object; and to determine the residual rate assessment score based on the first residual rate assessment score, the second residual rate assessment score and the third residual rate assessment score.

[0103] Optionally, the scoring module 44 is further configured to: acquire the fault alarm history records of the first monitored object within a first time period, wherein the first time period is prior to the current time and the time difference between the first time period and the current time is less than a second time threshold; classify multiple alarm records in the fault alarm history records to obtain a set of alarm records for multiple alarm levels; determine the alarm deduction score of alarm records for different alarm levels according to a first scoring strategy, wherein the alarm deduction score is proportional to the alarm level; and determine the second residual rate evaluation score corresponding to the fault alarm history records based on the alarm deduction scores for different alarm levels.

[0104] Optionally, the scoring module 44 is further configured to query the total number of query operations performed by the first monitored object within a second time period, and determine the query frequency based on the total number of operations and the second time period, wherein the second time period is prior to the current time and the time difference between the second time period and the current time is less than a third time threshold; determine the query score loss corresponding to the query frequency based on a second scoring strategy, wherein the query score loss is inversely proportional to the query frequency; and determine the third residual rate evaluation score corresponding to the query frequency based on the query score loss.

[0105] Optionally, the scoring module 44 described above is also used to calculate the score using the formula P=aP. A +bP B +cP C Calculate the residual rate assessment score, where P is the residual rate assessment score. A P is the score for the first residual rate assessment. B P is the score for the second residual rate assessment. C The score for the third residual rate assessment is given, where a, b, and c are weighting constants.

[0106] Optionally, the second determining module 46 described above is further configured to obtain real-time system parameters of the target system, wherein the target file set is stored in the target system, and the real-time system parameters include: system load, data query frequency, and hard disk occupancy rate; by dynamically adjusting the model, the optimal residual rate threshold is predicted based on the real-time system parameters to obtain the optimal residual rate threshold corresponding to the current moment; and the residual rate threshold is updated to the optimal residual rate threshold.

[0107] Optionally, the second determining module 46 is further configured to acquire historical storage data of the target system, wherein the historical storage data includes historical residual rate thresholds under different system states, as well as the corresponding storage space utilization rate of the target system and the data preservation integrity results of the multiple target files; and to train the dynamic adjustment model using the historical storage data.

[0108] Optionally, the first determining module 42 is further configured to count the access frequency and storage period of the third target file among the multiple target files, and determine the target compression algorithm corresponding to the third target file based on the access frequency and the storage period; when the storage time of the third target file reaches a fourth time threshold, or the file size of the third target file reaches a preset compression threshold, the third target file is compressed using the target compression algorithm, and the compressed third target file is stored in the target system, wherein the set of target files is stored in the target system.

[0109] Optionally, the second determining module 46 described above is further configured to remove the first target file from the target system, wherein the set of target files is stored in the target system.

[0110] For a description of the features in the embodiment corresponding to the residual file determination device, please refer to the relevant description in the embodiment corresponding to the residual file determination method, which will not be repeated here.

[0111] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the method for determining residual files.

[0112] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the method for determining residual files when it is run.

[0113] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0114] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described methods for determining residual files.

[0115] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described methods for determining residual files.

[0116] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] The foregoing has provided a detailed description of the method, apparatus, storage medium, and computer program product for determining residual files provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for determining residual files, characterized in that, include: If, in the case that the first target file stops updating among multiple target files in the target file set, the first target file is identified as a potential residual file; The residual rate of the first monitored object is scored based on the first target file to obtain the residual rate evaluation score corresponding to the first monitored object. The multiple target files correspond one-to-one with multiple monitored objects, and the multiple monitored objects include the first monitored object. If the residual rate assessment score is greater than the residual rate threshold, the first target file is determined to be a residual file, wherein the residual file is a file to be cleaned; Specifically, the residual rate of the first monitored object is scored based on the first target file to obtain the residual rate assessment score corresponding to the first monitored object, including: The first target file is parsed to determine the running status of the first monitored object based on the parsing results; Determine the first residual rate evaluation score of the first monitored object based on the operating status; If the first residual rate assessment score is a first value, the first residual rate assessment score is determined as the residual rate assessment score; After determining the first residual rate evaluation score of the first monitored object based on the operating status, the method further includes: If the first residual rate assessment score is not the first value, the second residual rate assessment score of the first monitored object is determined based on the fault alarm history of the first monitored object, and the third residual rate assessment score of the first monitored object is determined based on the query frequency of the first monitored object. The residual rate assessment score is determined based on the first residual rate assessment score, the second residual rate assessment score, and the third residual rate assessment score.

2. The method for determining residual files according to claim 1, characterized in that, Before identifying the first target file as a potential residual file when multiple target files in the target file set are detected to have stopped updating, the method further includes: The last modification timestamp of the second target file in the plurality of target files is obtained at preset time intervals. Determine whether the difference between the last modified timestamp and the current timestamp is greater than a first time threshold; If the time threshold is determined to be greater than the first time threshold, query the current status of the second monitoring object corresponding to the second target file; If the current state of the second monitored object is determined to be inactive, the second target file is determined to stop updating.

3. The method for determining residual files according to claim 1, characterized in that, The second residual rate assessment score of the first monitored object is determined based on the fault alarm history of the first monitored object, including: Obtain the fault alarm history of the first monitored object within a first time period, wherein the first time period is before the current time and the time difference between the first time period and the current time is less than a second time threshold; Multiple alarm records in the fault alarm history are classified to obtain alarm record sets with multiple alarm levels; The alarm score deduction for alarm records of different alarm levels is determined according to the first scoring strategy, wherein the alarm score deduction is proportional to the alarm level; The second residual rate evaluation score corresponding to the fault alarm history is determined based on the alarm deduction scores of different alarm levels.

4. The method for determining residual files according to claim 1, characterized in that, The third residual rate assessment score of the first monitored object is determined based on the query frequency of the first monitored object, including: The total number of query operations performed on the first monitored object within the second time period is queried, and the query frequency is determined based on the total number of operations and the second time period, wherein the second time period is before the current time and the time difference between the second time period and the current time is less than a third time threshold. The query score loss corresponding to the query frequency is determined according to the second scoring strategy, wherein the query score loss is inversely proportional to the query frequency; The third residual rate assessment score corresponding to the query frequency is determined based on the query loss.

5. The method for determining residual files according to claim 1, characterized in that, The residue rate assessment score is determined based on the first residue rate assessment score, the second residue rate assessment score, and the third residue rate assessment score, including: Using the formula P=aP A +bP B +cP C Calculate the residual rate assessment score, where P is the residual rate assessment score. A P is the score for the first residual rate assessment. B P is the score for the second residual rate assessment. C The score for the third residual rate assessment is given, where a, b, and c are weighting constants.

6. The method for determining residual files according to claim 1, characterized in that, Before determining the first target file as a residual file if the residual rate assessment score is greater than the residual rate threshold, the method further includes: Obtain real-time system parameters of the target system, wherein the target file set is stored in the target system, and the real-time system parameters include: system load, data query frequency, and hard disk occupancy rate; By dynamically adjusting the model, the optimal residual rate threshold is predicted based on the real-time system parameters to obtain the optimal residual rate threshold at the current moment. Update the residual rate threshold to the optimal residual rate threshold.

7. The method for determining residual files according to claim 6, characterized in that, Before dynamically adjusting the model and predicting the optimal residual rate threshold based on the real-time system parameters, the method further includes: Obtain historical storage data of the target system, wherein the historical storage data includes historical residual rate thresholds under different system states, as well as the corresponding storage space utilization rate of the target system and the data preservation integrity results of the multiple target files; The dynamic adjustment model is trained using the historical stored data.

8. The method for determining residual files according to claim 1, characterized in that, The method further includes: The access frequency and storage period of the third target file among the multiple target files are statistically analyzed, and the target compression algorithm corresponding to the third target file is determined based on the access frequency and the storage period. When the storage time of the third target file reaches a fourth time threshold, or when the file size of the third target file reaches a preset compression threshold, the third target file is compressed using the target compression algorithm, and the compressed third target file is stored in the target system, wherein the target file set is stored in the target system.

9. The method for determining residual files according to claim 1, characterized in that, After determining that the first target file is a residual file when the residual rate assessment score is greater than the residual rate threshold, the method further includes: The first target file is removed from the target system, wherein the set of target files is stored in the target system.

10. A device for determining residual documents, characterized in that, include: The first determination module is used to determine the first target file as a potential residual file when it is detected that the first target file has stopped updating among multiple target files in the target file set; The scoring module is used to score the residual rate of the first monitoring object based on the first target file, and obtain the residual rate evaluation score corresponding to the first monitoring object. The multiple target files correspond one-to-one with multiple monitoring objects, and the multiple monitoring objects include the first monitoring object. The second determining module is used to determine the first target file as a residual file when the residual rate assessment score is greater than the residual rate threshold, wherein the residual file is a file to be cleaned; The scoring module is further configured to parse the first target file to determine the operating status of the first monitored object based on the parsing result; determine a first residual rate evaluation score of the first monitored object based on the operating status; and determine the first residual rate evaluation score as the residual rate evaluation score if the first residual rate evaluation score is a first value. The scoring module is further configured to determine a second residual rate assessment score of the first monitored object based on the fault alarm history of the first monitored object when the first residual rate assessment score is not the first value, and to determine a third residual rate assessment score of the first monitored object based on the query frequency of the first monitored object; and to determine the residual rate assessment score based on the first residual rate assessment score, the second residual rate assessment score and the third residual rate assessment score.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as claimed in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 9.

Images