Cleaning rule acquisition method and system based on big data analysis

Abstract

The present invention discloses a method and system for obtaining cleaning rules based on big data analysis, wherein the method includes: collecting big data to obtain cleaning log files; analyzing and processing the cleaning log files to obtain cleaning target results associated with system performance; updating the cleaning rules based on the cleaning target results, and sending the updated cleaning rules to the cleaning engine and the cleaning client so that the cleaning engine and the cleaning client can clean storage space according to the updated cleaning rules. The solution of the present invention can, on the one hand, analyze the cleaning log big data and intelligently update the cleaning rules based on the analysis results; on the other hand, by performing big data analysis on the cleaning log, generate cleaning rules that can optimize system performance, thereby improving the accuracy and effectiveness of cleaning.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for obtaining cleaning rules based on big data analysis. Background Art

[0002] With the rapid development of internet technology, an increasing number of applications have been developed, including food ordering apps, travel apps, payment apps, shopping apps, audio and video apps, and even food ordering apps. These applications touch every aspect of people's lives and provide tremendous convenience. However, the more applications installed on a device, the more storage space they occupy, placing an increasing burden on the system. To address this storage space issue, cleaning engines and cleaning clients have emerged. Installing a cleaning engine or cleaning client on a device allows users to quickly and easily clean up storage space.

[0003] However, in the process of implementing the present invention, the inventors found that in the prior art, the cleaning engine or cleaning client often adopts a unified cleaning rule, which stipulates which objects and directories to clean, the proportion of cleaning space, etc. However, this method of cleaning with a unified cleaning rule has poor flexibility, the cleaning effect often cannot achieve the optimal effect, and intelligent cleaning cannot be achieved. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a method and system for obtaining cleaning rules based on big data analysis that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a method for obtaining cleaning rules based on big data analysis is provided, comprising:

[0006] Collect big data and obtain clean log files;

[0007] Analyze and process the cleanup log files to obtain cleanup target results related to system performance;

[0008] The cleaning rules are updated according to the cleaning target results, and the updated cleaning rules are sent to the cleaning engine and the cleaning client, so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules.

[0009] Optionally, the cleaning target result associated with the system performance specifically includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

[0010] Optionally, updating the cleaning rules according to the cleaning target results and sending the updated cleaning rules to the cleaning engine and the cleaning client specifically includes:

[0011] The cleaning target results are sent to the cleaning operation platform so that the cleaning operation platform can complete the updating process of the cleaning rules according to the pre-made cleaning rule template and the cleaning target results, and send the updated cleaning rules to the cleaning engine and the cleaning client.

[0012] Optionally, analyzing and processing the cleanup log files further includes:

[0013] Perform field splitting on the cleanup log file to obtain each log field, and then perform analysis and processing on each log field.

[0014] Optionally, analyzing and processing the cleanup log file to obtain a cleanup target result associated with system performance further includes:

[0015] Perform statistics on the cleanup log files to obtain log statistics results; the log statistics results include: terminal type information and / or cleanup file information and / or cleanup directory information corresponding to the cleanup log files;

[0016] Based on the log statistics, the cleaning target results related to system performance are obtained;

[0017] The terminal type information is determined based on the operating system information and / or system version information and / or manufacturer information of the terminal.

[0018] Optionally, the method is implemented based on an ELK cluster; the method further includes: displaying the log statistics results.

[0019] Optionally, the big data is collected to obtain cleanup log files, specifically including:

[0020] Monitors the cleanup log files processed by the Hive server. The Hive server pulls the original cleanup log files from the HDFS server for processing.

[0021] According to another aspect of the present invention, a system for obtaining cleaning rules based on big data analysis is provided, comprising:

[0022] The collection module is suitable for collecting big data and obtaining clean log files;

[0023] An analysis module, adapted to analyze and process the cleaning log files to obtain cleaning target results associated with system performance;

[0024] The updating module is adapted to update the cleaning rules according to the cleaning target results, and send the updated cleaning rules to the cleaning engine and the cleaning client so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules.

[0025] Optionally, the cleaning target result associated with the system performance specifically includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

[0026] Optionally, the updating module further includes a cleaning operation platform; the system further includes: a sending module adapted to send the cleaning target result to the cleaning operation platform;

[0027] The cleaning operation platform is suitable for completing the update processing of the cleaning rules according to the pre-made cleaning rule templates and cleaning target results, and sending the updated cleaning rules to the cleaning engine and the cleaning client.

[0028] Optionally, the analysis module is further adapted to:

[0029] Perform field splitting on the cleanup log file to obtain each log field, and then perform analysis and processing on each log field.

[0030] Optionally, the analysis module is further adapted to:

[0031] Perform statistics on the cleanup log files to obtain log statistics results; the log statistics results include: terminal type information and / or cleanup file information and / or cleanup directory information corresponding to the cleanup log files;

[0032] Based on the log statistics, the cleaning target results related to system performance are obtained;

[0033] The terminal type information is determined based on the operating system information and / or system version information and / or manufacturer information of the terminal.

[0034] Optionally, the system is implemented based on an ELK cluster; the Kibana platform displays the log statistics results.

[0035] Optionally, the collection module is further adapted to: monitor the cleanup log files processed by the Hive server; wherein the Hive server pulls the original cleanup log files from the HDFS server for processing.

[0036] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0037] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned cleaning rule acquisition method based on big data analysis.

[0038] According to another aspect of the present invention, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned method for obtaining cleaning rules based on big data analysis.

[0039] The present invention provides a method and system for obtaining cleaning rules based on big data analysis. The method includes: collecting big data to obtain cleaning log files; analyzing and processing the cleaning log files to obtain cleaning target results associated with system performance; updating the cleaning rules based on the cleaning target results, and sending the updated cleaning rules to the cleaning engine and the cleaning client so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules. The solution of the present invention can, on the one hand, analyze the cleaning log big data and intelligently update the cleaning rules based on the analysis results. On the other hand, by performing big data analysis on the cleaning log, it is possible to generate cleaning rules that can optimize system performance, thereby improving the accuracy and effectiveness of cleaning.

[0040] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0042] Figure 1 A schematic diagram of a process for obtaining cleaning rules based on big data analysis according to an embodiment of the present invention is shown;

[0043] Figure 2 A schematic flow chart of a method for obtaining cleaning rules based on big data analysis according to another embodiment of the present invention is shown;

[0044] Figure 3 A functional module diagram of a cleaning rule acquisition system based on big data analysis according to another embodiment of the present invention is shown;

[0045] Figure 4 A functional module diagram of a cleaning rule acquisition system based on big data analysis according to another embodiment of the present invention is shown;

[0046] Figure 5A schematic structural diagram of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0048] Figure 1 A flow chart of a method for obtaining cleaning rules based on big data analysis according to an embodiment of the present invention is shown. Figure 1 As shown, the method includes:

[0049] Step S101: collect big data to obtain a cleanup log file.

[0050] The cleaning engine or cleaning client generates cleaning log files during cleaning, such as cleaning dot logs, cleaning influencer logs, cleaning app logs, etc. In actual applications, the cleaning log files can be uploaded to the file cluster server and pulled from the file cluster server.

[0051] Step S102: Analyze and process the cleaning log file to obtain a cleaning target result associated with system performance.

[0052] The cleanup log can indicate the optimal system performance at the specific cleanup level. Therefore, by analyzing and processing the cleanup log file, we can determine the cleanup target result that optimizes system performance. The cleanup target result can include the target cleanup files and the percentage of the target cleanup space. For example, if the cleanup target result is "tmp-40%," where "tmp" represents the target cleanup files and "40%" represents the percentage of the target cleanup space, then this cleanup target result indicates that 40% of the files in the tmp directory will be cleaned.

[0053] Step S103: updating the cleaning rules according to the cleaning target result, and sending the updated cleaning rules to the cleaning engine and the cleaning client, so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules.

[0054] The cleanup rules are updated based on the cleanup target results. For example, if the cleanup rule before the update was to clean 50% of the files in the tmp directory, the cleanup rule will be updated to clean 40% of the files in the tmp directory based on the cleanup target results. As mentioned above, since the cleanup target results are related to system performance, and cleaning according to the cleanup target results can achieve optimal system performance, the cleanup rules obtained by updating the cleanup rules based on the cleanup target results are also the cleanup rules that can achieve optimal system performance.

[0055] After the cleanup rules are updated, the updated cleanup rules are sent to the cleanup engine and cleanup client. The cleanup engine and cleanup client then delete the corresponding files or data according to the updated cleanup rules. In the above example, before the cleanup rule update, the cleanup engine and cleanup client cleaned 50% of the files in the tmp directory. After the update, the cleanup engine and cleanup client cleaned 40% of the files in the tmp directory.

[0056] According to the cleaning rule acquisition method based on big data analysis provided by this embodiment, by collecting big data, a cleaning log file is obtained; the cleaning log file is analyzed and processed to obtain a cleaning target result associated with system performance; the cleaning rules are updated according to the cleaning target result, and the updated cleaning rules are sent to the cleaning engine and the cleaning client so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules. Using the method of this embodiment, on the one hand, by analyzing the cleaning log big data, the cleaning rules can be intelligently updated according to the analysis results; on the other hand, by performing big data analysis on the cleaning log, cleaning rules that can optimize system performance can be generated, which can improve the accuracy and effect of cleaning.

[0057] Figure 2 A flow chart of a method for obtaining cleansing rules based on big data analysis according to another embodiment of the present invention is shown. The method is implemented based on an ELK cluster (including Elasticsearch, Logstash, and Kibana), wherein Elasticsearch is an open source distributed search engine that provides three major functions: collecting, analyzing, and storing data. Logstash is a tool mainly used for collecting, analyzing, and filtering logs, and supports a large number of data acquisition methods. Kibana is a log analysis-friendly Web interface provided by Logstash and Elasticsearch, which can help aggregate, analyze, and search important data logs. Of course, this is only a specific implementation of the present invention, and the solution of the present invention is not limited to this. As Figure 2 As shown, the method includes:

[0058] Step S201 , monitoring the cleanup log files processed by the Hive server, and the Hive server pulls the original cleanup log files from the HDFS server for processing.

[0059] The cleanup engine or cleanup client generates cleanup log files during cleanup operations, such as cleaning dot logs, cleaning influencer logs, and cleaning app logs. These cleanup log files are uploaded to the HDFS server (file cluster server). The Hive server then pulls the original cleanup log files from the HDFS server for processing, generating cleanup log files.

[0060] Logstash listens to the cleanup log files processed by Hive and then inserts them into the Elasticsearch cluster, which analyzes and processes them. Optionally, after listening to the cleanup log files, Logstash performs field segmentation on them to obtain individual log fields. These fields are then inserted into the Elasticsearch cluster, which then analyzes and processes them.

[0061] Step S202 , performing statistics on the cleaning log files to obtain log statistics results, which include: terminal type information and / or cleaning file information and / or cleaning directory information corresponding to the cleaning log files.

[0062] Statistics can be collected on the cleaning log files. For example, the number of terminals from various manufacturers cleaned during a specific period of time can be counted, or specific directories and files cleaned by terminals with different operating systems (such as Android 8.0 and Android 9.0) can be counted, or specific directories and files cleaned by terminals from different manufacturers (such as Huawei and Xiaomi) can be counted, or specific directories and files cleaned by terminals with different operating system versions from the same manufacturer can be counted. This allows obtaining terminal type information and / or cleaned file information and / or cleaned directory information from the cleaning log files.

[0063] Optionally, the Kibana platform in the ELK cluster displays the statistical results.

[0064] Step S203: Obtain a cleaning target result associated with system performance based on the log statistics result.

[0065] Among them, the cleaning log can reflect the degree of cleaning at which the system's operating performance is optimal. Therefore, by analyzing and processing the log statistical results, the cleaning target result that makes the system performance reach the optimal level is obtained, wherein the cleaning target result includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

[0066] For example, the cleanup target results include: tmp1-40%, tmp2-30%, tmp3-35%, etc., where temp1, tmp2, and tmp3 represent the target cleanup directories, and the corresponding percentages of 40%, 30%, and 35% represent the target cleanup space usage. This cleanup target result indicates: deleting 40% of the files in the tmp1 directory, 30% of the files in the tmp2 directory, 35% of the files in the tmp3 directory, and so on.

[0067] Optionally, based on the log statistics results, the cleaning target results associated with the corresponding system performance of various types of terminals are determined. For example, by analyzing the log statistics results, the cleaning target results associated with the system performance of terminals with different versions of the Android system are obtained, such as the cleaning target results associated with the system performance of terminals with different operating systems such as Android 8.0 or Android 9.0. For another example, by analyzing the log statistics results, the cleaning target results associated with the system performance of terminals from different manufacturers are obtained, such as the cleaning target results associated with the system performance of Huawei smartphones and Xiaomi smartphones. In addition, terminals from the same manufacturer may also have different versions of the system. Therefore, it is possible to further determine the cleaning target results associated with the system performance for terminals with different system versions from the same manufacturer.

[0068] In step S204, the cleaning target result is sent to the cleaning operation platform, so that the cleaning operation platform completes the updating process of the cleaning rule according to the pre-made cleaning rule template and the cleaning target result, and sends the updated cleaning rule to the cleaning engine and the cleaning client.

[0069] A cleaning rule template is set up in the cleaning operation platform. The operator uploads and maintains the cleaning rule template in the operation platform. In actual application, the specific objects, directories, files and cleaning intensity information that need to be cleaned are all put into the cleaning rule template to generate the cleaning rules.

[0070] The cleaning operation platform receives the cleaning target result and inserts the target cleaning space ratio information (i.e., cleaning intensity) and / or target cleaning file information and / or target cleaning directory information contained in the cleaning target result into the cleaning rule template, replacing the relevant information of the cleaning rule already set in the cleaning rule template to obtain a new cleaning rule, thus completing the cleaning rule update. After that, the cleaning operation platform sends the updated cleaning rule to the cleaning engine and cleaning client, which clean the storage space according to the updated cleaning rule.

[0071] According to the above content, since the cleaning target result is related to the system performance, cleaning according to the cleaning target result can make the system performance reach the best. Then, the updated cleaning rules obtained by updating the cleaning rules according to the cleaning target result are the cleaning rules that can make the system performance reach the best. Therefore, after the cleaning engine and the cleaning client complete the cleaning according to the updated cleaning rules, the system performance of the terminal can reach the best.

[0072] For example, assuming that the relevant information of the cleaning rules already set in the cleaning rule template includes: tmp1-50%, tmp2-40%, tmp3-45%, etc., then the above cleaning target results are filled into the cleaning rule template, and the target cleaning space ratio information corresponding to tmp1 is updated from 50% to 40%, the target cleaning space ratio information corresponding to tmp2 is updated from 40% to 30%, and the target cleaning space ratio information corresponding to tmp3 is updated from 45% to 35%, thereby completing the update of the cleaning rules. Then, when the cleaning engine and the cleaning client perform cleaning according to the updated cleaning rules, 40% of the files in the tmp1 directory, 30% of the files in the tmp2 directory, and 35% of the files in the tmp3 directory are cleaned. After the cleaning is completed, the system performance reaches the optimal level.

[0073] Furthermore, based on the above, analysis can be performed to obtain cleaning target results associated with system performance corresponding to various types of terminals. Accordingly, updated cleaning rules corresponding to various types of terminals can be obtained. The operation platform will then distribute these updated cleaning rules corresponding to various types of terminals to the cleaning engine and cleaning client. During cleaning, the cleaning engine and cleaning client select the corresponding cleaning rule based on the type of terminal to be cleaned. Of course, this is merely one specific embodiment of the present invention and is not intended to limit the present invention to this.

[0074] As can be seen, the method of this embodiment uses the ELK cluster to collect cleanup log data processed by HDFS and Hive for cluster analysis, combines the analysis results with the pre-set cleanup rule templates in the cleanup operation platform, and completes the update of the cleanup rules, with high processing efficiency and accuracy. Secondly, by performing big data analysis on the cleanup logs, the cleanup rules can be intelligently adjusted based on the analysis results. In addition, by performing big data analysis on the cleanup logs, cleanup rules that can achieve optimal system performance can be generated, which can improve the accuracy and effectiveness of the cleanup.

[0075] Figure 3 FIG. 4 shows a functional module diagram of a cleaning rule acquisition system based on big data analysis according to another embodiment of the present invention. Figure 3 As shown, the system includes: a collection module 31 , an analysis module 32 and an update module 33 .

[0076] The collection module 31 is adapted to collect big data and obtain cleanup log files;

[0077] An analysis module 32 is adapted to analyze and process the cleaning log files to obtain cleaning target results associated with system performance;

[0078] The updating module 33 is adapted to update the cleaning rules according to the cleaning target result, and send the updated cleaning rules to the cleaning engine and the cleaning client so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules.

[0079] Figure 4 FIG. 4 shows a functional module diagram of a cleaning rule acquisition system based on big data analysis according to another embodiment of the present invention. Figure 4 As shown, the system includes Figure 3 The modules shown in FIG. 4 further include a sending module 41 .

[0080] The collection module 31 is adapted to collect big data and obtain cleanup log files;

[0081] An analysis module 32 is adapted to analyze and process the cleaning log files to obtain cleaning target results associated with system performance;

[0082] The update module 33 also includes a cleanup operation platform 42;

[0083] A sending module 41 adapted to send the cleaning target result to a cleaning operation platform 42;

[0084] The cleaning operation platform 42 is adapted to complete the updating process of the cleaning rules according to the pre-made cleaning rule template and the cleaning target result, and send the updated cleaning rules to the cleaning engine and the cleaning client.

[0085] Optionally, the cleaning target result associated with the system performance specifically includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

[0086] Optionally, the analysis module 32 is further adapted to:

[0087] Perform field splitting on the cleanup log file to obtain each log field, and then perform analysis and processing on each log field.

[0088] Optionally, the analysis module 32 is further adapted to:

[0089] Perform statistics on the cleanup log files to obtain log statistics results; the log statistics results include: terminal type information and / or cleanup file information and / or cleanup directory information corresponding to the cleanup log files;

[0090] Based on the log statistics, the cleaning target results related to system performance are obtained;

[0091] The terminal type information is determined based on the operating system information and / or system version information and / or manufacturer information of the terminal.

[0092] Optionally, the system is implemented based on an ELK cluster; the Kibana platform displays the log statistics results.

[0093] Optionally, the collection module 31 is further adapted to: monitor the cleanup log files processed by the Hive server; wherein the Hive server pulls the original cleanup log files from the HDFS server for processing.

[0094] The specific structure and working principle of each of the above modules can be referred to the description of the corresponding steps in the method embodiment, which will not be repeated here.

[0095] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction. The computer executable instruction can execute the cleaning rule acquisition method based on big data analysis in any of the above method embodiments.

[0096] Figure 5 A schematic structural diagram of a computing device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0097] like Figure 5 As shown, the computing device may include: a processor (processor) 502 , a communications interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0098] in:

[0099] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0100] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0101] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned embodiment of the cleaning rule acquisition method based on big data analysis.

[0102] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0103] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in a computing device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0104] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0105] Program 510 can be specifically used to enable the processor 502 to perform the following operations: collect big data to obtain a cleaning log file; analyze and process the cleaning log file to obtain a cleaning target result associated with system performance; update the cleaning rules based on the cleaning target result, and send the updated cleaning rules to the cleaning engine and the cleaning client, so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules.

[0106] In an optional manner, the cleaning target result associated with the system performance specifically includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

[0107] In an optional manner, the program 510 can also be specifically used to enable the processor 502 to perform the following operations: send the cleaning target results to the cleaning operation platform, so that the cleaning operation platform completes the update processing of the cleaning rules according to the pre-made cleaning rule template and the cleaning target results, and sends the updated cleaning rules to the cleaning engine and the cleaning client.

[0108] In an optional manner, the program 510 may be specifically configured to enable the processor 502 to perform the following operations: performing field segmentation processing on the cleanup log file to obtain various log fields, and performing analysis processing on each log field.

[0109] In an optional manner, the program 510 can also be specifically used to enable the processor 502 to perform the following operations: perform statistics on the cleanup log files to obtain log statistical results; the log statistical results include: terminal type information and / or cleanup file information and / or cleanup directory information corresponding to the cleanup log files; based on the log statistical results, obtain the cleanup target results associated with system performance; wherein the terminal type information is determined based on the terminal's operating system information and / or system version information and / or manufacturer information.

[0110] In an optional manner, the program 510 may be specifically configured to enable the processor 502 to perform the following operations: displaying log statistics results.

[0111] In an optional manner, the program 510 may be specifically configured to enable the processor 502 to perform the following operations: monitor the cleanup log files processed by the Hive server; wherein the Hive server pulls the original cleanup log files from the HDFS server for processing.

[0112] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general-purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0113] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0114] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0115] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0116] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0117] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the computing device according to an embodiment of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing a part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0118] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of suitably programmed computers. In a unit claim enumerating several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A method for obtaining cleaning rules based on big data analysis, comprising: Collect big data and obtain clean log files; Analyzing and processing the cleaning log file to obtain a cleaning target result associated with system performance; Updating the cleaning rules according to the cleaning target result, and sending the updated cleaning rules to the cleaning engine and the cleaning client, so that the cleaning engine and the cleaning client clean the storage space according to the updated cleaning rules; The analyzing and processing the cleaning log file to obtain a cleaning target result associated with system performance further includes: Performing statistics on the cleanup log files to obtain log statistics results; the log statistics results include: terminal type information and / or cleanup file information and / or cleanup directory information corresponding to the cleanup log files; Obtaining a cleaning target result associated with system performance based on the log statistics result; The terminal type information is determined based on the operating system information and / or system version information and / or manufacturer information of the terminal; Among them, the steps of counting the cleanup log files and obtaining the log statistical results include: counting the number of terminals of each manufacturer cleaned within a specific period of time, counting the directories and files cleaned by terminals of different operating systems, counting the directories and files cleaned by terminals of different manufacturers, or for terminals of the same manufacturer, counting the directories and files cleaned by terminals of different system versions to obtain the log statistical results.

2. The method according to claim 1, wherein The cleaning target result associated with the system performance specifically includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

3. The method according to claim 1, wherein The updating of the cleaning rules according to the cleaning target result and sending the updated cleaning rules to the cleaning engine and the cleaning client specifically includes: The cleaning target result is sent to the cleaning operation platform so that the cleaning operation platform completes the updating process of the cleaning rule according to the pre-made cleaning rule template and the cleaning target result, and sends the updated cleaning rule to the cleaning engine and the cleaning client.

4. The method according to any one of claims 1 to 3, wherein The analyzing and processing of the cleaning log file further includes: The cleanup log file is segmented to obtain various log fields, and analysis is performed on the various log fields.

5. The method according to claim 1, wherein The method is implemented based on an ELK cluster; the method further includes: displaying the log statistics results.

6. The method according to claim 5, wherein: The collection of big data to obtain cleanup log files specifically includes: Monitor the cleanup log files processed by the Hive server; wherein the Hive server pulls the original cleanup log files from the HDFS server for processing.

7. A cleaning rule acquisition system based on big data analysis, comprising: The collection module is suitable for collecting big data and obtaining clean log files; An analysis module, adapted to analyze and process the cleaning log file to obtain a cleaning target result associated with system performance; An updating module adapted to update the cleaning rules according to the cleaning target result, and to send the updated cleaning rules to the cleaning engine and the cleaning client so that the cleaning engine and the cleaning client can clean the storage space according to the updated cleaning rules; The analysis module is further adapted to perform statistics on the cleanup log files to obtain log statistics results; The log statistics result includes: terminal type information and / or cleaned file information and / or cleaned directory information corresponding to the cleaned log file; obtaining a cleanup target result associated with system performance based on the log statistics result; wherein the terminal type information is determined based on the operating system information and / or system version information and / or manufacturer information of the terminal; The analysis module is also suitable for counting the number of terminals of various manufacturers cleaned within a specific period of time, counting the directories and files cleaned by terminals of different operating systems, counting the directories and files cleaned by terminals of different manufacturers, or for terminals of the same manufacturer, counting the directories and files cleaned by terminals of different system versions, so as to obtain log statistical results.

8. The system according to claim 7, wherein: The cleaning target result associated with the system performance specifically includes: target cleaning space ratio information and / or target cleaning file information and / or target cleaning directory information.

9. The system according to claim 7, wherein: The updating module also includes a cleaning operation platform; the system further includes: a sending module adapted to send the cleaning target result to the cleaning operation platform; The cleaning operation platform is adapted to complete the updating process of the cleaning rules according to the prefabricated cleaning rule template and the cleaning target result, and send the updated cleaning rules to the cleaning engine and the cleaning client.

10. The system according to any one of claims 7 to 9, wherein: The analysis module is further adapted to: The cleanup log file is segmented to obtain various log fields, and analysis is performed on the various log fields.

11. The system according to claim 7, wherein: The system is implemented based on the ELK cluster; wherein, the Kibana platform displays the log statistics results.

12. The system according to claim 11, wherein The collection module is further adapted to: monitor the cleanup log files processed by the Hive server; wherein the Hive server pulls the original cleanup log files from the HDFS server for processing.

13. A computing device comprising: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the cleaning rule acquisition method based on big data analysis as described in any one of claims 1 to 6.

14. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the cleaning rule acquisition method based on big data analysis as described in any one of claims 1 to 6.

Images