Method for managing garbage files in public cloud based on AI intelligent classification

Abstract

The application provides a kind of method for garbage file management in public cloud based on AI intelligent classification, specifically: 1. cloud computing data management center initializes system parameters, 2. execute server-side garbage file classification algorithm, establish garbage file list, 3. server-side public garbage recovery station recycles garbage files regularly, releases storage space, 4. public cloud user end establishes garbage recovery classifier. The application proposes the primary standard of garbage file classification in public cloud, establishes the buffer zone of public garbage recovery station in public cloud, uniformly manages the files in garbage station, designs diversified garbage classification recovery device that is beneficial to user operation, realizes the diversified needs of users for different file garbage recovery and management, provides an effective solution for the problem of garbage space recovery in public cloud, provides a beneficial attempt for cloud user operation, and increases the satisfaction of users for system service.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, cloud computing and computer software, and in particular relates to a method for managing junk files in a public cloud based on AI intelligent classification. Background Technology

[0002] With the rapid development of computer networks, big data, and artificial intelligence technologies, people's demand for data storage is constantly increasing, and the problems in data management are becoming increasingly prominent. These problems can be summarized in two aspects: First, because a large amount of user data is stored in the cloud, users' file management has gradually evolved into a demand for different levels of storage management and data security. Taking the management of junk files as an example, just like garbage sorting in daily life, people put different types of garbage into different trash cans according to requirements. Obviously, a trash can on the user's operating system desktop or in the cloud is no longer sufficient for users to classify and manage junk files. Second, because the physical storage of cloud files stored by users at different times overlaps and is updated in data centers, some long-unused data files, or those falling between junk files and expired files, bring a huge storage burden and management difficulties to data migration.

[0003] To more clearly explain the technical problem to be solved, we first define the concept of junk files: Junk files refer to expired, redundant, and no longer-used files generated during the operation of a computer operating system or by a user. In general operating systems, such as personal operating systems, the space occupied by junk files is cleaned up according to the user's needs. For example, just like people cleaning up their own household garbage, each person can clean it up regularly. Even if junk files are not cleaned up, they occupy personal storage space, and as long as they do not affect the user's personal use, junk files will not harm the system. Secondly, again using the example of garbage disposal in daily life, if household garbage is not disposed of in a timely manner, the biggest harm is only the inconvenience caused to the user; however, if garbage in public garbage stations is not disposed of in a timely manner, it will seriously affect people's daily lives. Obviously, if too many junk files accumulate in public cloud space and are not processed in a timely manner, it will affect the data storage of the public cloud. In addition, current operating systems generally only provide one garbage collection bin. Obviously, with the increasing demand for data and the frequency of use, the management of junk files is becoming increasingly complex and diverse, which is very similar to the real-world household waste sorting.

[0004] Public clouds are generally limited network space provided by third-party providers to computer users for storing personal data. Users access the cloud computing center via the internet or a local area network (LAN) to store, process, and manage their personal data. Public clouds are typically offered free of charge or at very low prices, and their fundamental characteristic is a public, shared resource service. From the perspective of resource optimization and social development, public clouds significantly reduce social costs and provide substantial economic and social value. For example, if each person needs 1TB of storage space, but users only utilize 70% of that, leaving the remaining space unused, this waste of social resources is severe, especially for a large number of computer users. If users store their data in the cloud, not only do they save social costs, but the efficient sharing and optimized allocation of resources, including the power consumption of computer storage devices, also contribute significantly to social development, resulting in substantial economic and social value.

[0005] With the increasing popularity of public clouds, more and more organizations and individuals are choosing to migrate their data and applications to cloud platforms. However, due to a combination of factors, including frequent file operations by users, poor management of data residue and expired files, and the failure to promptly clean up temporary cache files generated by applications and systems, a large number of junk files are generated in public cloud environments. These junk files consume valuable storage resources, reduce system performance and response speed, making the cleanup of junk files in public clouds an urgent matter. However, detecting junk files in public clouds requires collecting file features for detection. Most methods use the raw feature set for detection, but these features contain a large amount of redundant or irrelevant information, resulting in excessively high feature dimensionality, increasing computational and storage complexity, and reducing system detection efficiency. Secondly, treating features as independent variables ignores the correlation between features, making it impossible to fully extract information from the feature set, affecting the accuracy of junk file detection. Using Principal Component Analysis (PCA) can reduce the dimensionality of junk file features, and by sorting principal components according to the variance of features, it can better reflect the characteristics of junk files and improve the efficiency of processing cloud junk files. For example: Example 1: A company used an internal public cloud storage system for data ten years ago. With the explosive growth of business and data volume, the required storage space became severely insufficient. Since the company's data from several years ago is part of its big data analytics, it needs to be retained for a long time. However, migrating this data is difficult, and it is taking up a large amount of available storage space.

[0006] Example 2: A medical research institution stores medical data in a public cloud, including patient CT images, clinical medical research records, and test data. However, due to long-term data backup and storage, a large number of expired files exist in these data. These files occupy a lot of valuable storage resources, increase the cost of storage and file management, and also suffer from slow retrieval speed and low management efficiency.

[0007] Example 3: A company has established an internal public cloud and an external service public cloud. The internal public cloud is mainly used by the company's employees for internal affairs management and inter-departmental services. The external service public cloud provides online space for service customers across regions and offers cloud services such as purchasing, online support, after-sales service, and upgrades. In order to save storage space on the external service public cloud, it is necessary to conduct statistical analysis on the structure of user files to provide better online service support. This analysis revealed a large number of expired and long-unused file types and data files. Therefore, the company's computer management department provides users with effective suggestions to help them improve the efficiency of their personal space usage.

[0008] Based on the above technical analysis, there are currently problems with excessive and large junk files in public clouds, which encroach on users' limited and effective network cloud space. In addition, the expired storage and complex management of junk files pose a huge challenge to the effective use of cloud space. Summary of the Invention

[0009] (a) Technical problems to be solved The purpose of this invention is to propose a method for managing junk files in public clouds based on AI intelligent classification, which improves the efficiency of file management in public clouds and makes the operation more user-friendly. The main technical issues are described below: The first technical issue: Currently, there is no systematic and standardized method for classifying junk files in public clouds. The issue of classifying junk files is still at the operating system level. However, with the continuous maturation and development of computer networks and cloud computing technologies, it is necessary to further standardize and manage the characteristics and features of junk files, just like sorting garbage in people's daily lives. The second technical problem: junk files are taking up a large amount of users' limited and effective network cloud space. Personal cloud users typically have limited network space. Therefore, effectively managing files is a complex problem that cloud service providers need to address. Solving this technical problem effectively can not only provide users with user-friendly file management services, but also enhance the technical service level of enterprises and improve service quality. The third technical issue: In cloud storage management systems, a single garbage collection point can no longer meet the diverse needs of current storage requirements. The single garbage collection bin in some operating systems can no longer meet the increasingly diverse needs of users for file processing. How to achieve garbage file classification management and establish a complete cloud space garbage collection system is a key issue to effectively improve cloud computing performance and data management efficiency.

[0010] (II) Technical Solution To address the aforementioned technical problems, this invention provides a method for managing junk files in a public cloud based on AI intelligent classification. The main technical approach includes: 1. To address the lack of a systematic and standardized method for classifying junk files in public clouds, this invention proposes a method for junk file classification standards. The technical indicators used mainly include file storage time, number of file backups, cumulative number of file modifications, cumulative number of file shares, and cumulative number of file reads as parameters, providing cloud service providers with a method for classifying and processing junk files. 2. To address the problem of junk files consuming users' limited and effective cloud storage space, this invention provides a software service solution for establishing a public junk recycling station for cloud service provider data management centers; this not only saves users' effective storage space but also greatly reduces the management overhead of stored files. 3. To address the issue of a single waste recycling station in the user interface, this invention provides a diversified method for waste sorting, recycling, and processing, enabling a user-friendly waste recycling management process.

[0011] To further illustrate the technical solution of the present invention, the implementation steps are described below: Part 1: The cloud computing data management center initializes system parameters and establishes information on the Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) systems belonging to the cloud service provider; it initializes the storage space and interfaces of the public garbage collection station system; it establishes a training set for garbage file classification, establishes a garbage file classification algorithm on the public cloud server side, and initializes the parameters and coefficients of the public garbage collection station; it establishes the public cloud garbage file public storage personal space management part and initializes the public cloud garbage file public storage personal space parameters; it establishes the client-side garbage classification and recycling station and initializes the user-side parameters. Part 2: Execute the server-side garbage file classification algorithm and build a list of garbage files; Part 3: The server-side public garbage collection station periodically recycles junk files to free up storage space; Part 4: Establish a waste recycling sorter on the public cloud user end.

[0012] The data structure and basic process definitions for the AI-based intelligent classification method for managing junk files in public clouds are as follows: (1) A junk file classifier, used on the server side to manage junk files in cloud files, specifically including: Data item 1, storage time of cloud files, denoted as: CloudFile_Time Data item 2, the number of cloud file backups, denoted as: CloudFile_NumBack Data item 3, the cumulative number of modifications to the cloud file, denoted as: CloudFile_NumEdit Data item 4, the cumulative number of times a cloud file has been shared, denoted as: CloudFile_NumShare Data item 5, the cumulative number of times the cloud file has been read, denoted as: CloudFile_NumRead Data item 6, a custom parameter array for the cloud junk file classifier, denoted as: CloudFile_ParaArr Data item 7, the cloud junk file classifier user parameter array, denoted as: CloudFile_UserArr (2) The public garbage collection data space of the server-side cloud service provider's data management center is defined as follows: Data item 1, data storage unit capacity, denoted as: CloudServer_UnitSave Data item 2, data storage area code range, denoted as: CloudServer_AreaRange Data item 3, junk file backup flag, denoted as: CloudServer_TagFileBack Data item 4, category garbage storage tag, denoted as: CloudServer_TagTypeFile Data item 5, periodically cleaned up, denoted as: CloudServer_ClearByTime (3) The user-side waste recycling and sorting collection is defined as follows: Data item 1, the number of elements in the user's garbage recycling collection set, denoted as: CloudUser_NumSet Data item 2, the user-side garbage collection classification array, denoted as: CloudUser_ArrTypeFile Data item 3, the user-side garbage recycling classifier tag array, denoted as: CloudUser_TagArrFile Data item 4, user-defined parameter array for the user-side garbage recycling classifier, denoted as: CloudUser_forUserArr Data item 5, a user-defined tag array for the user-side garbage recycling sorter, denoted as: CloudUser_forTagArr (4) File metadata, used to store various characteristic information of the file, specifically including: Data item 1, the specific name of the file, denoted as: File_Name Data item 2, file type, used to distinguish different file types, denoted as: File_Type Data item 3, the size of the cloud storage space occupied by the file, denoted as: File_Size Data item 4, the system time of file creation, denoted as: File_CreateDate Data item 5, the last modification time of the file, denoted as: File_ModifyDate Data item 6, the file owner, denoted as: File_Owner Data item 7, file access permissions, denoted as: File_Authority (5) The garbage file feature hash table FHTable is used for fast detection of garbage files and is defined as follows: Data item 1, the hash value of the main feature of the garbage file, denoted as: Hash_MainFeature Data item 2, the main characteristic identifier of junk files, denoted as: File_Identifier The procedures and functions are defined as follows: (1) Σ(Value_Sum) represents the summation operation on the variable Value_Sum, which in the specific operation represents a set, array, vector, etc.; (2) sqrt(Value_sqrt) means to perform the square root operation on the variable Value_sqrt. In the specific operation, the variable Value_sqrt represents a real number, an integer, an evaluation expression, etc. (3) (Value_exNum1) Value_exNum2 This means performing the exponentiation operation of Value_exNum2 on the variable Value_exNum1, that is, multiplying Value_exNum2 by Value_exNum1; (4) (Value_Vector) TThis represents the transpose of the variable Value_Vector, which can be represented as a matrix, vector, etc. The data structure definition and basic procedure definition are now complete.

[0013] The specific process of Part 1 of the method for managing junk files in public clouds based on AI intelligent classification is described as follows: The main functions of this section are as follows: First, establish the interfaces and parameter lists for the IaaS and PaaS provided by the cloud service provider; second, establish and initialize the storage space and interfaces for the public garbage collection system; establish a garbage file classification algorithm on the public cloud server side and initialize the parameters and coefficients of the public garbage collection system; initialize the garbage file classification training set AITrainingSet; third, establish the public cloud garbage file public storage personal space management part and initialize the public cloud garbage file public storage personal space parameters; finally, establish the client-side garbage classification and recycling station and initialize the user-side parameters based on the default parameters of the training set. The specific content is described below: Procedure_1: Since cloud service providers generally use IaaS and PaaS architectures to build their underlying structures, this invention provides a unified interface to support IaaS and PaaS architecture interfaces. The specific process is as follows: Procedure_1_FunctionIaaSInterface(Input parameter array IaaS_InArr, Output parameter array IaaS_OutArr) is a function that provides support for the IaaS structure. The input parameter array IaaS_InArr implements the parameters input from the IaaS to this system, and the output parameter array IaaS_OutArr implements the parameters from this system to the underlying IaaS. Procedure_1_FunctionPaaSInterface(Input parameter array PaaS_InArr, Output parameter array PaaS_OutArr) is a function that provides support for the PaaS structure. The input parameter array PaaS_InArr implements the parameters input from PaaS to this system, and the output parameter array PaaS_OutArr implements the parameters from this system to the underlying PaaS.

[0014] To establish storage space for the public garbage collection system, this system uses a designated portion of the underlying storage system as a buffer for storing and managing garbage files. This is implemented by Procedure_1_FunctionSerBuff, and the specific process is as follows: The Procedure_1_FunctionIaaSInterface and Procedure_1_FunctionPaaSInterface processes allocate a specified storage space in the underlying storage area and initialize CloudServer_UnitSave to format the data storage unit capacity. This is done to facilitate the unified handling of junk files during migration. Secondly, the CloudServer_AreaRange value is specified; the data storage area range will be specified by the server's data management center. A larger range results in a larger buffer, which improves the effective management of data space. Finally, CloudServer_TagFileBack is initialized to back up junk files. Unified management of junk files allows for the removal of redundant backups before the buffer is full, ensuring efficient use of buffer space. If a user reclaims a junk file, they can choose to back it up again without affecting their use of the file. Initializing `CloudServer_TagTypeFile` to tag categorized junk files allows for their registration in the buffer, facilitating the categorization and management of junk files in cloud computing data centers. Initializing `CloudServer_ClearByTime` enables the periodic cleanup of junk file content. For example, files in the junk bin of frozen users who have not paid their cloud bills on time will be periodically cleaned up, and the cloud space of deactivated users will be reclaimed, and the contents of their junk bin will be uniformly emptied.

[0015] Procedure_1_FunctionSer2UserBufList establishes a public buffer list for junk files in the public cloud. This process involves migrating junk file collection content from users' personal spaces to the system buffer. The purpose of this is to save user storage space, facilitate unified system junk file collection, and optimize system resources to improve the efficiency of cloud space sharing. Within the established user-specific junk file buffer, CloudUser_NumSet is initialized to record the number of elements in the user-side junk file collection category. Based on the training parameters, CloudUser_ArrTypeFile is initialized to allocate storage space for the user-side junk file collection category array, and CloudUser_TagArr is initialized. The `File` function allocates storage space for the user-side garbage collection classifier's tag array. `CloudUser_forUserArr` allocates storage space for the user-side garbage collection classifier's user-defined parameter array, and `CloudUser_forTagArr` allocates storage space for the user-side garbage collection classifier's user-defined tag array. Although allocating space for garbage files in user space increases the storage overhead of the cloud computing data center, it facilitates regular garbage file recycling and increases the effective use of user space, thereby increasing user satisfaction and providing more effective services to cloud computing users. Compared to the gradually decreasing price per unit of storage space, this benefits cloud computing companies in expanding their software services and increasing the capacity of their cloud computing center storage systems. Part 1 is now complete.

[0016] The specific process of Part 2 of the method for managing junk files in public clouds based on AI intelligent classification is described as follows: The main function of this section is to execute the server-side garbage file classification algorithm and build a list of garbage files. The initial feature set of the file is initialized as follows: Set_IF={S_if_val_1, S_if_val_2, ..., S_if_val_index Set_IF , …, S_if_val_LenNum Set_IF} The elements in the set Set_IF represent initial file characteristics such as "File_Name" and "File_Size", where S_if_val_index... Set_IF Represents the index in the set Set_IF elements, LenNum Set_IF This represents the total number of elements in the set Set_IF.

[0017] The document standardization feature set is initialized as follows: Set_UF={S_uf_val_1, S_uf_val_2, ..., S_uf_val_index Set_UF , …, S_uf_val_LenNum Set_UF} The elements in the set Set_UF represent file standardization characteristics such as "lowercase File_Name without special characters" and "normalized File_Size". Among them, S_uf_val_index... Set_UF Represents the index in the set Set_UF elements, LenNum Set_UF This represents the total number of elements in the set Set_UF.

[0018] The elements in the Set_UF set form the initial feature matrix of garbage files, which is used for dimensionality reduction of file features. The initial feature matrix of garbage files is initialized as follows: Matrix_IF={M_if_val_1, M_if_val_2, …, M_if_val_index Matrix_IF , …, M_if_val_LenNum Matrix_IF} Among them, element M_if_val_index Matrix_IF Indicates the index Matrix_IF A Num_FileSample*1-dimensional feature vector, where Num_FileSample represents the number of junk file samples, and LenNum Matrix_IF This represents the total number of eigenvectors in the matrix Matrix_IF. The size of the Matrix_IF matrix is ​​Num_FileSample * LenNum. Matrix_IF .

[0019] After Z-score normalization of all elements in the Matrix_IF matrix, the garbage file normalized feature matrix is ​​formed as follows: Matrix_UF={M_uf_val_1, M_uf_val_2, …, M_uf_val_index Matrix_UF , …, M_uf_val_LenNum Matrix_UF} Among them, element M_uf_val_index Matrix_UF Indicates the index Matrix_UF Num_FileSample*1-dimensional feature vectors, LenNum Matrix_UFThis represents the total number of eigenvectors in the matrix Matrix_UF. The Matrix_UF matrix and the Matrix_IF matrix have the same size.

[0020] The set of principal features for junk file features, after dimensionality reduction, is initialized as follows: Set_MF={S_mf_val_1, S_mf_val_2, ..., S_mf_val_index Set_MF , …, S_mf_val_LenNum Set_MF} The elements in the set Set_MF represent the features after dimensionality reduction, such as "Main Feature 1", "Main Feature 2", etc., where S_mf_val_index Set_MF Represents the index in the set Set_MF elements, LenNum Set_MF This represents the total number of elements in the Set_MF collection.

[0021] Procedure 2: To effectively identify junk files, relevant features are extracted from the FileMetadata of similar junk files and standardized into a Set_IF set. A Matrix_IF matrix is ​​constructed and standardized using the Z-score method to obtain the Matrix_UF matrix, which is used for subsequent file feature dimensionality reduction. The specific process is as follows: The Procedure_2_FunctionUFFeature(input parameter UFF_InParm, output parameter UFF_OutParm) function extracts features from junk files and performs feature standardization. The input parameter UFF_InParm represents the FileMetadata of the junk file, and the output parameter UFF_OutParm is a Set_UF. The specific implementation process is described as follows: Step 1: Extract relevant file features such as File_Name, File_Type, File_Size, and File_CreateDate from the FileMetadata of garbage files; Step 1_2: Add the extracted feature information to the Set_IF set in sequence; Step 1_3: Traverse the Set_IF collection and standardize each element in the Set_IF collection one by one; Step 1_3_1: Convert the File_Name, File_Type, File_Owner, and File_Authority feature information to lowercase and remove punctuation marks, spaces, and other special characters; Step 1_3_2: Convert File_CreateDate and File_ModifyDate to a unified timestamp standard format; Step 1_3_3: Use the Min-Max Scaling method to normalize the File_Size, File_CreateDate, and File_ModifyDate features. Eq_ScaleValue represents the scaled feature value, ranging from 0 to 1. The calculation formula is as follows: Eq_ScaleValue=(Size_One − min(Size_All)) / (max(Size_All) − min(Size_All)) Where Size_One represents the feature value to be scaled, Size_All is the set of all feature values ​​of the same type, min(Size_All) means to get the minimum value in Size_All, and max(Size_All) means to get the maximum value in Size_All. Step 1_4: Add the standardized file feature information to the Set_UF set in sequence; Step 1_5: Return to UFF_OutParm; Step 1_6: The function Procedure_2_FunctionUFFeature ends.

[0022] The `Procedure_2_FunctionBFMatrix(Input parameter BFM_InParm, Output parameter BFM_OutParm)` function constructs a `Matrix_IF` matrix and performs normalization. The input parameter `BFM_InParm` is a `Set_UF` set, and the output parameter `BFM_OutParm` is a `Matrix_UF` matrix. The specific implementation process is described as follows: Step 2_1: Construct the Matrix_IF matrix using BFM_InParm; Step 2: Perform one-hot encoding on the feature values ​​of File_Name, File_Type, File_Owner, and File_Authority in the Matrix_IF matrix; Step 2_3: Standardize the Matrix_IF matrix using the Z-score method; Step 2_3_1: Calculate the mean and standard deviation of each column of the Matrix_IF matrix. Column_avg[m_col] calculates the mean of the m_col-th column, and Sigma[m_col] calculates the standard deviation of the m_col-th column. m_col represents a column of the Matrix_IF matrix. The specific calculation formula is as follows: Column_avg[m_col]=(1 / Num_FileSample)*Σ(Matrix_IF[m_row,m_col]) Sigma[m_col]=sqrt((1 / Num_FileSample)*Σ((Matrix_IF[m_row,m_col]-Column_avg[m_col]) 2 )) Where m_row represents the row of the Matrix_IF matrix, and the value of m_row ranges from 1 to Num_FileSample, and the value of m_col ranges from 1 to LenNum Matrix_IF between; Step 2_3_2: Subtract the mean of the column containing the element from the value of each element in the Matrix_IF matrix, and then divide the result by the standard deviation of the column containing the element to obtain the standardized value. This eliminates the influence between different features and data sizes. Eq_StandardValue represents the standardized value of the element in the Matrix_IF matrix, and the calculation formula is as follows: Eq_StandardValue=(Matrix_IF[m_row, m_col] - Column_avg[m_col]) / Sigma[m_col] Step 2_4: Construct a Matrix_UF matrix with the same size as the Matrix_IF matrix; Step 2_5: Add the Eq_StandardValue data one by one to the Matrix_UF matrix; Step 2_6: Return to BFM_OutParm; Step 2_7: The function Procedure_2_FunctionBFMatrix ends.

[0023] To increase processing efficiency, principal component analysis (PCA) is used to reduce the dimensionality of the Matrix_UF matrix. The covariance matrix of the Matrix_UF matrix is ​​then calculated, and eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. A projection matrix is ​​constructed by selecting the desired number of eigenvectors, and the Set_UF set is then projected through this matrix to obtain the Set_MF set. The specific process is as follows: Procedure_2_FunctionPCAFeature (input parameters include PCAF_InParm_1 and PCAF_InParm_2, output parameter PCAF_OutParm) is a function that performs dimensionality reduction of a Matrix_UF matrix and constructs a Set_MF set. The input parameters PCAF_InParm_1 and PCAF_InParm_2 are the Matrix_UF matrix and the desired dimensionality after reduction, respectively. The output parameter PCAF_OutParm represents the Set_MF set. The specific implementation process is described as follows: Step 3_1: Calculate the covariance matrix of the Matrix_UF matrix; Eq_CovarianceMatrix represents the covariance matrix, and the specific calculation formula is as follows: Eq_CovarianceMatrix = (Matrix_UF – mean(Matrix_UF)) * (Matrix_UF –mean(Matrix_UF)) T / ( Num_FileSample – 1) Where mean(Matrix_UF) represents the mean matrix of the Matrix_UF matrix; Step 3_2: Perform eigenvalue decomposition on Eq_CovarianceMatrix to obtain eigenvalues ​​and corresponding eigenvectors; the specific formula for eigenvalue decomposition is as follows: Eq_CovarianceMatrix=Matrix_Eigenvector*Matrix_Diagonal*(Matrix_Eigenvector) T Among them, Matrix_Eigenvector is a matrix composed of the eigenvectors of Eq_CovarianceMatrix, and Matrix_Diagonal is a diagonal matrix whose diagonal elements are the eigenvalues ​​of Eq_CovarianceMatrix. Step 3: Sort the eigenvalues ​​by size and select the eigenvectors corresponding to the first PCAF_InParm_2 eigenvalues. Step 3_4: Combine the selected PCAF_InParm_2 eigenvectors into a projection matrix Matrix_Project; Step 3_5: Project the Set_UF set onto a lower-dimensional space to obtain the Set_MF set; the calculation formula for the projection operation is: Set_MF=(Matrix_Project) T * Set_UF Step 3_6: Return PCAF_OutParm; Step 3_7: The function Procedure_2_FunctionPCAFeature ends.

[0024] To achieve effective file management, hash functions such as MD5 or SHA1 are used to hash the main features in the Set_MF set, mapping them to unique hash values, and an FHTable hash table is constructed to store these feature hash values. The feature hash value of the file to be detected is calculated, and the FHTable hash table is used to detect junk files in the cloud. A list of junk files is then returned. The specific process is as follows: The `Procedure_2_FunctionCHTable(input parameter CHT_InParm, output parameter CHT_OutParm)` function constructs an `FHTable` hash table. The input parameter `CHT_InParm` represents the `Set_MF` set, and the output parameter `CHT_OutParm` is the `FHTable` hash table. The specific implementation process is described as follows: Step 4_1: Traverse the Set_MF set and obtain the S_mf_val_index of the set. Set_MF element; Step 4_2, for the S_mf_val_index Set_MF Each element is subjected to a hash algorithm to obtain the feature hash value Hash_MainFeature; Step 4_3: Based on the garbage file sample data, iterate through the File_Identifier corresponding to Hash_MainFeature in the Set_MF set; Step 4: Create an FHTable hash table; Step 4-5: Add Hash_MainFeature as the key and the corresponding File_Identifier as the value to the FHTable hash table; Step 4_6: If there are still elements in the Set_MF set whose Hash_MainFeature and File_Identifier have not been added to the FHTable hash table, then go to Step 4_2; otherwise, go to Step 4_7. Step 4_7: Return CHT_OutParm; Step 4_8: The function Procedure_2_FunctionCHTable ends.

[0025] The `Procedure_2_FunctionFDetection(Input parameter FD_InParm, Output parameter FD_OutParm)` function determines whether a file to be detected is a junk file. The input parameter `FD_InParm` represents the file to be detected in the public cloud, and the output parameter `FD_OutParm` is a list of junk files. The specific implementation process is described as follows: Step 5_1: Use the API of the object storage service provided by the public cloud to obtain the FileMetadata of FD_InParm; Step 5_2: Construct the Set_IF set of FD_InParm; Step 5_3: Extract the relevant feature information from the FD_InParm file and store it in the Set_IF set; Step 5_4: Traverse the Set_IF set, for the S_if_val_index... Set_IF The feature is applied using a hash algorithm to obtain the feature hash value Hash_TestFile of the file to be detected; Step 5: Traverse the FHTable hash table and check if Hash_TestFile exists. Step 5_5_1: If it exists, then FD_InParm has similar characteristics to a certain type of junk file, and it can be determined that FD_InParm is a junk file; Step 5_5_1_1: Insert a record into the junk file list to store FD_InParm information, including file information such as File_Name, File_Type, and File_Size; if there are still files to be detected in the public cloud, go to Step 5_1 to continue the detection; otherwise, go to Step 5_6. Step 5_5_2: If it does not exist, FD_InParm determines it to be a non-junk file; if there are still files to be detected in the public cloud, go to Step 5_1 to continue detecting the next file; otherwise, go to Step 5_6. Step 5-6: Return to FD_OutParm; Step 5_7: The function Procedure_2_FunctionFDetection ends; Part 2 is now complete.

[0026] The detailed process of Part 3 of the method for managing junk files in public clouds based on AI intelligent classification is described as follows: The main function of this section is: the server-side public garbage collection station periodically recycles junk files to free up storage space; Initialization vector P_cff={P_cff_val_1, P_cff_val_2, …, P_cff_val_LenNum p_cff}, where P_cff_val_1 is the first parameter of the vector P_cff, and P_cff_val_LenNum p_cff It's the last parameter, LenNum p_cff Σ(P_cff) = 1 is the total number of parameters in the vector P_cff. The sum of all parameters in P_cff is 1. Each parameter of P_cff has a value between 0 and 1.

[0027] Eq_ClearFileFactor represents the recycling factor, calculated using the following formula: Eq_ClearFileFactor = P_cff_val_1*CloudFile_Time - P_cff_val_2*CloudFile_NumBack + P_cff_val_3* CloudFile_NumEdit +P_cff_val_4*CloudFile_NumShare + P_cff_val_5*CloudFile_NumRead; Procedure_3 executes the process of periodically reclaiming junk files from the server-side public garbage collection station, specifically described as follows: The server updates the P_cff parameter value periodically according to the agreed time and the optimized configuration of the training set. It clears the junk files of deactivated cloud users in the public garbage collection system storage space. When the public garbage collection system storage space has reached 80% of its capacity, it clears the backup files of the junk files of frozen cloud users. The server administrator sets the recycling factor threshold to ValMax_ClearFileFactor, calculates the Eq_ClearFileFactor value of the files in the garbage classifier, and clears the junk file from the buffer when Eq_ClearFileFactor > ValMax_ClearFileFactor. Part 3 is now complete.

[0028] The specific process of Part 4 of the method for managing junk files in public clouds based on AI intelligent classification is described as follows: The main function of this section is to enable public cloud users to establish a waste recycling sorter; Procedure_4 executes the server-side push to the client-side user-defined CloudUser_forUserArr and CloudUser_forTagArr, and establishes the user-side garbage recycling classifier and user-defined classifier tag content in the user's cloud. After logging into the cloud, the user can modify their own garbage recycling classification tag and set their own garbage sorting and recycling station. Part 4 is now complete.

[0029] (III) Beneficial Effects The beneficial effects of this invention are reflected in: 1. This invention establishes a preliminary standard for classifying junk files in public clouds. By using this standard to classify and process junk files in public clouds in different ways, it can respond to the diverse needs of users for cloud file usage. By classifying and processing junk files that appear in the system through this standard, it can meet the different usage needs of users.

[0030] 2. This invention establishes a public garbage bin buffer in the public cloud, which manages the files in the garbage bin in a unified manner, eliminates redundant backups and multi-user copies, and saves a lot of cloud space; it not only frees up storage space for users' personal space, but also facilitates the management of files controlled by the server, making the classification and management of garbage files simpler, more convenient and faster.

[0031] 3. This invention designs a diverse garbage sorting and recycling system that is user-friendly, fulfilling users' diverse needs for garbage recycling and management of different files. It provides an effective solution to the persistent problem of difficult garbage space recycling in public clouds. At the same time, it offers a beneficial approach to cloud-based user operations, increasing user satisfaction with the system services. Attached Figure Description

[0032] Figure 1 This is a structural diagram of a public cloud-based method for managing junk files based on AI-powered intelligent classification. Detailed Implementation

[0033] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0034] Example 1: System initialization process Procedure_1 executes the initialization of the underlying architecture of the cloud service provider's IaaS and PaaS. Assuming Company A has chosen cloud service provider B's IaaS as its underlying infrastructure, Procedure_1_FunctionIaaSInterface is executed to create an input parameter array IaaS_InArr that supports the IaaS structure, implementing the parameters input from the IaaS to this system, and an output parameter array IaaS_OutArr that implements the parameters from this system to the IaaS underlying layer. AITrainingSet is then initialized.

[0035] Secondly, the `Procedure_1_FunctionSerBuff` function establishes a shared garbage collection system storage space, specifying a portion of the IaaS system storage space as a buffer for managing garbage files. The specific process is as follows: Initialize `CloudServer_UnitSave=1T` to format the data storage unit capacity. In storage systems using disk arrays, each array is a 1T disk. When expansion is needed, a larger `CloudServer_UnitSave`, such as 10T, can be used for data migration. Next, set `CloudServer_AreaRange` to 0-1000, using approximately 1000T / `CloudServer_UnitSave` of storage space as a buffer. Establish `CloudServer_TagFileBack` to manage garbage file backup tags, and `CloudServer_TagTypeFile` to register and manage categorized garbage storage tags. Initialize `CloudServer_ClearByTime=2 hours` to periodically clean up garbage file content.

[0036] Third, the Procedure_1_FunctionSer2UserBufList is executed to establish the public cloud junk file public storage personal buffer list; in the established user personal junk file buffer, CloudUser_NumSet is initialized to record the number of elements in the user-side junk recycling classification collection, CloudUser_ArrTypeFile is initialized to allocate storage space for the user-side junk recycling classification array, CloudUser_TagArrFile is initialized to allocate storage space for the user-side junk recycling classifier tag array, CloudUser_forUserArr is initialized to allocate storage space for the user-side junk recycling classifier user-defined parameter array, and CloudUser_forTagArr is initialized to allocate storage space for the user-side junk recycling classifier user-defined tag array.

[0037] Example 2: Executing a server-side junk file classification algorithm and building a junk file list. Procedure_2 executes the server-side garbage file classification algorithm and establishes a list of garbage files; First, Procedure_2_FunctionUFFeature is executed to extract and standardize the features of junk files. This effectively identifies junk files. Assuming the server stores junk files of three cache types, features are extracted from the FileMetadata of the junk files and added to their respective Set_IF sets. The Set_IF sets for each cache type of junk file are as follows: File 1: Set_IF={gamecache, temp, 1024, 2010-06-16T12:30:00, 2010-09-16T13:30:00, user1, read} File 2: Set_IF={photocache, temp, 2048, 2021-06-20T20:30:00, 2021-07-12T21:00:00, user2, write} File 3: Set_IF={vidocache,log,512,2011-01-12T08:20:00,2011-06-26T03:30:00,user3,execute} For each file, the feature information in the Set_IF set is converted to lowercase, and punctuation marks, spaces, and other special characters are removed. File_CreateDate and File_ModifyDate are converted to a unified timestamp standard format. Then, the File_Size, File_CreateDate, and File_ModifyDate features are normalized using a min-max scaling method. This aims to eliminate differences between different file data, ensuring feature consistency and further improving data quality. The normalized file feature information is then added sequentially to the Set_UF set for each file, resulting in the following Set_UF set for each cached type of garbage file: File 1: Set_UF={gamecache, temp, 0.5, 0, 0, user1, read} File 2: Set_UF={photocache, temp, 1, 1, 1, user2, write} File 3: Set_UF={vidocache, log, 0, 0.144, 0.094, user3, execute} Next, Procedure_2_FunctionBFMatrix is ​​executed to construct the Matrix_IF matrix and standardize it using the Z-score method. This ensures that each element has the same scale and importance, making the data comparable across the entire dataset for subsequent PCA dimensionality reduction. The specific process is as follows: The Matrix_IF matrix is ​​constructed using the Set_UF set of garbage files for each cache type, resulting in Matrix_IF={(gamecache, photocache, vidocache), (temp, temp, log), (0.5, 1, 0), (0, 1, 0.144), (0, 1, 0.094), (user1, user2, user3), (read, write, Next, one-hot encoding is performed on the string-type eigenvalues ​​in the Matrix_IF matrix. Then, Column_avg[m_col] and Sigma[m_col] of the Matrix_IF matrix are calculated. The value of each element is subtracted from the Column_avg[m_col] of the column containing that element, and the result is divided by the Sigma[m_col] of the column containing that element to obtain Eq_StandardValue. The values ​​of ue are added sequentially to the Matrix_UF matrix, resulting in Matrix_UF={(-1.5,0.75,2.25),(0.091,-0.091,-0.091),(0,2.25,-0.75),(-3,6,-1.712),(-4.583,8.333,-2.785),(-1.665,-1.665,4.5),(-1.665,-1.665,4.5)}, thus eliminating the influence between different features and data sizes.

[0038] Then, Procedure_2_FunctionPCAFeature is executed to reduce the dimensionality of the Matrix_UF matrix using PCA and to construct the Set_MF set, which increases processing efficiency. The specific process is as follows: Calculate the covariance matrix Eq_CovarianceMatrix of the Matrix_UF matrix, then perform eigenvalue decomposition on Eq_CovarianceMatrix to obtain eigenvalues ​​52.471, 12.398, 4.705, 1.391, 0.025, 0, 0 and their corresponding eigenvectors. Then, sort the eigenvalues ​​by size, and if the eigenvectors corresponding to the first two eigenvalues ​​are selected, form the projection matrix Matrix_Project. Project the Set_UF set onto the low-dimensional space to obtain Set_MF={gamecache,vidocache,photocache,temp,log}.

[0039] Finally, Procedure_2_FunctionCHTable is executed to construct the FHTable hash table. This is done to achieve efficient file management. The specific process is as follows: Suppose the MD5 hash algorithm is applied to the temp element in the Set_MF set to obtain the feature hash value Hash_MainFeature=32c1cec3ee82d87a. Hash_MainFeature=32c1cec3ee82d87a is used as the key and the corresponding File_Identifier=temp is used as the value, and then added to the FHTable hash table. Then, Procedure_2_FunctionFDetection is executed to determine whether the file to be detected is a junk file. The specific process is as follows: Assume that the file to be detected, Set_IF={filecache, temp, 4096, 2015-02-12T03:20:00, 2020-05-10T05:35:00, user4, read}, is obtained from the FileMetadata of the file to be detected in the Set_IF set. When the MD5 hash algorithm is applied to the temp feature of the file to be detected, the hash value Hash_TestFile=32c1cec3ee82d87a is obtained. It is found that Hash_TestFile exists in the FHTable hash table. Therefore, the file has similar characteristics to the cache type junk file and can be judged as a junk file. A record is inserted into the junk file list to store the file information such as filecache, temp, and 4096.

[0040] Typical junk files and those with high frequency of current junk files are marked in AITrainingSet, and the P_cff parameter is adjusted.

[0041] Example 3: Setting up a waste recycling sorter on a public cloud client The server executes Procedure_4 to push user-defined CloudUser_forUserArr and CloudUser_forTagArr to the client, establishing a user-side garbage collection classifier and user-defined classifier tagging content in the user's cloud. For example, if the client CloudUser_IDTest has created a video garbage bin RubbishStation_Media and a personal file garbage bin RubbishStation_Myfile; and RubbishStation_Media is marked as being managed in a public garbage bin buffer, meaning unwanted videos will be placed in the public garbage bin buffer; and RubbishStation_Myfile is configured with personal space storage, meaning that junk files from the personal file recycle bin will not be stored in the public garbage bin buffer.

[0042] The purpose of this setup is to effectively utilize the user's limited space. Storing too many large video files, such as movie files, would take up limited personal storage space. Therefore, some unimportant large video files can be stored in the public buffer. However, personal files involve some private data, and users do not store them in the public buffer for security reasons. This protects data security while also effectively utilizing limited personal storage space.

[0043] Example 4: Regularly clean up junk files on the server side Procedure_3 executes the process of periodically reclaiming junk files from the server-side public garbage collection station, specifically as follows: The server updates the P_cff parameter value every 2 hours and according to the parameters of the training set; then it clears the junk files of cloud users who have been deactivated from the public junk bin system storage space; when the public junk bin system storage space has reached 80% of its capacity, it continues to clear the backup files of the junk files of frozen cloud users. The server-side administrator sets the recycling factor ValMax_ClearFileFactor=80; Calculate the Eq_ClearFileFactor value of video junk files in the junk file sorter for user CloudUser_IDTest. When Eq_ClearFileFactor > 80, clear the user's video junk files from the buffer and continue clearing files of other users in the junk file sorter, so that the storage space ratio of the public junk bin system is less than 80% of the capacity. Clearly, in Example 4, junk files in the user's personal space were not cleaned up. This not only protects the effective use of the user's personal space, but also ensures that the user can save a certain number of large files in the limited space. These files are stored in the public buffer pool, which does not occupy the user's limited personal space, increases user satisfaction, and improves the service quality of the cloud service provider.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for managing junk files in a public cloud based on AI intelligent classification, characterized in that: Part 1, Cloud Computing Data Management Center initializes system parameters, including: establishing information on the Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) systems belonging to the cloud service provider; initializing the storage space and interfaces of the public garbage collection station system; establishing a training set for garbage file classification; establishing a garbage file classification algorithm on the public cloud server side; initializing the parameters and coefficients of the public garbage collection station; establishing the public cloud garbage file public storage personal space management part; initializing the public cloud garbage file public storage personal space parameters; establishing the client-side garbage classification and recycling station; and initializing the user-side parameters. Part 2: Execute the server-side garbage file classification algorithm and build a list of garbage files; Part 3: The server-side public garbage collection station periodically recycles junk files to free up storage space; Part 4: Establishing a waste recycling sorter on the public cloud user end; The data structure and basic process definitions for the AI-based intelligent classification method for managing junk files in public clouds are as follows: (1) A junk file classifier, used on the server side to manage junk files in cloud files, specifically including: Data item 1, storage time of cloud files, denoted as: CloudFile_Time Data item 2, the number of cloud file backups, denoted as: CloudFile_NumBack Data item 3, the cumulative number of modifications to the cloud file, denoted as: CloudFile_NumEdit Data item 4, the cumulative number of times a cloud file has been shared, denoted as: CloudFile_NumShare Data item 5, the cumulative number of times the cloud file has been read, denoted as: CloudFile_NumRead Data item 6, a custom parameter array for the cloud junk file classifier, denoted as: CloudFile_ParaArr Data item 7, the cloud junk file classifier user parameter array, denoted as: CloudFile_UserArr (2) The public garbage collection data space of the server-side cloud service provider's data management center is defined as follows: Data item 1, data storage unit capacity, denoted as: CloudServer_UnitSave Data item 2, data storage area code range, denoted as: CloudServer_AreaRange Data item 3, junk file backup flag, denoted as: CloudServer_TagFileBack Data item 4, category garbage storage tag, denoted as: CloudServer_TagTypeFile Data item 5, periodically cleaned up, denoted as: CloudServer_ClearByTime (3) The user-side waste recycling and sorting collection is defined as follows: Data item 1, the number of elements in the user's garbage recycling collection set, denoted as: CloudUser_NumSet Data item 2, the user-side garbage collection classification array, denoted as: CloudUser_ArrTypeFile Data item 3, the user-side garbage recycling classifier tag array, denoted as: CloudUser_TagArrFile Data item 4, a user-defined parameter array for the user-side garbage recycling classifier, denoted as: CloudUser_forUserArr Data item 5, a user-defined tag array for the user-side garbage recycling sorter, denoted as: CloudUser_forTagArr (4) File Metadata, used to store file characteristic information, is defined as follows: Data item 1, the specific name of the file, denoted as: File_Name Data item 2, the file type, denoted as: File_Type Data item 3, the size of the cloud storage space occupied by the file, denoted as: File_Size Data item 4, the system time of file creation, denoted as: File_CreateDate Data item 5, the last modification time of the file, denoted as: File_ModifyDate Data item 6, the file owner, denoted as: File_Owner Data item 7, file access permissions, denoted as: File_Authority (5) The garbage file feature hash table FHTable is used to detect garbage files and is defined as follows: Data item 1, the garbage file system feature hash value, denoted as: Hash_MainFeature Data item 2, the garbage file system characteristic identifier, denoted as: File_Identifier The procedures and functions are defined as follows: (1) Σ(Value_sum) represents the summation operation on the variable Value_sum, where Value_sum represents a set, array, or vector in the operation; (2) sqrt(Value_sqrt) means to perform the square root operation on the variable Value_sqrt, where the variable Value_sqrt represents a real number, an integer, or an evaluation expression in the operation; (3) (Value_exNum1) Value_exNum2 This means that the variable Value_exNum1 is multiplied by Value_exNum2, which is equivalent to multiplying Value_exNum2 times by Value_exNum1. (4) (Value_vector) T This represents the transpose of the variable Value_vector, which can be represented as a matrix or vector in operations. The data structure definition and basic procedure definition are complete; The specific process of Part_1 is described as follows: First, establish the interfaces and parameter lists for the IaaS and PaaS provided by the cloud service provider; second, establish the storage space and interfaces for the public garbage collection system and perform initialization operations; establish a garbage file classification algorithm on the public cloud server side and initialize the public garbage collection station parameters and coefficients; initialize the garbage file classification training set AITrainingSet; third, establish the public cloud garbage file public storage personal space management part and initialize the public cloud garbage file public storage personal space parameters; establish a client-side garbage classification and recycling station, and initialize the user-side parameters according to the default parameters of the training set; details are as follows: Procedure_1: Establish the underlying structure based on the IaaS and PaaS architecture adopted by the cloud service provider, and initialize the interfaces of the IaaS and PaaS architectures. The specific process of unifying the interfaces is as follows: Procedure_1_FunctionIaaSInterface(Input parameter array IaaS_InArr, Output parameter array IaaS_OutArr) is a function that provides support for the IaaS structure. The input parameter array IaaS_InArr implements the parameters input from the IaaS to this system, and the output parameter array IaaS_OutArr implements the parameters from this system to the underlying IaaS. Procedure_1_FunctionPaaSInterface(Input parameter array PaaS_InArr, Output parameter array PaaS_OutArr) is a function that provides support for the PaaS structure. The input parameter array PaaS_InArr implements the parameters input from the PaaS to this system, and the output parameter array PaaS_OutArr implements the parameters from this system to the underlying PaaS. A public garbage collection system storage space is established, using a designated area of ​​the underlying storage system as a buffer for garbage file storage management. This is implemented by Procedure_1_FunctionSerBuff. Specifically, the Procedure_1_FunctionIaaSInterface and Procedure_1_FunctionPaaSInterface procedures allocate a designated storage space in the underlying storage area and initialize CloudServer_UnitSave to format the data storage unit capacity. Next, the CloudServer_AreaRange value is specified; the data storage area range will be determined by the server data management system. The management center specifies that CloudServer_TagFileBack is initialized to centrally manage junk file backup tags. Before the buffer is full, junk file backups are cleared. If a user recycles the junk file again, the user can choose to back it up again. CloudServer_TagTypeFile is initialized to register junk files in the buffer by classifying junk storage tags. CloudServer_ClearByTime is initialized to periodically clean up junk file contents. Users who have been frozen due to non-payment in cloud computing will have their files in their junk bins periodically cleaned up. Cloud users who choose to cancel their accounts will have their cloud space reclaimed and their junk bin contents emptied. Procedure_1_FunctionSer2UserBufList establishes a public cloud junk file public storage personal buffer list. In the established user personal junk file buffer, it initializes CloudUser_NumSet to record the number of elements in the user's junk recycling classification collection. Based on the AITrainingSet training parameters, it initializes CloudUser_ArrTypeFile to allocate storage space for the user's junk recycling classification array, initializes CloudUser_TagArrFile to allocate storage space for the user's junk recycling classifier tag array, initializes CloudUser_forUserArr to allocate storage space for the user's junk recycling classifier user-defined parameter array, and initializes CloudUser_forTagArr to allocate storage space for the user's junk recycling classifier user-defined tag array. Part 1 is now complete; The specific process of Part 2 is described as follows: Execute the server-side garbage file classification algorithm to create a list of garbage files; The initial feature set of the file is initialized as follows: Set_IF={S_if_val_1, S_if_val_2, ..., S_if_val_index Set_IF , …, S_if_val_LenNum Set_IF } The elements in the set Set_IF represent the initial characteristics of a file, where S_if_val_index Set_IF Represents the index in the set Set_IF elements, LenNum Set_IF Represents the total number of elements in the set Set_IF; The document standardization feature set is initialized as follows: Set_UF={S_uf_val_1, S_uf_val_2, ..., S_uf_val_index Set_UF , …, S_uf_val_LenNum Set_UF } The elements in the set Set_UF represent file normalization characteristics, where S_uf_val_index Set_UF Represents the index in the set Set_UF elements, LenNum Set_UF This represents the total number of elements in the set Set_UF. The elements in the Set_UF set form the initial feature matrix of garbage files, which performs dimensionality reduction on the file features. The initial feature matrix of garbage files is initialized as follows: Matrix_IF={M_if_val_1, …, M_if_val_index Matrix_IF , …, M_if_val_LenNum Matrix _IF } Among them, element M_if_val_index Matrix_IF Indicates the index Matrix_IF A Num_FileSample*1-dimensional feature vector, where Num_FileSample represents the number of junk file samples, and LenNum Matrix_IF This represents the total number of eigenvectors in the matrix Matrix_IF. The size of the Matrix_IF matrix is ​​Num_FileSample * LenNum. Matrix_IF ; Perform Z-score normalization on all elements of the Matrix_IF matrix to obtain the garbage file normalized feature matrix: Matrix_UF={M_uf_val_1, …, M_uf_val_index Matrix_UF , …, M_uf_val_LenNum Matrix _UF } Among them, element M_uf_val_index Matrix_UF Indicates the index Matrix_UF Num_FileSample*1-dimensional feature vectors, LenNum Matrix_UF This represents the total number of eigenvectors in the matrix Matrix_UF. The Matrix_UF matrix and the Matrix_IF matrix have the same size. The set of principal features for junk file features, after dimensionality reduction, is initialized as follows: Set_MF={S_mf_val_1, S_mf_val_2, ..., S_mf_val_index Set_MF , …, S_mf_val_LenNum Set_MF } The elements in the set Set_MF represent the principal features after dimensionality reduction, where S_mf_val_index Set_MF Represents the index in the set Set_MF elements, LenNum Set_MF Represents the total number of elements in the Set_MF set; Procedure_2: Extract relevant features from the FileMetadata of similar garbage files, form a Set_IF set for standardization; construct a Matrix_IF matrix, standardize it using the Z-score method to obtain a Matrix_UF matrix, and then perform dimensionality reduction processing on the file features. The specific process is as follows: The `Procedure_2_FunctionUFFeature(input parameter UFF_InParm, output parameter UFF_OutParm)` function extracts features from junk files and performs feature standardization. The input parameter `UFF_InParm` represents the `FileMetadata` of the junk file, and the output parameter `UFF_OutParm` is a `Set_UF` set. Step 1: Extract file features such as File_Name, File_Type, File_Size, File_CreateDate, File_Owner, and File_Authority from the FileMetadata of the garbage files; Step 1_2: Add the extracted feature information to the Set_IF set in sequence; Step 1_3: Traverse the Set_IF set and standardize each element in the Set_IF set one by one; Step 1_3_1: Convert the File_Name, File_Type, File_Owner, and File_Authority feature information to lowercase and remove special characters; Step 1_3_2: Convert File_CreateDate and File_ModifyDate to a unified timestamp standard format; Step 1_3_3: Use the Min-Max Scaling method to normalize the File_Size, File_CreateDate, and File_ModifyDate features. Eq_ScaleValue represents the scaled feature value, ranging from 0 to 1. The calculation formula is: Eq_ScaleValue=(Size_One−min(Size_All)) / (max(Size_All)−min(Size_All)), where Size_One represents the feature value to be scaled, Size_All is the set of all feature values ​​of the same type, min(Size_All) means to get the minimum value in Size_All, and max(Size_All) means to get the maximum value in Size_All. Step 1_4: Add the standardized file feature information to the Set_UF set in sequence; Step 1_5: Return to UFF_OutParm; Step 1_6: The function Procedure_2_FunctionUFFeature ends; The `Procedure_2_FunctionBFMatrix(Input parameter BFM_InParm, Output parameter BFM_OutParm)` function constructs a `Matrix_IF` matrix and performs normalization. The input parameter `BFM_InParm` is a `Set_UF` set, and the output parameter `BFM_OutParm` is a `Matrix_UF` matrix. The specific process is as follows: Step 2_1: Construct the Matrix_IF matrix using BFM_InParm; Step 2: Perform One-Hot encoding on the feature values ​​of File_Name, File_Type, File_Owner, and File_Authority in the Matrix_IF matrix; Step 2_3: Standardize the Matrix_IF matrix using the Z-score method; Step 2_3_1: Calculate the mean and standard deviation of each column feature of the Matrix_IF matrix. Column_avg[m_col] calculates the mean of the m_col-th column, and Sigma[m_col] calculates the standard deviation of the m_col-th column. m_col represents the column of the Matrix_IF matrix. The calculation formula is as follows: Column_avg[m_col]=(1 / Num_FileSample)*Σ(Matrix_IF[m_row, m_col]) Sigma[m_col]=sqrt((1 / Num_FileSample)*Σ((Matrix_IF[m_row,m_col]-Column_avg[m_col]) 2 In this context, m_row represents the row of the Matrix_IF matrix, with values ​​ranging from 1 to Num_FileSample, and m_col with values ​​ranging from 1 to LenNum. Matrix_IF between; Step 2_3_2: Subtract the mean of the column containing the element from the value of each element in the Matrix_IF matrix, and then divide the result by the standard deviation of the column containing the element to obtain the standardized value. Eq_StandardValue represents the standardized value of the element in the Matrix_IF matrix. The calculation formula is as follows: Eq_StandardValue=(Matrix_IF[m_row,m_col]-Column_avg[m_col]) / Sigma[m_col] Step 2_4: Construct a Matrix_UF matrix with the same size as the Matrix_IF matrix; Step 2_5: Add the Eq_StandardValue data one by one to the Matrix_UF matrix; Step 2_6: Return to BFM_OutParm; Step 2_7: The function Procedure_2_FunctionBFMatrix ends; Principal component analysis (PCA) is used to reduce the dimensionality of the Matrix_UF matrix, and the covariance matrix of the Matrix_UF matrix is ​​calculated. Then, eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The desired number of eigenvectors are selected to construct a projection matrix. The Set_UF set is then projected through the projection matrix to obtain the Set_MF set. The specific process is as follows: Procedure_2_FunctionPCAFeature (input parameters include PCAF_InParm_1 and PCAF_InParm_2, output parameter PCAF_OutParm) is a function that performs dimensionality reduction of a Matrix_UF matrix and constructs a Set_MF set. The input parameters PCAF_InParm_1 and PCAF_InParm_2 are the Matrix_UF matrix and the dimension of the desired dimensionality reduction, respectively. The output parameter PCAF_OutParm represents the Set_MF set. The specific process is as follows: Step 3_1: Calculate the covariance matrix of the Matrix_UF matrix; Eq_CovarianceMatrix represents the covariance matrix, and mean(Matrix_UF) represents the mean matrix of the Matrix_UF matrix, denoted as: mean(Matrix_UF) = mean MUF The calculation formula is: Eq_CovarianceMatrix=(Matrix_UF-mean MUF )*(Matrix_UF-mean MUF ) T / (Num_FileSample-1) Step 3_2: Perform eigenvalue decomposition on Eq_CovarianceMatrix to obtain eigenvalues ​​and corresponding eigenvectors; the eigenvalue decomposition calculation formula is as follows: Eq_CovarianceMatrix=Matrix_Eigenvector*Matrix_Diagonal*(Matrix_Eigenvector) T Among them, Matrix_Eigenvector is a matrix composed of the eigenvectors of Eq_CovarianceMatrix, and Matrix_Diagonal is a diagonal matrix whose diagonal elements are the eigenvalues ​​of Eq_CovarianceMatrix. Step 3: Sort the features by their magnitude and select the eigenvectors corresponding to the first two PCAF_InParm_2 eigenvalues. Step 3_4: Combine the selected PCAF_InParm_2 eigenvectors into a projection matrix Matrix_Project; Step 3_5: Project the Set_UF set onto a lower-dimensional space to obtain the Set_MF set; the calculation formula for the projection operation is: Set_MF=(Matrix_Project) T * Set_UF Step 3_6: Return PCAF_OutParm; Step 3_7: The function Procedure 2_Function PCAFeature ends; The system administrator determines the hash function to use to hash the main features in the Set_MF set, mapping them to unique hash values, and constructs an FHTable hash table to store the feature hash values; calculates the feature hash value of the file to be detected, uses the FHTable hash table to detect junk files in the cloud, and then returns a list of junk files. The specific process is as follows: The `Procedure_2_FunctionCHTable(input parameter CHT_InParm, output parameter CHT_OutParm)` function constructs an `FHTable` hash table. The input parameter `CHT_InParm` represents the `Set_MF` set, and the output parameter `CHT_OutParm` is the `FHTable` hash table. Specifically: Step 4_1: Traverse the Set_MF set and obtain the S_mf_val_index of the set. Set_MF element; Step 4_2: Analyze the element S_mf_val_index Set_MF Perform hash calculation to obtain the feature hash value Hash_MainFeature; Step 4_3: Based on the garbage file sample data, iterate through the File_Identifier corresponding to Hash_MainFeature in the Set_MF set; Step 4: Create an FHTable hash table; Step 4-5: Add Hash_MainFeature as the key and the corresponding File_Identifier as the value to the FHTable hash table; Step 4_6: If there are still elements in the Set_MF set whose Hash_MainFeature and File_Identifier have not been added to the FHTable hash table, then go to Step 4_2; otherwise, go to Step 4_7. Step 4_7: Return CHT_OutParm; Step 4_8: The function Procedure 2_Function CHTable ends; The `Procedure_2_FunctionFDetection(input parameter FD_InParm, output parameter FD_OutParm)` function determines whether a file to be detected is a junk file. The input parameter `FD_InParm` represents the file to be detected in the public cloud, and the output parameter `FD_OutParm` is a list of junk files. Step 5_1: Use the API of the object storage service provided by the public cloud to obtain the FileMetadata of FD_InParm; Step 5_2: Construct the Set_IF set of FD_InParm; Step 5_3: Extract the relevant feature information from the FD_InParm file and store it in the Set_IF set; Step 5_4: Traverse the Set_IF set and check the feature S_if_val_index. Set_IF Apply a hash algorithm to obtain the feature hash value Hash_TestFile of the file to be tested; Step 5: Traverse the FHTable hash table. The function Find(Hash_TestFile) checks if Hash_TestFile exists. Step 5_5_1: If Find(Hash_TestFile)=true, then FD_InParm has matching characteristics with the specified type of garbage file, and FD_InParm is determined to be a garbage file; Step 5_5_1_1: Insert a record into the junk file list to store FD_InParm information, including file characteristic information; if there are still files to be detected in the public cloud, go to Step 5_1 to continue detection; otherwise, go to Step 5_6. Step 5_5_2: If Find(Hash_TestFile) = false, then FD_InParm is a non-junk file; if there are still files to be tested in the public cloud, go to Step 5_1 to continue testing the next file; otherwise, go to Step 5_6. Step 5-6: Return to FD_OutParm; Step 5_7: The function Procedure_2_FunctionFDetection ends; Part 2 is now complete; The specific process of Part 3 is described as follows: The server-side public garbage collection station periodically recycles junk files, freeing up storage space. Initialization vector P_cff={P_cff_val_1, P_cff_val_2, …, P_cff_val_LenNum p_cff }, where P_cff_val_1 is the first parameter of the vector P_cff, and P_cff_val_LenNum p_cff It's the last parameter, LenNum p _cff This represents the total number of parameters in the vector. The sum of all parameters in P_cff is 1, denoted as Σ(P_cff)=1. Each parameter of P_cff takes a value between 0 and 1. Eq_ClearFileFactor represents the recycling factor, calculated using the following formula: Eq_ClearFileFactor=P_cff_val_1*CloudFile_Time-P_cff_val_2*CloudFile_NumBack+P_cff_val_3*CloudFile_NumEdit+P_cff_val_4*CloudFile_NumShare+P_cff_val_5*CloudFile_NumRead; Procedure_3 executes the process of periodically reclaiming junk files from the server-side public garbage collection station, specifically as follows: The server updates the P_cff parameter value periodically according to the agreed time and training set configuration, clears the junk files of deactivated cloud users in the public garbage collection system storage space, and clears the backup files of frozen cloud users' junk files when the public garbage collection system storage space reaches 80% of its capacity; the server administrator sets the recycling factor threshold to ValMax_ClearFileFactor, calculates the Eq_ClearFileFactor value of files in the garbage classifier, and clears the junk file from the buffer when Eq_ClearFileFactor > ValMax_ClearFileFactor. Part 3 is now complete; The specific process of Part 4 is described as follows: Establish a waste recycling sorter on the public cloud user end; Procedure_4 executes the server-side push to the client-side user-defined CloudUser_forUserArr and CloudUser_forTagArr, and establishes the user-side garbage recycling classifier and user-defined classifier tag content in the user's cloud. After logging into the cloud, the user can modify their own garbage recycling classification tag and set their own garbage sorting and recycling station. Part 4 is now complete.

Images