An intelligent archive management system and method based on big data

Abstract

The present invention discloses an intelligent archive management system and method based on big data, and relates to the technical field of archive management. The intelligent archive management method based on big data ensures that only authorized personnel can upload files through identity authentication, and the upload feasibility check avoids the storage of non-compliant archives. Intelligent classification and dynamic retrieval further optimize the organizational structure of archives, so that the archive library remains up-to-date and most relevant. At the same time, automatic backup and removal of outdated or no longer needed archives reduce the storage burden and ensure the integrity and accessibility of data. Intelligent classification and regular cleaning mechanisms help organizations effectively manage storage space and reduce unnecessary storage costs, which can effectively improve the efficiency and security of archive management.

Description

Technical Field

[0001] The present invention relates to the technical field of archive management, and in particular to an intelligent archive management system and method based on big data. Background Art

[0002] Through the rational use of electronic archive management systems, not only can the archives of enterprises and institutions be managed electronically, but also the multi-dimensional information in the archives can be processed, thereby promoting the development of the corresponding units and increasing the added value of the enterprises.

[0003] In the process of realizing the management and utilization of archival information, searching for electronic archives is a common link. The current electronic archive management system already supports a variety of search methods, among which full-text search, fuzzy search and precise search are the mainstream search methods. Full-text search is suitable for when the search personnel only have an impression of part of the document content, that is, when the search personnel enter part of the document content in the full-text search engine, the electronic archive management system automatically searches for electronic archives containing the corresponding document content in the full-text search engine vocabulary. However, because the search personnel enter less content in the full-text search engine, it is impossible to accurately locate the target document. Therefore, the full-text search engine vocabulary is mostly a massive database, so as to return and display documents containing the corresponding document content as much as possible.

[0004] The existing technology is not convenient for intelligent classification of archives based on archive keywords when managing archives. The classification process is cumbersome and increases the storage burden. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an intelligent archive management system and method based on big data, which solves the problem that in the prior art, it is inconvenient to intelligently classify archives based on their keywords during archive management, the classification process is cumbersome, and the storage burden is increased.

[0006] To achieve the above purpose, the present invention is implemented through the following technical solutions: an intelligent file management method based on big data, comprising the following steps: verifying the identity of the person uploading the file, and performing an upload feasibility check on the file to be uploaded after the verification is passed. If the upload feasibility check fails, uploading is not allowed, and if the upload feasibility check succeeds, uploading is allowed; intelligently classifying the files to be uploaded that are allowed to be uploaded, and determining the final recommended classification type corresponding to the files to be uploaded that are allowed to be uploaded in the database; performing file retrieval on the files under the final recommended classification type corresponding to the determined files to be uploaded that are allowed to be uploaded in the database, and determining whether there are files to be removed under the final recommended classification type, and if so, backing up the files to be removed and removing them.

[0007] Furthermore, the identity of the person uploading the file is verified, and after the verification is passed, the feasibility of uploading the file to be uploaded is checked, including the following steps: obtaining the identity information of the person uploading the file, and verifying the identity information of the person uploading the file, wherein the identity information of the person uploading the file includes the account number, password and employee number of the person uploading the file: if the identity information of the person uploading the file is not the corresponding person stored in the database, the verification fails, and a login prompt for the non-permitted person is given; if the identity information of the person uploading the file is the corresponding person stored in the database, the verification passes; receiving the file to be uploaded, extracting keywords from the file to be uploaded, and obtaining a content keyword set; comparing the obtained content keyword set with the banned word set stored in the database: if the content keyword set and the banned word set have an intersection, the upload feasibility check fails, and uploading is not allowed; if the content keyword set and the banned word set do not have an intersection, the upload feasibility check succeeds, and uploading is allowed.

[0008] Further, the intelligent classification of the archives to be uploaded that are allowed to be uploaded and determining the final recommended classification type corresponding to the archives to be uploaded that are allowed to be uploaded in the database includes the following steps: extracting keywords from the referenced content of the archives to be uploaded that are allowed to be uploaded and obtaining a reference document keyword set; extracting keywords from the annotations of the archives to be uploaded that are allowed to be uploaded and obtaining an annotation keyword set; obtaining the ratio of structured content to unstructured content of the archives to be uploaded that are allowed to be uploaded; obtaining a content keyword reference set, a reference document keyword reference set, an annotation keyword reference set and a reference ratio of structured content to unstructured content of the archives to be uploaded corresponding to each classification type stored in the database; fusing the content keyword set, the reference document keyword set, the annotation keyword set, the ratio of structured content to unstructured content of the archives to be uploaded that are allowed to be uploaded, the content keyword reference set, the reference document keyword reference set, the annotation keyword reference set and the reference ratio of structured content to unstructured content corresponding to each classification type stored in the database to obtain a classification matching index corresponding to each classification type; the calculation formula of the classification matching index is as follows:

[0009]

[0010] In the formula, i is the number of the classification type stored in the database, Fz i is the classification matching index corresponding to the i-th classification type, A 1 is the set of keywords for the content of the files to be uploaded that are allowed to be uploaded, 1 is the keyword set of the referenced documents of the uploaded archives that are allowed to be uploaded, C 1 is the set of caption keywords of the files to be uploaded that are allowed to be uploaded, BL is the ratio of structured content to unstructured content of the files to be uploaded that are allowed to be uploaded, and A 2iis the content keyword reference set corresponding to the i-th classification type stored in the database, B 2i is the reference document keyword reference set corresponding to the i-th classification type stored in the database, C 2i is the reference set of caption keywords corresponding to the i-th classification type stored in the database, cBL i is the reference ratio of structured content to unstructured content corresponding to the i-th classification type stored in the database, e is a natural constant, σ(A 1 ,A 2i ) is the similarity between the content keyword set and the reference document keyword reference set corresponding to the i-th classification type stored in the database, σ(B 1 ,B 2i ) is the similarity between the reference document keyword set and the reference document keyword reference set corresponding to the i-th classification type stored in the database, σ(C 1 ,C 2i ) is the similarity between the caption keyword set and the caption keyword reference set corresponding to the ith classification type stored in the database; the classification matching indexes are arranged in ascending order, and based on the selection number N set in the database, the classification types corresponding to the first N classification matching indexes are obtained, which are recorded as the recommended classification types to be determined; a classification type is determined from the recommended classification types to be determined as the final recommended classification type.

[0011] Furthermore, the method of determining a classification type from the to-be-determined recommended classification types as the final recommended classification type includes the following steps: pre-storing the to-be-uploaded files that are allowed to be uploaded into each to-be-determined recommended classification type; retrieving the pre-stored to-be-uploaded files that are allowed to be uploaded from each to-be-determined recommended classification type, and obtaining the actual retrieval situation data corresponding to each to-be-determined recommended classification type, the actual retrieval situation data including the average response time, retrieval success rate, average CPU usage rate and average broadband usage rate; obtaining the parameter retrieval situation data corresponding to each to-be-determined recommended classification type stored in the database, the parameter retrieval situation data including the parameter average response time, the parameter retrieval success rate, the CPU parameter average usage rate and the broadband parameter average Utilization rate; obtaining the allowable deviation data of the retrieval situation corresponding to each to-be-determined recommended classification type, wherein the allowable deviation data of the retrieval situation includes the allowable deviation of the average response time, the allowable deviation of the retrieval success rate, the allowable deviation of the average CPU usage rate and the allowable deviation of the average bandwidth utilization rate; fusing the actual retrieval situation data corresponding to each to-be-determined recommended classification type, the parameter retrieval situation data corresponding to each to-be-determined recommended classification type stored in the database and the allowable deviation data of the retrieval situation corresponding to each to-be-determined recommended classification type, and obtaining the retrieval situation evaluation value corresponding to each to-be-determined recommended classification type; obtaining the minimum retrieval situation evaluation value, and determining the to-be-determined recommended classification type corresponding to the minimum retrieval situation evaluation value as the final recommended classification type.

[0012] Furthermore, the calculation formula of the retrieval situation evaluation value is as follows:

[0013]

[0014] In the formula, j is the number of the recommended classification type to be determined, Ts is the average response time, Ds is the call success rate, CPs is the average CPU usage, Ks is the average bandwidth utilization, Tc j Dc is the average response time corresponding to the jth recommended classification type to be determined stored in the database, j is the parameter retrieval success rate corresponding to the jth recommended classification type to be determined stored in the database, CPc j Kc is the average CPU usage corresponding to the jth recommended classification type to be determined stored in the database, j is the average utilization rate of broadband parameters corresponding to the jth recommended classification type to be determined stored in the database, ΔTc j ΔDc is the average response time tolerance corresponding to the jth recommended classification type to be determined stored in the database, j ΔCPc is the allowable deviation of the retrieval success rate corresponding to the jth recommended classification type to be determined stored in the database, jΔKc is the allowable deviation of the average CPU usage corresponding to the jth recommended classification type to be determined stored in the database, j Dp is the allowable deviation of the average bandwidth utilization corresponding to the jth recommended classification type to be determined stored in the database, j It is the evaluation value of the retrieval situation corresponding to the jth recommended classification type to be determined.

[0015] Furthermore, the obtaining of the allowable deviation data for retrieval conditions corresponding to each to-be-determined recommended classification type comprises the following steps: obtaining the archive storage condition data under each to-be-determined recommended classification type, the archive storage condition data comprising the archive storage quantity, the total archive storage space and the data transmission rate; fusing the archive storage condition data under each to-be-determined recommended classification type to obtain the allowable deviation data comparison parameters corresponding to each to-be-determined recommended classification type; performing difference calculation between the allowable deviation data comparison parameters corresponding to each to-be-determined recommended classification type and the comparison standard values ​​corresponding to each to-be-determined recommended classification type stored in the database to obtain the deviation values ​​corresponding to each to-be-determined recommended classification type; comparing the deviation values ​​corresponding to each to-be-determined recommended classification type with the deviation values ​​corresponding to each to-be-determined recommended classification type stored in the database to obtain the allowable deviation data for retrieval conditions corresponding to the deviation values ​​corresponding to the to-be-determined recommended classification types stored in the database that have been successfully compared.

[0016] Furthermore, the calculation formula of the allowable deviation data comparison parameter is as follows:

[0017]

[0018] In the formula, j is the number of the recommended classification type to be determined, YB j is the allowable deviation data comparison parameter corresponding to the jth recommended classification type to be determined, Sd j is the number of archives stored for the jth recommended classification type to be determined, ck j is the total storage space of the archives corresponding to the jth recommended classification type to be determined, cL j is the data transmission rate corresponding to the jth recommended classification type to be determined, α 1 is the weight factor of the number of archives stored in the database, α 2 is the total storage space weight factor of the archives stored in the database, α 3 is the data transmission rate weight factor stored in the database, and e is a natural constant.

[0019] Furthermore, the file search for the files under the final recommended classification type corresponding to the determined files to be uploaded and allowed to be uploaded in the database to determine whether there are files to be removed under the final recommended classification type includes the following steps: obtaining historical reference data of each file under the final recommended classification type, the historical reference data including the number of references, average opening duration and the proportion of duplicate content; obtaining reference definition data of the files corresponding to the final recommended classification type stored in the database, the reference definition data including the number of reference definitions, average opening duration and the proportion of duplicate content definition; matching the historical reference data of each file under the final recommended classification type with the final recommended classification type stored in the database The review definition data is integrated and processed to obtain the historical review evaluation value of each file; the historical review evaluation value of each file is compared with the historical review evaluation threshold corresponding to the final recommended classification type stored in the database to determine whether the historical review evaluation value of each file is less than the historical review evaluation threshold corresponding to the preset recommended classification type stored in the database: if the historical review evaluation value is less than the historical review evaluation threshold corresponding to the preset recommended classification type stored in the database, the file corresponding to the historical review evaluation value needs to be removed; if the historical review evaluation value is not less than the historical review evaluation threshold corresponding to the preset recommended classification type stored in the database, the file corresponding to the historical review evaluation value does not need to be removed.

[0020] Furthermore, the calculation formula of the historical review evaluation value is as follows:

[0021]

[0022] In the formula, f is the file number, LCo f is the historical review evaluation value of the f-th file, Sc f is the number of times the f-th file is viewed, Dc f is the average opening duration of the f-th file, cZ f is the percentage of duplicate content in the f-th file, Ds is the number of access definitions, DD is the average open continuous definition time, ScZ is the percentage of duplicate content definitions, and e is a natural constant.

[0023] A smart archive management system based on big data, used for the above-mentioned smart archive management method based on big data, comprises an upload feasibility check module, an archive classification module and an existing archive processing module, wherein: the upload feasibility check module is used to verify the identity of the archive uploader, and after the verification, perform an upload feasibility check on the archive to be uploaded. If the upload feasibility check fails, uploading is not allowed, and if the upload feasibility check succeeds, uploading is allowed; the archive classification module is used to intelligently classify the archives to be uploaded that are allowed to be uploaded, and determine the corresponding pre-set recommended classification type of the archives to be uploaded that are allowed to be uploaded in the database; the existing archive processing module is used to perform archive retrieval on the archives under the corresponding pre-set recommended classification type in the database of the determined archives to be uploaded that are allowed to be uploaded, and determine whether there are archives to be removed, and if so, back up the archives to be removed and then remove them.

[0024] The present invention has the following beneficial effects:

[0025] This intelligent archive management method based on big data ensures that only authorized personnel can upload files through identity authentication, and the upload feasibility check avoids the storage of non-compliant archives. Intelligent classification and dynamic retrieval further optimize the organizational structure of archives, keeping the archives up to date and most relevant. At the same time, automatic backup and removal of outdated or no longer needed archives reduce the storage burden and ensure data integrity and accessibility. Intelligent classification and regular cleanup mechanisms help organizations effectively manage storage space and reduce unnecessary storage costs, which can effectively improve the efficiency and security of archive management.

[0026] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The figure is a flow chart of the intelligent archive management method based on big data of the present invention.

[0028] Figure 2 This is a flowchart of the intelligent archive management system based on big data of the present invention. DETAILED DESCRIPTION

[0029] The embodiment of the present application uses an intelligent archive management method and system based on big data. Identity authentication ensures that only authorized personnel can upload files. Upload feasibility checks avoid the storage of non-compliant archives. Intelligent classification and dynamic retrieval further optimize the organizational structure of archives, keeping the archive library up to date and most relevant. At the same time, automatic backup and removal of outdated or no longer needed archives reduce the storage burden and ensure data integrity and accessibility. Intelligent classification and regular cleanup mechanisms help organizations effectively manage storage space and reduce unnecessary storage costs, which can effectively improve the efficiency and security of archive management.

[0030] See also Figure 1 The embodiment of the present invention provides a technical solution: an intelligent file management method based on big data, comprising the following steps: verifying the identity of the file uploader, and performing an upload feasibility check on the file to be uploaded after the verification is passed. If the upload feasibility check fails, uploading is not allowed, and if the upload feasibility check succeeds, uploading is allowed; intelligently classifying the files to be uploaded that are allowed to be uploaded, and determining the final recommended classification type corresponding to the files to be uploaded that are allowed to be uploaded in the database; performing file retrieval on the files under the final recommended classification type corresponding to the determined files to be uploaded that are allowed to be uploaded in the database, and determining whether there are files to be removed under the final recommended classification type, and if so, backing up the files to be removed and removing them.

[0031] First, the identity of the person uploading the file is verified to ensure that only authorized users can upload files. This step is to ensure information security and compliance with relevant laws and regulations. Before the file is uploaded, the compliance of the file will be automatically checked to ensure that all uploaded files meet the standards and requirements of the organization. The uploaded files are automatically classified into the corresponding database classification through algorithms, which improves the search efficiency based on the content of the file and other information. Retrieve files under the same category and evaluate whether there are files that need to be updated or removed. This includes automatically backing up and deleting outdated or redundant files to ensure the latest and accuracy of the archive.

[0032] Through strict identity authentication and file inspection, the risk of information leakage and illegal access is reduced. Intelligent classification and regular cleanup mechanisms help organizations effectively manage storage space and reduce unnecessary storage costs. It can effectively improve the efficiency and security of archive management. Identity authentication ensures that only authorized personnel can upload files, and upload feasibility checks avoid the storage of non-compliant archives. Intelligent classification and dynamic retrieval further optimize the organizational structure of archives, keeping the archive library up to date and most relevant. At the same time, automatic backup and removal of outdated or no longer needed archives reduce the storage burden and ensure the integrity and accessibility of data. This archive management method is particularly suitable for environments that need to process large amounts of data and have high requirements for data security, such as government agencies, large enterprises, and research institutions.

[0033] Specifically, the identity of the person who uploaded the file is verified, and after the verification is passed, the feasibility of uploading the file to be uploaded is checked, including the following steps: obtaining the identity information of the person who uploaded the file, and verifying the identity information of the person who uploaded the file, the identity information of the person who uploaded the file includes the account number, password and employee number of the person who uploaded the file: if the identity information of the person who uploaded the file is not the corresponding person stored in the database, the verification fails, and a prompt for non-permitted personnel to log in is given; if the identity information of the person who uploaded the file is the corresponding person stored in the database, the verification passes; receiving the file to be uploaded, extracting keywords from the file to be uploaded, and obtaining a content keyword set; comparing the obtained content keyword set with the banned word set stored in the database: if the content keyword set and the banned word set have an intersection, the upload feasibility check fails, and uploading is not allowed; if the content keyword set and the banned word set do not have an intersection, the upload feasibility check succeeds, and uploading is allowed.

[0034] In this implementation scheme, the identity information of the person uploading the file, including the account number, password and work number, is obtained, and the obtained identity information is compared with the records in the database. If the information does not match, the verification fails, and the system will prompt the non-permitted person to log in and prevent further operations. If the information matches, the verification is successful and the user can proceed to the next step. The file to be uploaded is received, and the keywords therein are extracted. The extracted keywords are compared with the set of banned words pre-stored in the database. If there is an intersection between the keywords and the banned words, it means that the file may contain inappropriate or sensitive content, so uploading is not allowed. If there is no intersection, it means that the content of the file meets the requirements, the upload feasibility check passes, and the file is allowed to be uploaded.

[0035] Authentication ensures that only authorized personnel can upload archives, preventing unauthorized access and potential data leakage. By checking banned words, sensitive or non-compliant content can be avoided, protecting organizations from legal and reputation risks. By combining authentication and content monitoring, not only information security is ensured, but also the quality of data management and user operations is optimized, making the archive management process more intelligent and in line with the needs of modern enterprises.

[0036] Natural language processing (NLP) technology can be used to obtain a set of content keywords. This process first requires preprocessing of the archive content, including removing useless information and standardizing the text format. Then, existing algorithms such as TF-IDF (term frequency-inverse document frequency), text mining algorithms, or deep learning models (such as BERT, etc.) can be applied to identify and extract keywords from the text. These keywords are a high-level summary of the main content and theme of the document, which can effectively reflect the core information of the archive. Finally, the extracted keyword set can be used for further content review, classification, and indexing, thereby improving the efficiency and accuracy of the archive management system.

[0037] Specifically, intelligent classification is performed on the archives to be uploaded that are allowed to be uploaded, and the final recommended classification type corresponding to the archives to be uploaded that are allowed to be uploaded in the database is determined, including the following steps: keyword extraction is performed on the reference content of the archives to be uploaded that are allowed to be uploaded, and a keyword set of the referenced document is obtained (first, the reference content part in the archives to be uploaded is obtained, including document citations, quoted paragraphs or annotations. Next, the natural language processing (NLP) technology, such as word segmentation and part-of-speech tagging, is used to perform text analysis on these reference contents. Then, a keyword extraction algorithm (such as TF-IDF or text mining model) is applied to identify high-frequency and key words from the text. These words constitute the keyword set of the referenced document, which represent the core themes and concepts of the cited documents. This keyword set can then be used for further content analysis and archive classification); keyword extraction is performed on the annotations of the archives to be uploaded that are allowed to be uploaded, and a keyword set of the annotations is obtained (first, the annotation texts under all figures and tables in the archives are identified and extracted. Next, these annotation texts will be preprocessed, such as removing noise and standardizing the format. After that, the natural language processing (NLP) technology, such as word segmentation and part-of-speech tagging, is used to extract the annotations. Note: Analyze the caption text. Apply keyword extraction algorithms (such as TF-IDF or other text mining techniques) to identify important words in the captions, which reflect the main content and information of the graphics or tables. Ultimately, these extracted keywords form a caption keyword set, which can be used to enhance the indexing and search functions of archive content, improve the overall efficiency and data availability of the archive management system); obtain the ratio of structured content to unstructured content of the archives to be uploaded that are allowed to be uploaded (first, scan and parse the uploaded archives to identify the structured content (such as tables, lists, database export data, etc.) and unstructured content (such as free text, image descriptions, etc.). This usually requires the use of document parsing technology, such as OCR (optical character recognition) or specific document parsing algorithms, to extract and mark different types of data. The system then calculates the amount of data for each content type, which may be in units of characters, words, or data blocks. By comparing the measurements of these two types of content, the system is able to calculate the ratio of structured to unstructured content. This ratio helps the intelligent classification system better understand the content composition of the archives, so that it can process and classify them more accurately).

[0038] Obtain the content keyword reference set, reference document keyword reference set, caption keyword reference set and reference ratio of structured content and unstructured content of the archive to be uploaded corresponding to each classification type stored in the database. The process of obtaining the reference set and reference ratio corresponding to each classification type stored in the database is as follows: Collect samples from existing archives and data. These samples should represent different classification types. Perform detailed content analysis on these sample archives, including text, reference documents and captions. Use natural language processing techniques such as TF-IDF and LDA (latent Dirichlet allocation) model to extract keywords from these contents. Parse the sample archives to identify structured content (such as tables, database records) and unstructured content (such as paragraph text, image annotations). Calculate the ratio of these two types of content in each sample. Based on the above analysis, create a keyword reference set for each classification type, including content keywords, reference document keywords, and caption keywords. At the same time, determine the typical ratio of structured and unstructured content for each classification.

[0039] The content keyword set, reference document keyword set, caption keyword set, ratio of structured content to unstructured content of the archives to be uploaded that are allowed to be uploaded, the content keyword reference set, reference document keyword reference set, caption keyword reference set and reference ratio of structured content to unstructured content corresponding to each classification type stored in the database are integrated to obtain the classification matching index corresponding to each classification type; the classification matching indexes are arranged in ascending order, and based on the selection number N set in the database, the classification types corresponding to the classification matching indexes ranked in the top N are obtained, which are recorded as the recommended classification types to be determined; and a classification type is determined from the recommended classification types to be determined as the final recommended classification type.

[0040] Extract keywords from different parts of the archives that are allowed to be uploaded (such as reference documents and captions), and analyze the ratio of structured content (such as tables and lists) to unstructured content (such as free text) in the archives. Obtain the classification reference set preset in the database, which includes the content keywords, reference document keywords, caption keyword set, and structured and unstructured content ratio of each category, and fuse the keyword set and content ratio of the archive to be uploaded with the reference set in the database to generate a classification matching index. According to the classification matching index, sort from high to low, select the top N categories with the highest matching degree, and determine a final recommended classification type from these candidate categories.

[0041] By comprehensively analyzing text keywords and content structure, this method can more accurately classify archives into the most relevant categories in the database, improving the efficiency and accuracy of data retrieval. This method is particularly suitable for processing large amounts of archival data and can be expanded to meet the needs of large-scale information systems.

[0042] The calculation formula of classification matching index is as follows:

[0043]

[0044] In the formula, i is the number of the classification type stored in the database, Fz i is the classification matching index corresponding to the i-th classification type, A 1 is the set of keywords for the content of the files to be uploaded that are allowed to be uploaded, 1 is the keyword set of the referenced documents of the uploaded archives that are allowed to be uploaded, C 1 is the set of caption keywords of the files to be uploaded that are allowed to be uploaded, BL is the ratio of structured content to unstructured content of the files to be uploaded that are allowed to be uploaded, and A 2i is the content keyword reference set corresponding to the i-th classification type stored in the database, B 2i is the reference document keyword reference set corresponding to the i-th classification type stored in the database, C 2i is the reference set of caption keywords corresponding to the i-th classification type stored in the database, cBL i is the reference ratio of structured content to unstructured content corresponding to the i-th classification type stored in the database, e is a natural constant, σ(A 1 ,A 2i ) is the similarity between the content keyword set and the reference document keyword reference set corresponding to the i-th classification type stored in the database, σ(B 1 ,B 2i ) is the similarity between the reference document keyword set and the reference document keyword reference set corresponding to the i-th classification type stored in the database, σ(C 1 ,C 2i ) is the similarity between the caption keyword set and the caption keyword reference set corresponding to the i-th classification type stored in the database.

[0045] In this implementation scheme, the content keyword set represents the keyword set of the main text of the archive, which directly reflects the theme and content focus of the archive. It is the most direct source of information in classification decisions. When used in combination with other parameters, it can provide a core perspective of the archive content. The reference document keyword set contains keywords extracted from the referenced part of the archive. This part usually contains supporting arguments or supplementary information, which helps to understand the academic or documentary background of the archive. Combined with the keywords of the main content, the reference document keywords help reveal the depth and scope of the archive, providing a supplementary perspective that is crucial to the accuracy of classification. The caption keyword set involves explanatory text of graphics, tables, or other visual content. These are usually visual explanations or supplementary explanations of the main content. Caption keywords provide an understanding of the visual information of the archive, which is especially important when dealing with archives rich in graphics and tables, and helps to more comprehensively evaluate the multi-dimensional information of the content. The ratio of structured content to unstructured content indicates the proportion of structured data (such as tables and lists) to unstructured text (such as free text descriptions) in the archive, which reflects the organization of information and the form of data. The ratio of structured to unstructured content helps to understand the information density and organizational complexity of the archive. Combined with the keyword set, it can classify the archive more accurately, especially in situations where it is necessary to distinguish between highly structured reports and general text documents.

[0046] The combined use of these four parameters can comprehensively evaluate the content characteristics and adaptability of the files to be uploaded from different dimensions. By comparing these detailed data points, the intelligent file management system can accurately classify the files into the most appropriate category, ensuring rapid retrieval and efficient management of information. This multi-dimensional analysis method improves the accuracy of classification and reduces the possibility of misclassification, thereby improving the overall data processing efficiency and system practicality.

[0047] Use σ(A 1 ,A 2i ),σ(B 1 ,B 2i ),σ(C 1 ,C 2i ) represents the similarity between the content keyword set, reference document keyword set, caption keyword set and the reference set of the corresponding classification in the database. These similarity values ​​can be calculated by, for example, cosine similarity (cosine similarity is applicable to text data in high-dimensional space, and can effectively measure the similarity of two sets of keywords in direction, while ignoring the difference in the number of keywords), which measures the correlation and consistency between keywords. The influence of keyword similarity is adjusted by an exponential function, so that the case with high similarity contributes more to the matching index. The difference between the ratio of structured and unstructured content of the uploaded archive and the standard ratio of each category in the database is taken into account, which partly reflects the consistency of the content structure. The use of logarithms and square roots is to adjust the impact of the ratio differences on the results to make them smoother and more manageable.

[0048] Specifically, a classification type is determined as the final recommended classification type from the recommended classification types to be determined, including the following steps: pre-storing the files to be uploaded that are allowed to be uploaded under each recommended classification type to be determined; retrieving the pre-stored files to be uploaded that are allowed to be uploaded from each recommended classification type to be determined, and obtaining the actual retrieval situation data corresponding to each recommended classification type to be determined, and the actual retrieval situation data includes average response time, retrieval success rate, average CPU usage rate and average broadband utilization rate. The average response time refers to the average time from sending a request to receiving a response, and the start and end time of each file or data retrieval request is recorded through the system log, and the response time of all requests is summed and divided by the total number of requests. The retrieval success rate refers to the ratio of successfully completed retrieval requests to the total retrieval requests, monitoring all retrieval requests and recording the number of successes and failures, and calculating the number of successes divided by the total number of requests to obtain the success rate. The average CPU usage rate refers to the proportion of time that the CPU is occupied by the retrieval task within a certain period of time, and the CPU usage is monitored using tools provided by the operating system (such as the top command in Linux, the task manager in Windows) or special performance monitoring software. Broadband utilization refers to the utilization rate of network bandwidth within a certain period of time. Network monitoring tools (such as Wireshark, NetFlow, etc.) are used to capture network traffic data, calculate the ratio of the amount of data transmitted within a given period of time to the total capacity of the network bandwidth, and obtain the broadband utilization rate.

[0049] Obtain the parameter call data corresponding to each undetermined recommended classification type stored in the database, and the parameter call data includes the average parameter response time, parameter call success rate, CPU parameter average utilization rate and broadband parameter average utilization rate. The parameter call data corresponding to each undetermined recommended classification type stored in the database, such as the average parameter response time, parameter call success rate, CPU parameter average utilization rate and broadband parameter average utilization rate, are usually obtained based on historical performance data and expected performance standards. These data reflect the performance of each classification type under ideal or typical conditions, and are used as a benchmark for comparing and evaluating actual performance. The process of obtaining these data includes the following steps: Analyze the historical call records of different classification types, and calculate the average response time, success rate, CPU and broadband utilization rate of each classification. By designing and executing targeted performance tests, simulate various operations to ensure the reliability and accuracy of various indicators. According to the organization's service level agreement (SLA) or performance goals, set the parameter values ​​of each performance indicator, and these parameter values ​​are based on the optimal performance standards. Store these parameter values ​​in the database.

[0050] Obtain the allowable deviation data of the retrieval situation corresponding to each to-be-determined recommended classification type, the allowable deviation data of the retrieval situation including the allowable deviation of the average response time, the allowable deviation of the retrieval success rate, the allowable deviation of the average CPU usage rate and the allowable deviation of the average bandwidth utilization rate; fuse the actual retrieval situation data corresponding to each to-be-determined recommended classification type, the parameter retrieval situation data corresponding to each to-be-determined recommended classification type stored in the database and the allowable deviation data of the retrieval situation corresponding to each to-be-determined recommended classification type to obtain the retrieval situation evaluation value corresponding to each to-be-determined recommended classification type; obtain the minimum retrieval situation evaluation value, and determine the to-be-determined recommended classification type corresponding to the minimum retrieval situation evaluation value as the final recommended classification type.

[0051] Temporarily store the files to be uploaded that are allowed to be uploaded in each candidate recommendation category. This step simulates the storage status of the files under each category. Test the retrieval performance of the pre-stored files under each category and collect key performance indicators, such as average response time, retrieval success rate, CPU usage, and broadband utilization. Obtain the preset retrieval performance standard data (parameter data) for each category, including the predetermined response time, success rate, CPU and broadband usage standards. Define the allowable deviation for each performance indicator. These deviation indicators help evaluate the difference between the actual performance data and the predetermined standard. Fusion the actual retrieval data, predetermined data, and allowable deviation data to calculate a comprehensive retrieval situation evaluation value. This evaluation value reflects the performance adaptability of the file under each category. Select the category with the lowest retrieval situation evaluation value (best performance) from all candidate categories as the final recommended category.

[0052] By actually testing and evaluating the performance of archive retrieval under different categories, we can ensure that the selected category has the best performance, which is crucial for efficient data access. Evaluating CPU usage and bandwidth utilization helps optimize resource allocation and management and reduce operating costs. Improving the success rate and response time of retrieval directly improves the user experience, especially in environments that require frequent access and processing of large amounts of data. Using a data-driven approach to select categories reduces the interference of subjective judgment and ensures the objectivity and scientificity of decision-making.

[0053] The calculation formula for the retrieval situation assessment value is as follows:

[0054]

[0055] In the formula, j is the number of the recommended classification type to be determined, Ts is the average response time, Ds is the call success rate, CPs is the average CPU usage, Ks is the average bandwidth utilization, Tc j Dc is the average response time corresponding to the jth recommended classification type to be determined stored in the database, j is the parameter retrieval success rate corresponding to the jth recommended classification type to be determined stored in the database, CPc j Kc is the average CPU usage corresponding to the jth recommended classification type to be determined stored in the database, j is the average utilization rate of broadband parameters corresponding to the jth recommended classification type to be determined stored in the database, ΔTc j ΔDc is the average response time tolerance corresponding to the jth recommended classification type to be determined stored in the database, j ΔCPc is the allowable deviation of the retrieval success rate corresponding to the jth recommended classification type to be determined stored in the database, j ΔKc is the allowable deviation of the average CPU usage corresponding to the jth recommended classification type to be determined stored in the database, j Dp is the allowable deviation of the average bandwidth utilization corresponding to the jth recommended classification type to be determined stored in the database, j It is the evaluation value of the retrieval situation corresponding to the jth recommended classification type to be determined.

[0056] In this implementation scheme, for each performance indicator, the difference between the actual value and the parameter value is calculated and then divided by the allowable deviation to obtain a standardized performance deviation value. All standardized deviation values ​​are added together and increased by 1. This "add 1" operation can prevent the result from being zero after applying the hyperbolic tangent function when all performance indicators are completely consistent with the parameter values ​​(the deviation is zero).

[0057] The purpose of making parameter values ​​close to the set standards (parameter values) rather than just optimizing to the highest or lowest is to ensure that the overall performance of the system matches the expected business needs and operating environment. Response time is a direct indicator of user experience and reflects the speed at which the system processes requests. In data retrieval systems, fast response time means efficient data retrieval and processing capabilities. Response time is affected by CPU and bandwidth utilization. High resource utilization efficiency leads to faster response. However, if too fast a response time is pursued, unnecessary or uneconomical resources (such as CPU and bandwidth) may be consumed, resulting in unnecessary increase in operating costs. The success rate measures the proportion of requests that are correctly and completely responded to, which is directly related to reliability and error handling capabilities. The retrieval success rate is closely related to the stability of the system and resource management (such as CPU and bandwidth). Insufficient resources or improper management may lead to retrieval failures. CPU utilization reflects the occupation of processor resources. Efficient CPU utilization can support more concurrent requests and faster processing speeds. CPU utilization affects response time and overall system performance. Excessive CPU utilization may cause processing delays and service degradation. Broadband utilization reflects the use of network resources. It is particularly important for data retrieval systems that rely on network transmission. It affects the speed and stability of data transmission. The efficiency of broadband use directly affects the response time and success rate of data retrieval. Insufficient resources may lead to network congestion and data retrieval failure.

[0058] The average response time (Ts), call success rate (Ds), average CPU usage (CPs), and average bandwidth utilization (Ks) are selected as key parameters for performance evaluation because these indicators comprehensively reflect the efficiency, reliability, and resource management capabilities of a data processing system.

[0059] By dividing the performance deviation by the allowable deviation, the importance and influence of different performance indicators can be standardized to ensure that no single indicator has an excessive impact on the evaluation results. Allowing different parameter values ​​and deviation limits to be set for different classification types provides a high degree of customization to adapt to different business needs and performance goals.

[0060] Specifically, obtaining the allowable deviation data of retrieval situation corresponding to each to-be-determined recommended classification type includes the following steps: obtaining the archive storage situation data under each to-be-determined recommended classification type, the archive storage situation data including the archive storage quantity, the total archive storage space and the data transmission rate; fusing the archive storage situation data under each to-be-determined recommended classification type to obtain the allowable deviation data comparison parameters corresponding to each to-be-determined recommended classification type; performing difference calculation between the allowable deviation data comparison parameters corresponding to each to-be-determined recommended classification type and the comparison standard values ​​corresponding to each to-be-determined recommended classification type stored in the database to obtain the deviation value corresponding to each to-be-determined recommended classification type; comparing the deviation value corresponding to each to-be-determined recommended classification type with each deviation value corresponding to each to-be-determined recommended classification type stored in the database to obtain the allowable deviation data of retrieval situation corresponding to the deviation value corresponding to the to-be-determined recommended classification type stored in the database that has been successfully compared (for example, if the deviation values ​​stored in the database are 1, 2 and 3, and the calculated deviation value is 2, then the allowable deviation data of retrieval situation corresponding to 2 that has been successfully compared is obtained).

[0061] The retrieval situation tolerance data stored in the database is set based on historical performance data, expected business needs, and system capabilities. First, analyze historical operation data to understand the performance of different classification types in actual operation, including response time, success rate, CPU and bandwidth utilization, etc. Next, consider business needs and system design goals to determine the ideal or acceptable range for each performance indicator. Before these ranges are assessed as tolerances, they are usually reviewed by experts or verified using simulation data to ensure that they are both realistic and meet business goals. Finally, these tolerance values ​​are standardized and stored in the database for subsequent performance monitoring and management to ensure that the system operates stably within these preset parameters. In this way, the retrieval situation tolerance data helps manage system performance and prevent unexpected performance fluctuations, while also providing a basis for adjustment and optimization.

[0062] First, the archive storage data for each recommended classification type to be determined is collected, including the storage quantity, total storage space, and data transmission rate. These data provide preliminary information about the efficiency and capacity of archive storage. The collected data is fused and processed to generate the allowable deviation data comparison parameters representing the storage efficiency and performance of each classification type. The obtained comparison parameters are calculated with the standard values ​​preset in the database. This step is to find the deviation between the actual storage situation and the expected standard. The calculated deviation value is compared with the deviation value preset in the database.

[0063] The calculation formula for the allowable deviation data comparison parameter is as follows:

[0064]

[0065] In the formula, j is the number of the recommended classification type to be determined, YB j is the allowable deviation data comparison parameter corresponding to the jth recommended classification type to be determined, Sd j is the number of archives stored for the jth recommended classification type to be determined, ck j is the total storage space of the archives corresponding to the jth recommended classification type to be determined, cL j is the data transmission rate corresponding to the jth recommended classification type to be determined, α 1 is the weight factor of the number of archives stored in the database, α 2 is the total storage space weight factor of the archives stored in the database, α 3 is the data transmission rate weight factor stored in the database, and e is a natural constant.

[0066] In this implementation scheme, the weight factors are obtained through historical data, which usually involves statistical analysis and machine learning techniques. First, past performance data is collected, including data on the number of archive storage, storage space, data transmission rate, and system operation efficiency and stability. Then, multiple regression analysis is used to analyze the actual impact of these indicators on system performance, and it is possible to identify which indicators have the greatest impact on performance, and determine the weight of each indicator accordingly. These weight factors will then be used to convert historical data into input parameters in a predictive model or decision support tool to optimize future performance and resource allocation strategies. By combining the three key performance indicators, the performance and resource requirements of a classification type can be comprehensively evaluated to ensure that decisions are based on comprehensive data analysis.

[0067] Specifically, the archives to be uploaded that are allowed to be uploaded are searched for archives under the final recommended classification type corresponding to the determined archives in the database, and it is determined whether there are archives to be removed under the final recommended classification type, including the following steps: obtaining the historical access data of each archive under the final recommended classification type, and the historical access data includes the number of accesses, the average opening duration, and the proportion of duplicate content. Whenever a user or system accesses a certain archive, the system automatically records a access, which is implemented through middleware or database triggers. Each time a file is opened, the corresponding access counter is increased by one. The number of accesses is usually stored in the database and associated with the metadata of the corresponding archive. Track the timestamps of each time the archive is opened and closed. When the archive is opened, a timestamp is recorded, and when the archive is closed, another timestamp is recorded. The difference between the two is the duration of the access. The duration of all accesses of a single archive is accumulated and divided by the number of accesses to obtain the average opening duration. This data is also stored in the database and associated with the corresponding archive. The proportion of duplicate content can use natural language processing (NLP) tools or proprietary data comparison software to evaluate the repetition rate of content.

[0068] Obtain the review definition data of the archives corresponding to the final recommended classification type stored in the database. The review definition data includes the number of review definitions, the average open continuous definition time, and the duplicate content definition ratio. First, collect a large amount of historical review data, which reflects the performance of different archives in actual use. Perform statistical analysis on the collected data to determine the median of the number of reviews and the average open duration, which are defined as the number of review definitions and the average open continuous definition time. The duplicate content ratio of the archives corresponding to the median of the number of reviews and the average open duration is defined as the duplicate content definition ratio.

[0069] The historical review data of each file under the final recommended classification type is integrated with the review definition data corresponding to the final recommended classification type stored in the database to obtain the historical review evaluation value of each file; the historical review evaluation value of each file is compared with the historical review evaluation threshold corresponding to the final recommended classification type stored in the database to determine whether the historical review evaluation value of each file is less than the historical review evaluation threshold corresponding to the pre-set recommended classification type stored in the database: if the historical review evaluation value is less than the historical review evaluation threshold corresponding to the pre-set recommended classification type stored in the database, the file corresponding to the historical review evaluation value needs to be removed; if the historical review evaluation value is not less than the historical review evaluation threshold corresponding to the pre-set recommended classification type stored in the database, the file corresponding to the historical review evaluation value does not need to be removed.

[0070] The historical review evaluation threshold corresponding to the final recommended classification type stored in the database is obtained through historical data analysis, which includes collecting and evaluating a large amount of data on the historical review behavior of the relevant classification. The analyst will calculate statistical indicators such as the mean, median, standard deviation, etc. of the number of views, average opening duration, and proportion of duplicate content, and consider business needs and actual usage to set the review evaluation threshold suitable for the classification.

[0071] Collect historical access data for each archive, including the number of accesses, average opening duration, and the proportion of duplicate content. These data provide an intuitive understanding of the frequency of archive use and the user's interest in the archive content. Set access standards (definition data) for archives under each category, such as the threshold for the number of accesses, the minimum requirement for the average opening time, and the acceptable proportion of duplicate content. Combine the collected historical access data with the access definition data, perform fusion processing, and generate a historical access evaluation value for each archive. Compare the calculated historical access evaluation value with the preset threshold. If the evaluation value of a certain archive is lower than the threshold, it means that the actual access situation of the archive has not reached the expected frequency of use or attention, so it is considered no longer needed or outdated and recommended for removal. Removing archives with low usage frequency or high duplication can free up valuable storage space and make room for more active or important content. By reducing redundant content in the database, retrieval efficiency can be improved, allowing users to find the information they need more quickly.

[0072] The calculation formula for the historical review valuation is as follows:

[0073]

[0074] In the formula, f is the file number, LCo f is the historical review evaluation value of the f-th file, Sc f is the number of times the f-th file is viewed, Dc f is the average opening duration of the f-th file, cZ f is the percentage of duplicate content in the f-th file, Ds is the number of access definitions, DD is the average open continuous definition time, ScZ is the percentage of duplicate content definitions, and e is a natural constant.

[0075] In this implementation, the number of accesses directly reflects the popularity or frequency of use of the archive. A high number of accesses usually indicates that the archive content has a high value or demand for users. This is a direct indicator of the importance of the archive and is used to evaluate the attractiveness of the archive to the target user group. The average open duration measures the average time spent by users after opening the archive, reflecting the user's involvement and interest in the archive content. A longer open time may mean that the content is more fascinating or more informative. Together with the number of accesses, it helps to further understand the user's actual investment and value assessment of the archive content. A high content duplication ratio may indicate that there is a large amount of duplicate information in the archive, which may affect the uniqueness and value of the archive. This indicator is used to evaluate the uniqueness and novelty of the content, and is particularly important for maintaining high-quality and high-originality archives. The two parameters of the number of accesses and the average open duration are usually positively correlated, because users may spend more time reading or processing the archive if they find the content useful or interesting. In theory, if the content of an archive is highly duplicated, it may attract fewer repeated visits unless the duplicate information has important reference value. If the archive content is highly duplicated, users may close the archive in a shorter period of time because they may quickly find that the information does not provide new knowledge or data.

[0076] The first part of the formula is an exponential function, which evaluates the ratio of the number of times the archive is consulted and the average duration of opening to the set standard. Through the exponential function processing, it emphasizes the impact of the frequency and duration of consultation exceeding or failing to meet the standard on the evaluation value, so that a significant deviation of any indicator will significantly affect the evaluation result. The second part is processed by the hyperbolic tangent function, which takes into account the difference between the proportion of duplicate content in the archive and the set threshold. This part reflects the impact of the proportion of duplicate content on the value of the archive. Archives with high content duplication have lower value.

[0077] An intelligent archive management system based on big data, such as Figure 2 As shown, it includes an upload feasibility check module, an archive classification module and an existing archive processing module, wherein: the upload feasibility check module is used to verify the identity of the archive uploader, and after the verification, the upload feasibility check is performed on the archive to be uploaded. If the upload feasibility check fails, the upload is not allowed, and if the upload feasibility check is successful, the upload is allowed; the archive classification module is used to intelligently classify the archives to be uploaded that are allowed to be uploaded, and determine the pre-set recommended classification type corresponding to the archives to be uploaded that are allowed to be uploaded in the database; the existing archive processing module is used to perform archive retrieval on the archives under the pre-set recommended classification type corresponding to the determined archives to be uploaded that are allowed to be uploaded in the database, and determine whether there are archives that need to be removed. If so, the archives to be removed are backed up and removed.

[0078] An electronic device includes: a processor and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the above-mentioned intelligent archive management method based on big data.

[0079] A computer-readable storage medium is used to store a program, and when the program is executed by a processor, the above-mentioned intelligent archive management method based on big data is implemented.

[0080] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0084] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0085] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent archive management method based on big data, characterized in that: The following steps are involved: The identity of the person uploading the file is verified. After the verification is passed, the feasibility check of uploading the file to be uploaded is carried out. If the feasibility check fails, uploading is not allowed. If the feasibility check succeeds, uploading is allowed. Intelligently classify the files to be uploaded that are allowed to be uploaded, and determine the final recommended classification type corresponding to the files to be uploaded that are allowed to be uploaded in the database; Performing file search for the files under the final recommended classification type corresponding to the determined files to be uploaded and allowed to be uploaded in the database, determining whether there are files to be removed under the final recommended classification type, and if so, backing up and removing the files to be removed; The method of intelligently classifying the files to be uploaded that are allowed to be uploaded and determining the final recommended classification type corresponding to the files to be uploaded that are allowed to be uploaded in the database includes the following steps: Extract keywords from the referenced content of the archives to be uploaded that are allowed to be uploaded, and obtain a keyword set of the referenced documents; Extract keywords from the image captions of the archives to be uploaded that are allowed to be uploaded, and obtain an image caption keyword set; Obtain the ratio of structured content to unstructured content of the archives to be uploaded that are allowed to be uploaded; Obtaining a content keyword reference set, a reference document keyword reference set, a caption keyword reference set, and a reference ratio of structured content to unstructured content of the archive to be uploaded corresponding to each classification type stored in the database; The content keyword set, reference document keyword set, caption keyword set, ratio of structured content to unstructured content of the archives to be uploaded that are allowed to be uploaded, the content keyword reference set, reference document keyword reference set, caption keyword reference set and the ratio of structured content to unstructured content corresponding to each classification type stored in the database are integrated to obtain the classification matching index corresponding to each classification type; The calculation formula of classification matching index is as follows: In the formula, i is the number of the classification type stored in the database, Fz i is the classification matching index corresponding to the i-th classification type, A1 is the content keyword set of the files to be uploaded that are allowed to be uploaded, B1 is the reference document keyword set of the files to be uploaded that are allowed to be uploaded, C1 is the caption keyword set of the files to be uploaded that are allowed to be uploaded, BL is the ratio of structured content to unstructured content of the files to be uploaded that are allowed to be uploaded, A 2i is the content keyword reference set corresponding to the i-th classification type stored in the database, B 2i is the reference document keyword reference set corresponding to the i-th classification type stored in the database, C 2i is the reference set of caption keywords corresponding to the i-th classification type stored in the database, cBL i is the reference ratio of structured content to unstructured content corresponding to the i-th classification type stored in the database, e is a natural constant, σ(A1,A 2i ) is the similarity between the content keyword set and the reference document keyword reference set corresponding to the i-th classification type stored in the database, σ(B1,B 2i ) is the similarity between the reference document keyword set and the reference document keyword reference set corresponding to the i-th classification type stored in the database, σ(C1,C 2i ) is the similarity between the caption keyword set and the caption keyword reference set corresponding to the i-th classification type stored in the database; Arrange the classification matching indexes in ascending order, and based on the number of selections N set in the database, obtain the classification types corresponding to the top N classification matching indexes, and record them as the recommended classification types to be determined; Determine a classification type from the recommended classification types to be determined as the final recommended classification type; The step of determining a classification type from the recommended classification types to be determined as the final recommended classification type comprises the following steps: Pre-store the files to be uploaded that are allowed to be uploaded into each recommended classification type to be determined; Retrieving the pre-stored files to be uploaded that are allowed to be uploaded from each of the recommended classification types to be determined, and obtaining actual retrieval situation data corresponding to each of the recommended classification types to be determined, wherein the actual retrieval situation data includes average response time, retrieval success rate, average CPU usage rate, and average bandwidth utilization rate; Obtaining the parameter retrieval situation data corresponding to each to-be-determined recommended classification type stored in the database, wherein the parameter retrieval situation data includes the average parameter response time, the parameter retrieval success rate, the average CPU parameter utilization rate, and the average broadband parameter utilization rate; Obtain the allowable deviation data of the retrieval situation corresponding to each recommended classification type to be determined, wherein the allowable deviation data of the retrieval situation includes the allowable deviation of the average response time, the allowable deviation of the retrieval success rate, the allowable deviation of the average CPU usage rate, and the allowable deviation of the average bandwidth utilization rate; The actual retrieval situation data corresponding to each to-be-determined recommended classification type, the parameter retrieval situation data corresponding to each to-be-determined recommended classification type stored in the database, and the retrieval situation allowable deviation data corresponding to each to-be-determined recommended classification type are integrated to obtain the retrieval situation evaluation value corresponding to each to-be-determined recommended classification type; Obtaining a minimum retrieval situation evaluation value, and determining the to-be-determined recommended classification type corresponding to the minimum retrieval situation evaluation value as the final recommended classification type; The calculation formula of the evaluation value of the retrieval situation is as follows: In the formula, j is the number of the recommended classification type to be determined, TS is the average response time, Ds is the call success rate, CPs is the average CPU usage, Ks is the average bandwidth utilization, Tc j Dc is the average response time corresponding to the jth recommended classification type to be determined stored in the database, j is the parameter retrieval success rate corresponding to the jth recommended classification type to be determined stored in the database, CPc j Kc is the average CPU usage corresponding to the jth recommended classification type to be determined stored in the database, j is the average utilization rate of broadband parameters corresponding to the jth recommended classification type to be determined stored in the database, ΔTc j ΔDc is the average response time tolerance corresponding to the jth recommended classification type to be determined stored in the database, j ΔCPc is the allowable deviation of the retrieval success rate corresponding to the jth recommended classification type to be determined stored in the database, j ΔKc is the allowable deviation of the average CPU usage corresponding to the jth recommended classification type to be determined stored in the database, j Dp is the allowable deviation of the average bandwidth utilization corresponding to the jth recommended classification type to be determined stored in the database, j is the evaluation value of the retrieval situation corresponding to the jth recommended classification type to be determined; The step of obtaining the allowable deviation data of the retrieval situation corresponding to each recommended classification type to be determined includes the following steps: Acquire the archive storage data under each to-be-determined recommended classification type, wherein the archive storage data includes the number of archive storage, the total archive storage space and the data transmission rate; Perform fusion processing on the archive storage situation data under each recommended classification type to be determined, and obtain the allowable deviation data comparison parameters corresponding to each recommended classification type to be determined; Calculate the difference between the allowable deviation data comparison parameter corresponding to each recommended classification type to be determined and the comparison standard value corresponding to each recommended classification type to be determined stored in the database to obtain the deviation value corresponding to each recommended classification type to be determined; Compare the deviation values ​​corresponding to each to-be-determined recommended classification type with the deviation values ​​corresponding to each to-be-determined recommended classification type stored in the database, and obtain the retrieval situation allowable deviation data corresponding to the deviation values ​​corresponding to the to-be-determined recommended classification type stored in the database that have been successfully compared; The calculation formula of the allowable deviation data comparison parameter is as follows: In the formula, j is the number of the recommended classification type to be determined, YB j is the allowable deviation data comparison parameter corresponding to the jth recommended classification type to be determined, Sd j is the number of archives stored for the jth recommended classification type to be determined, ck j is the total storage space of the archives corresponding to the jth recommended classification type to be determined, cL j is the data transmission rate corresponding to the jth recommended classification type to be determined, α1 is the weight factor of the number of archives stored in the database, α2 is the weight factor of the total storage space of archives stored in the database, α3 is the weight factor of the data transmission rate stored in the database, and e is a natural constant; The step of searching for files under the final recommended classification type corresponding to the determined files to be uploaded and allowed to be uploaded in the database to determine whether there are files to be removed under the final recommended classification type includes the following steps: Obtaining historical access data of each archive under the final recommended classification type, wherein the historical access data includes the number of accesses, average opening duration, and percentage of duplicate content; Obtaining the review definition data of the archive corresponding to the final recommended classification type stored in the database, wherein the review definition data includes the number of review definitions, the average continuous opening definition time, and the percentage of repeated content definitions; The historical review data of each file under the final recommended classification type is integrated with the review definition data corresponding to the final recommended classification type stored in the database to obtain the historical review evaluation value of each file; Compare the historical review evaluation value of each file with the historical review evaluation threshold corresponding to the final recommended classification type stored in the database to determine whether the historical review evaluation value of each file is less than the historical review evaluation threshold corresponding to the preset recommended classification type stored in the database: If the historical review evaluation value is less than the historical review evaluation threshold corresponding to the preset recommendation classification type stored in the database, the file corresponding to the historical review evaluation value needs to be removed; If the historical review evaluation value is not less than the historical review evaluation threshold corresponding to the preset recommendation classification type stored in the database, the file corresponding to the historical review evaluation value does not need to be removed; The calculation formula of the historical review evaluation value is as follows: In the formula, f is the file number, LCo f is the historical review evaluation value of the f-th file, Sc f is the number of times the f-th file is viewed, Dc f is the average opening duration of the f-th file, cZ f is the percentage of duplicate content in the f-th file, Ds is the number of access definitions, DD is the average open continuous definition time, ScZ is the percentage of duplicate content definitions, and e is a natural constant.

2. The intelligent archive management method based on big data according to claim 1 is characterized in that: The identity of the person uploading the file is verified, and after the verification is passed, the feasibility check of uploading the file to be uploaded is performed, including the following steps: Obtain the identity information of the person who uploaded the file and verify the identity information of the person who uploaded the file. The identity information of the person who uploaded the file includes the account number, password and employee number of the person who uploaded the file: If the identity information of the person uploading the file is not the corresponding person stored in the database, the verification will fail and a login prompt will be given for the unauthorized person; If the identity information of the person uploading the file is the corresponding person stored in the database, the verification is successful; Receive files to be uploaded, extract keywords from the uploaded files, and obtain a set of content keywords; Compare the obtained content keyword set with the banned word set stored in the database: If the content keyword set and the banned word set intersect, the upload feasibility check fails and uploading is not allowed; If there is no intersection between the content keyword set and the banned word set, the upload feasibility check is successful and uploading is allowed.

3. An intelligent archive management system based on big data, used in the intelligent archive management method based on big data as claimed in any one of claims 1 to 2, characterized in that: It includes upload feasibility check module, file classification module and existing file processing module, among which: The upload feasibility check module is used to verify the identity of the person uploading the file. After the verification, the upload feasibility check is performed on the file to be uploaded. If the upload feasibility check fails, the upload is not allowed. If the upload feasibility check succeeds, the upload is allowed. The file classification module is used to intelligently classify the files to be uploaded that are allowed to be uploaded, and determine the corresponding pre-set recommended classification type in the database for the files to be uploaded that are allowed to be uploaded; The existing archive processing module is used to perform archive retrieval on the archives under the preset recommended classification type corresponding to the determined archives to be uploaded and allowed to be uploaded in the database, and determine whether there are archives to be removed. If so, the archives to be removed are backed up and removed.

Images