File receiving and reorganizing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a file receiving and reorganizing method and device, equipment and a storage medium. The method comprises the following steps: acquiring archive data, identifying, analyzing and uniformly formatting the archive data, screening the archive data according to a preset rule, and acquiring preprocessed archive data; and extracting metadata in the preprocessed archive data, matching the metadata with a metadata template in the format library, and storing the preprocessed archive data in a reorganization library if no metadata is missing in the preprocessed archive data. And performing four-property detection on the reorganized archive data in the reorganized library. If the reorganized archive data passes the four-property detection, the reorganized archive data is stored in an archive library through distributed storage, and if the reorganized archive data does not pass the four-property detection, the position of an error in the reorganized archive data is recorded, and a repair suggestion is provided. In this way, the efficiency and quality of archive reorganization can be improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of archive management, and in particular to an archive receiving and compiling method, device, equipment and storage medium. Background Art

[0002] As an important carrier of information and a voucher of historical records, the standardized management of archives is of great significance to the operation, decision-making and cultural inheritance of institutions. In today's era of rapid development of digital information, the number of archives generated by various institutions and organizations has exploded, and traditional archive management methods can no longer meet the needs of efficient and accurate archive processing. The existing archive receiving and reorganization process often has problems such as cumbersome procedures, easy errors in manual operations, difficulty in ensuring data consistency and integrity, insufficient compatibility with archives of different sources and types, and insufficient intelligence. For example, in the data supplementation and four-property detection (authenticity, integrity, availability, and security) links, manual operations are not only inefficient, but also prone to inaccurate archive data or detection omissions due to subjective factors. In addition, for archives from third-party archive production systems and some manually entered archives, there is a lack of a unified and intelligent integration and processing mechanism, which affects the overall quality and efficiency of archive management.

[0003] Therefore, how to improve the efficiency and quality of archive management and ensure the consistency, integrity and security of archive data has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] According to the embodiments of the present application, a method, device, equipment and storage medium for receiving and compiling archives are provided, which can ensure the consistency, integrity and security of archive data and greatly improve the efficiency and quality of archive compilation.

[0005] In a first aspect of the present application, a method for receiving and compiling archives is provided. The method comprises: Obtain archival data, identify, parse and uniformly format the archival data, and filter the archival data according to preset rules to obtain pre-processed archival data; Extract metadata from pre-processed archive data, match the metadata with the metadata template in the format library, and if there is no metadata missing in the pre-processed archive data, save the pre-processed archive data in the compilation library; Conduct four-property detection on the compiled archive data in the compilation library, including hash detection, metadata logic detection, format specification detection and personalization detection; If the compiled archive data passes the four-property test, the compiled archive data will be saved in the archive library through distributed storage. If the compiled archive data fails the four-property test, the location of the error in the compiled archive data will be recorded and repair suggestions will be provided.

[0006] In a possible implementation, metadata in the pre-processed archive data is extracted, the metadata is matched with a metadata template in a format library, and if there is no metadata missing in the pre-processed archive data, the pre-processed archive data is saved in an editing library, further comprising: If metadata is missing in the pre-processed archive data, relevant personnel will be reminded to complete the data, and when relevant personnel complete the data manually, intelligent prompts will be given based on the pre-processed archive data that has been entered.

[0007] In a possible implementation, performing hash detection on the compiled archive data in the compilation library includes: Performing a hash operation on the preprocessed archive data to generate a first hash value; Performing a secondary hash operation on the compilation archive data stored in the compilation library to generate a second hash value; The first hash value is compared with the second hash value to determine the integrity of the compiled archive data.

[0008] In a possible implementation, metadata logic detection and format specification detection are performed on the compiled archive data in the compilation library, including: Detect the format and logical relationship of metadata in compiled archival data; Perform corresponding format specification detection according to the file type of the compiled archive data.

[0009] In a possible implementation, the personalized detection is performed on the compilation archive data in the compilation library, including: Obtain a preset personalized detection data category, and perform personalized detection on the compiled archive data according to the type of the compiled archive data, the detection priority of the preset personalized detection data category, and the preset personalized detection data category.

[0010] In a possible implementation, the method further includes: Use text analysis algorithms and image recognition algorithms to intelligently classify and subject-label the text data and image data in the compiled archive data; Identify the compiled archival data according to its category and subject, establish archival associations, and build a complete archival chain.

[0011] In a possible implementation, the method further includes: During the archival data storage process, real-time monitoring of system resource usage; Dynamically adjust the detection parameters and CPU usage of each part according to the processing progress, data quality and system resource usage of the archive data.

[0012] In a second aspect of the present application, a device for receiving and compiling archives is provided. The device comprises: The acquisition module is used to acquire archival data, identify, parse and uniformly format the archival data, and filter the archival data according to preset rules to acquire pre-processed archival data; The first saving module is used to extract metadata from the pre-processed archive data, match the metadata with the metadata template in the format library, and if there is no metadata missing in the pre-processed archive data, save the pre-processed archive data in the compilation library; A detection module is used to perform hash detection, metadata logic detection, format specification detection and personalized detection on the compiled archive data in the compilation library in sequence; The second saving module saves the compiled archive data to the archive library through distributed storage if the compiled archive data passes the four-property test; if the compiled archive data fails the four-property test, the error location in the compiled archive data is recorded and repair suggestions are provided.

[0013] In a third aspect of the present application, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0014] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present application is implemented.

[0015] The archive receiving and compiling method provided in the embodiment of the present application, by acquiring archive data, identifies, parses and uniformly formats the archive data, and screens the archive data according to preset rules to obtain pre-processed archive data. Extract metadata in the pre-processed archive data, match metadata with metadata templates in the format library, if there is no metadata missing in the pre-processed archive data, save the pre-processed archive data in the compiling library. Four-property detection is performed on the compiled archive data in the compiling library, and the four-property detection includes hash detection, metadata logic detection, format specification detection and personalized detection. If the compiled archive data passes the four-property detection, the compiled archive data is saved in the archive library through distributed storage, if the compiled archive data does not pass the four-property detection, the wrong position in the compiled archive data is recorded, and repair suggestions are provided. The present application realizes intelligent archive compilation and preservation from the acquisition, processing, detection to storage of archive data, reduces manual intervention, improves the efficiency and accuracy of archive compilation, and uses hash detection, metadata logic detection, format specification detection and personalized detection to ensure the integrity, availability, security and reliability of archive data.

[0016] It should be understood that the contents described in the Summary of the Invention are not intended to limit the key or important features of the embodiments of the present application, nor are they intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein: Figure 1 is a flow chart of a file receiving and compiling method according to an embodiment of the present application; Figure 2 is a schematic diagram of an archive data detection interface according to an embodiment of the present application; Figure 3 A schematic diagram of the structure of file compilation management according to an embodiment of the present application; Figure 4 A block diagram of a file receiving and compiling device according to an embodiment of the present application; Figure 5 It is a schematic diagram of the structure of a terminal device or a server suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0019] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0020] Figure 1 FIG. 1 is a flowchart of a method for receiving and compiling archives according to an embodiment of the present disclosure. Figure 1 , the method comprising: S101, obtaining archival data, identifying, parsing and uniformly formatting the archival data, and screening the archival data according to preset rules to obtain pre-processed archival data.

[0021] The sources of archival data include but are not limited to manual entry and third-party archival systems, and the formats of archival data from third-party archival systems include but are not limited to JSON, XML, and CSV. For archival data from third-party archival systems, they are identified, parsed, and uniformly formatted according to the preset field mapping rules corresponding to the format of the third-party archival system, and converted into a unified internally recognizable standard format.

[0022] In addition, the archive data can be screened according to preset rules to remove obviously erroneous or incomplete archive data, improve the efficiency of subsequent archive data processing, and obtain pre-processed archive data. For example, the preset rules can be whether key fields are missing in the archive data, such as the archive number, archive creation time, and archive recorder. The preset rules can also be whether the field type and format of the archive data are correct, such as the date format, email format, and archive number format.

[0023] In this embodiment, archival data from different sources can be identified, parsed and uniformly formatted, and archival data can be screened according to preset rules, providing a basis for subsequent archival compilation.

[0024] S102, extracting metadata from the pre-processed archive data, matching the metadata with the metadata template in the format library, and if there is no metadata missing in the pre-processed archive data, saving the pre-processed archive data in the compilation library.

[0025] Among them, the metadata in the preprocessed archival data includes but is not limited to the source, creation time and data type description of the archival data. By matching the metadata in the extracted preprocessed archival data with the metadata template in the format library, the key information in the metadata is verified to ensure its integrity and accuracy, so as to assist in determining the format of the data and the subsequent parsing method. For example, check whether the creator information format of the metadata matches a known third-party system, or whether the timestamp format of the metadata conforms to the specific format set in the metadata template. In addition, the metadata matching process is not just a simple field correspondence, but also includes data type verification, data format conversion (normalization), and the most critical mandatory missing check. For example, the date field in the metadata needs to be converted from "January 15, 2023" to the format of "2023-01-15", the title field in the metadata needs to be reorganized according to the fixed pattern of "year + institution name + file type", and the archive number in the metadata cannot be missing.

[0026] In this embodiment, by matching with the metadata template in the format library, the standardization and consistency of the archive data during storage are ensured.

[0027] Optionally, extracting metadata from the preprocessed archive data, matching the metadata with a metadata template in a format library, and if there is no metadata missing in the preprocessed archive data, saving the preprocessed archive data in an editing library, further comprising: If metadata is missing in the pre-processed archive data, relevant personnel will be reminded to complete the data, and when relevant personnel complete the data manually, intelligent prompts will be given based on the pre-processed archive data that has been entered.

[0028] For example, there is a medical record data. The metadata template of the medical record data in the format library contains the patient's basic information, medical history, diagnosis results, etc. After matching, it is found that the metadata of "allergy history" and "family medical history" of the medical record data are partially missing. In this case, relevant personnel are reminded to complete the missing metadata parts and provide intelligent prompts based on the historical input data of the metadata template. The intelligent prompt for "allergy history" can be "cold air allergy", etc., and the intelligent prompt for "family medical history" can be "heart disease", "hypertension", etc.

[0029] In this embodiment, for pre-processed archival data with missing metadata, relevant personnel are promptly reminded to complete the data, and intelligent prompts are provided, providing high-quality assistance to relevant personnel, thereby completing the data supplement work quickly and accurately.

[0030] S103, performing four-property detection on the compiled archive data in the compilation library, the four-property detection including hash detection, metadata logic detection, format specification detection and personalization detection.

[0031] In this embodiment, the authenticity, integrity, availability and security of the compiled archival data in the compilation library are ensured by performing hash detection, metadata logic detection, format specification detection and personalized detection in sequence, and personalized intelligent customized detection is provided for users.

[0032] Optionally, hash detection is performed on the compiled archive data in the compilation library, including: Performing a hash operation on the preprocessed archive data to generate a first hash value; Performing a secondary hash operation on the compilation archive data stored in the compilation library to generate a second hash value; The first hash value is compared with the second hash value to determine the integrity of the compiled archive data.

[0033] Hash operation is a mathematical algorithm that converts input data of any length (such as strings, files, etc.) into an output value of fixed length (called a hash value or digital fingerprint). In hash operation, the same input data always produces the same hash value, and the original input data cannot be deduced from the hash value, which ensures the security of the data and can effectively verify whether the input data has been tampered with.

[0034] In the present invention, the hash operation algorithms used include but are not limited to MD5, SHA-1, and SHA-256. Among them, MD5 (Message-Digest Algorithm 5) is a hash algorithm that generates a 128-bit (16-byte) hash value. SHA-1 (Secure Hash Algorithm 1) is a hash algorithm that generates a 160-bit (20-byte) hash value. SHA-256 (Secure Hash Algorithm 256) is a hash algorithm that generates a 256-bit (32-byte) hash value.

[0035] In a possible implementation, first, the pre-processed archive data is input into a hash operation algorithm to generate a first hash value. Then, the compiled archive data stored in the compilation library is input into the same hash operation algorithm to generate a second hash value. Since the hash value has a high degree of uniqueness and irreversibility, even a slight change to the input data will result in a significant change in the hash value. Finally, the first hash value and the second hash value are compared. If the first hash value and the second hash value are consistent, it indicates that the archive data has not been tampered with and the integrity has been verified. On the contrary, if the first hash value and the second hash value are inconsistent, it indicates that the archive data may have been modified and there is a risk of tampering with the archive data. Through this integrity verification method based on hash operation, the integrity and authenticity of the archive data can be efficiently and accurately detected.

[0036] In this embodiment, hash detection can effectively determine whether the archive data is complete and has not been tampered with, thereby ensuring the integrity and reliability of the archive data.

[0037] Optionally, metadata logic detection and format specification detection are performed on the compiled archive data in the compilation library, including: Detect the format and logical relationship of metadata in compiled archival data; Perform corresponding format specification detection according to the file type of the compiled archive data.

[0038] Among them, metadata includes but is not limited to the creation time, creator, subject and category of the compiled archival data. The format and logical relationship of the metadata in the compiled archival data are detected, including but not limited to whether the required fields of the metadata are empty, whether the data type complies with the regulations, whether the date format is correct, and whether the logical relationship between different metadata is reasonable. For example, the file number in the metadata cannot be empty, the data type of the number of archive pages in the metadata should be an integer, the date format in the metadata should be "YYYY-MM-DD", and the signing date in the metadata of the contract archival data should be later than the drafting date. By detecting the integrity and consistency of the metadata, it is ensured that all relevant metadata exist and match each other. In addition, the detection of the logical relationship of the metadata can verify the authenticity and integrity of the archives from the metadata level. In addition, special metadata logical analysis rules can be set according to the needs of the archival data to improve the pertinence and accuracy of the detection.

[0039] The file types of the compiled archival data include but are not limited to PDF, DOCX and JPG. The corresponding format specification detection based on the file type of the compiled archival data includes but is not limited to the file structure, encoding method and element composition corresponding to the file type. For example, for the compiled archival data in PDF format, check whether its page layout, text typesetting and image quality meet the PDF format standard; for the compiled archival data in DOCX format, check whether its document structure, style setting and font information are correct; for the compiled archival data in XML format, use regular expressions (a data verification method for matching string patterns, composed of ordinary characters and special metacharacters, which can efficiently describe complex string rules) to match and verify key information in a specific format, check whether the tags in the compiled archival data are correctly closed, whether the attribute values ​​meet the regulations, etc. By performing corresponding format specification detection based on the file type, the correctness and consistency of the archive in format can be ensured, which helps to ensure the availability and long-term preservation of the compiled archival data, and avoid potential problems caused by format errors or non-standardization, such as the inability to open the file and abnormal content display. Furthermore, if a new file format for compiling archival data emerges, the existing format specification detection can be flexibly expanded and upgraded to meet the archival management needs of different periods.

[0040] In this embodiment, metadata logic detection and format specification detection are used to provide more comprehensive guarantees for the compilation, management and utilization of archival data, and customized detection is performed based on the characteristics of different archival data, thereby improving the pertinence and accuracy of the detection.

[0041] Optionally, the compiled archive data in the compilation library is subjected to personalized detection, including: Obtain a preset personalized detection data category, and perform personalized detection on the compiled archive data according to the type of the compiled archive data, the detection priority of the preset personalized detection data category, and the preset personalized detection data category.

[0042] The types of compiled archival data include but are not limited to text and image types. For compiled archival data of text type, attention should be paid to the authenticity and integrity of its information. For example, if the compiled archival data is a contract text, its archival document number, signing date, and signatures of all parties are the core elements to verify the validity of the compiled archival data. For compiled archival data of image type, attention should be paid to its quality in providing evidence and image details, such as the resolution, color mode, and image fidelity of photos or scanned images, to ensure that each image can be saved and managed with the highest resolution and correct color mode.

[0043] In the present application, the compiled archive data is personalized for detection according to the type of compiled archive data, the detection priority of the preset personalized detection data category, and the preset personalized detection data category. The detection category and priority can be customized according to specific needs, and high-risk content is detected first. For example, the type of compiled archive data is a text archive, and the detection priority of the preset personalized detection data category and the preset personalized detection data category are: sensitive information detection, priority 1; format standardization detection, priority 2; text error detection, priority 3. Then, sensitive information detection, format standardization detection, and text error detection are performed on the compiled archive data of the text archive type in sequence, wherein the sensitive information detection is to detect illegal words and personal sensitive information (telephone number, ID card number, etc.) in the text. For example, the type of compiled archive data is an image archive, and the detection priority of the preset personalized detection data category and the preset personalized detection data category are: clarity detection, priority 1; image size standardization detection, priority 2; image watermark detection, priority 3. Then, clarity detection, image size standardization detection, and image watermark detection are performed on the compiled archive data of the image archive type in sequence.

[0044] Figure 2 is a schematic diagram of an archive data detection interface according to an embodiment of the present application, such as Figure 2 As shown: In a possible implementation, the archive data detection interface can be set according to authenticity, integrity, availability and security. The authenticity detection includes the authenticity of the electronic archive source, the accuracy of the electronic archive metadata, the authenticity of the electronic archive content, the consistency of the metadata and content association, and the authenticity of the transfer information package. The integrity detection includes the total number of electronic archives, the integrity of the electronic archive metadata, the integrity of the electronic archive content, and the integrity of the transfer information package. The availability detection includes the availability of electronic archive metadata, the availability of electronic archive content, the hardware and software environment of the electronic archive, and the availability of the transfer information package. The security detection includes the transfer information package virus detection, the transfer carrier security, and the transfer process security. Users can set the items, priorities, and detailed parameters for archival data detection through personalized selection, and allow users to save and reuse customized detection schemes for direct use in subsequent archive detection. It can also collect user feedback information to optimize and improve the existing customized detection schemes, thereby improving the user's participation and satisfaction in archival data detection to meet the archival detection needs of different users and different business scenarios.

[0045] Furthermore, the authenticity of the source of electronic archives includes but is not limited to the validity test of fixed information, the accuracy of electronic archive metadata includes but is not limited to the detection of metadata item data length, metadata item data type format, metadata item value domain conformity test of set value domain, metadata item data value rationality test, metadata item data contains special characters test, file number standardization test, metadata item data duplication test, metadata item (archive number, catalog number and classification number) and archive consistency test, the authenticity of electronic archive content includes but is not limited to the electronic attribute consistency test of content data, the authenticity of transferred information package includes but is not limited to the standardization test of description file and catalog file, the standardization test of information package catalog structure, and the consistency test of information package, the total number of electronic archives includes but is not limited to the total number of pieces and the total number of bytes, the integrity of electronic archive metadata includes but is not limited to the integrity test of metadata items, the detection of metadata mandatory recording items, and the process information. The integrity of electronic archive content includes but is not limited to content data integrity detection and attachment data integrity detection. The integrity of the transferred information package includes but is not limited to information package content data integrity detection. The availability of electronic archive metadata includes but is not limited to metadata readability detection in the information package and metadata accessibility detection in the target database. The availability of electronic archive content includes but is not limited to content data format detection and content data readability detection. The software and hardware environment of electronic archives includes but is not limited to software and hardware compliance detection. The availability of transferred information package includes but is not limited to content data format compliance detection contained in the information package. Virus detection of transferred information package includes but is not limited to detection of whether antivirus software is installed in the system environment and virus infection detection. The security of the transferred carrier includes but is not limited to detection of redundant files in the carrier, carrier reading speed detection and carrier appearance detection. The security of the transfer process includes but is not limited to operation process security detection.

[0046] In this embodiment, customized detection of compiled archival data is achieved through personalized detection, in-depth detection is performed according to the needs of compiled archival data, the coverage and accuracy of detection are improved, and more powerful support is provided for the scientific management of archives.

[0047] S104: If the compiled archive data passes the four-property test, the compiled archive data is saved in the archive library through distributed storage; if the compiled archive data fails the four-property test, the location of the error in the compiled archive data is recorded and repair suggestions are provided.

[0048] The information stored in the archive includes but is not limited to archive data, metadata, test records and system configuration information. Distributed storage is used to save the compiled archive data in the archive, ensuring the security and reliability of the archive data, and it has efficient data retrieval and query functions, making it easy to obtain archive information at any time.

[0049] If the compiled archival data fails to pass the four-property test, the location of the error in the compiled archival data is recorded and repair suggestions are provided. Repair suggestions for compiled archival data of text type include but are not limited to grammatical error suggestions, semantic and logical error suggestions. First, grammatical error suggestions can be implemented through rule-based grammar checkers and pre-trained language models. For detected grammatical errors, corresponding modification suggestions are generated according to language rules and preset expressions, such as subject-verb inconsistency, improper wording, punctuation errors, etc. On the one hand, the rule-based grammar checker can identify common grammatical errors through a pre-defined language rule library. For example, if the sentence "We had a meeting yesterday" has the problem of improper tense use, it can be suggested to modify it to "We had a meeting yesterday". On the other hand, the pre-trained language model can be trained using a large-scale corpus, which can capture more complex grammatical structures and language patterns. It can not only identify simple grammatical errors, but also provide more accurate modification suggestions based on the context. For example, in the sentence "He went to the store to buy some fruit and went home to cook", the model will suggest adding "after" or "then" to enhance the coherence of the sentence through context prediction. By combining the rule-based grammar checker and the pre-trained language model, it can help discover grammatical problems in text-type compiled archival data at different levels and provide high-quality repair suggestions. Repair suggestions for non-text-type compiled archival data include but are not limited to image repair suggestions and audio repair suggestions. Possible problems in image-type compiled archival data include noise, blur, and partial area damage. For image-type repair suggestions, corresponding repair suggestions are generated according to the type and severity of the problem. For example, for noise problems, median filtering and Gaussian filtering denoising suggestions are provided according to different degrees of noise; for blur problems, image sharpening processing suggestions are provided, including but not limited to algorithms such as Laplace operator and Sobel operator; for partial area damage problems, suggestions for using image repair models based on texture synthesis or deep learning are provided. Possible problems in the compiled archive data of audio type include abnormal volume, noise, missing audio segments, etc. Similarly, repair suggestions for audio types will also generate corresponding repair suggestions according to the type and severity of the problem. For example, for the problem of too low volume, it is recommended to use the audio gain tool to appropriately increase the volume; for the noise problem, it is recommended to use audio noise reduction software, and recommend a suitable noise reduction algorithm based on the characteristics of the noise (such as white noise, periodic noise, etc.); for the problem of missing audio segments, it is recommended to restore it through methods such as audio splicing or interpolation.

[0050] In addition, before the compiled archival data is saved in the archive, it will be backed up. Once an abnormality occurs during the saving process, it can automatically roll back to the last stable state to prevent the compiled archival data from being damaged or lost. After the compiled archival data is saved in the archive, the archive index and directory information will be automatically updated, and a successful entry notification will be sent to relevant personnel, and detailed information such as the entry time and operator will be recorded for subsequent audit and tracking of archival data.

[0051] In a possible implementation, after receiving and compiling archival data, the archive can also analyze the value of the archival data, including but not limited to content relevance and importance, data integrity, accuracy, effectiveness, rarity, reusability and relevance. Through the six dimensions and their sub-indicators, the value of archival data is comprehensively and meticulously characterized, thereby quickly identifying the content type and potential value of archival data.

[0052] In terms of content relevance and importance, measure the relevance of the content in the archival data to a specific field, project or research topic, as well as the criticality of the information contained. First, evaluate the degree of match between the subject matter involved in the archival data and the pre-defined key subject areas. For example, in the management of corporate archival data, evaluate whether the archives whose subjects are related to the main business of the enterprise (such as electronic product research and development, production process, market sales, etc.) have a high relevance. Secondly, evaluate whether the archives contain information that is critical to a specific business or research. For example, whether medical archival data contains high-value archival content such as diagnostic details of rare diseases and new treatment trial data. These can be determined through natural language processing technologies such as named entity recognition (such as disease name, equipment model, etc.), keyword extraction (such as "safety factor", "treatment effect", etc.) or other data feature extraction methods.

[0053] In terms of data integrity, assess whether the archival data is complete and without missing information to ensure the comprehensiveness of the information. For text-based archival data, check whether there is any missing information. For example, a complete contract archival data should include information about both parties to the contract, contract terms, signatures and seals, etc. The integrity of the current text archival data can be judged by analyzing the text structure and grammatical rules of the archival data and learning the characteristics of the complete contract text-based archival data. For non-text archival data, such as charts in engineering design archives, it is necessary to check whether the legend is missing and whether the data series is complete. The integrity of the image is judged by detecting whether the resolution and color mode of the image meet the expected standards, and whether the image content is complete.

[0054] In terms of accuracy, the content of archival data is evaluated to ensure the reliability of information. Natural language processing technology can be used to check the grammar, spelling and semantic accuracy of text archival data. For example, in academic research archives, accurate descriptions of research methods, data citations and conclusions are crucial, and archival data with a large number of errors may be misleading, and the subsequent use of images has relatively low value. Then, for archival data containing multiple data forms (such as text and charts), it is necessary to check the consistency of the data. For example, in archival data of financial statements, the financial data mentioned in the text should be consistent with the data in the charts, otherwise, the inconsistent data will reduce the credibility and value of the archival data.

[0055] In terms of effectiveness, the update degree and validity period of archival data are evaluated to ensure that archival data keeps pace with the times. By comparing the newly received archival data with the existing archival data of the same type, it is determined whether it contains the latest information. In addition, some archival data also has specific time limits, such as design change notices valid within a specific project cycle, temporary construction plan adjustment documents, etc. The value of archival data within the validity period is higher, and the value of archival data after expiration will drop significantly.

[0056] In terms of rarity, the rarity of archival data is evaluated, and the rarer the archival data, the higher its value. For example, historical document archival data that records little-known social events or cultural phenomena in a specific period have a higher value due to their rarity.

[0057] In terms of reusability and relevance, evaluate the possibility of future application of the knowledge and experience of archival data and the degree of relevance to other archives. First, evaluate whether the knowledge, methods or technologies in the archival data can be reused in future projects or research. For example, in archival data of engineering design, common design standards, excellent design ideas and mature construction process records have high reusability and correspondingly high value. Then, by analyzing the associations between archival data, such as reference relationships, upstream and downstream relationships of project processes, etc., determine the value of archival data. For example, in a large engineering project, if an engineering design archive data is closely related to other existing project planning archive data, construction record archive data, etc., and can help improve the entire project archive system, then its value will increase.

[0058] In this embodiment, the archive data that has passed the four-property test is saved in the archive library, ensuring the quality of the archives stored in the library. In addition, for archive data that has not passed the four-property test, the location of the error in the compiled archive data is recorded and repair suggestions are provided to help relevant personnel quickly correct the error, thereby improving the efficiency of archive compilation.

[0059] Optionally, the method further comprises: Use text analysis algorithms and image recognition algorithms to intelligently classify and subject-label the text data and image data in the compiled archive data; Identify the compiled archival data according to its category and subject, establish archival associations, and build a complete archival chain.

[0060] According to the categories and themes of the compiled archival data, archives with related relationships, such as files at different stages of the same project or related records of the same event, are automatically identified and associated links are established through intelligent algorithms to form a complete archival chain.

[0061] Firstly, the text analysis algorithm and image recognition algorithm are used to perform subject labeling on the text data and image data in the compiled archival data respectively. For text data, word segmentation tools (including but not limited to jieba library) are used to decompose text data into individual words, and words of different parts of speech such as nouns (such as names of people, places, names of institutions, etc.), verbs, adjectives, etc. are identified. For example, in the meeting record archive data, relevant words such as "participants", "discussion topics", and "resolution results" are segmented. Then, the structure of the text data is analyzed to determine the relationship between words and the subject-predicate-object structure between sentences. For example, in the sentence "The project team discussed the R&D plan of the new product in the conference room", "project team" is the subject, "discussion" is the predicate, and "new product R&D plan" is the object. Secondly, key entities in the text data (names of people, places, organizations, etc.) are identified, and their roles in the event are understood. For example, in the news report archive data, "XX Company (organization name) held a new product launch conference (event) in XX City (place name) in XX month (time) of XX year" is accurately identified, thereby providing a basis for subsequent analysis. Finally, the topic model LDA (Linear Discriminant Analysis, Linear Discriminant Analysis) can be used to construct themes for text data and automatically summarize themes such as "business management", "technological innovation" and "cultural heritage", so as to grasp the content scope of text data at a macro level. Among them, LDA is a topic model that automatically discovers hidden topics in a document collection and their distribution in the document by analyzing the co-occurrence probability of words. It can represent text data as a mixture of topics and assign a word probability distribution to each topic. For image data, a convolutional neural network (CNN) is first used to extract image features, including but not limited to the color histogram features of the image data. Then, the edge contour features of the image are obtained through an edge detection algorithm. The image features and edge contour features extracted by VGG (Visual Geometry Group) are then used to mine more abstract and advanced image semantic features. For example, it is possible to identify whether the image data is mainly people, scenery, or specific objects, thereby performing subject labeling of the image data. Among them, CNN is a deep learning model that is good at processing image, video and other data. The edge detection algorithm identifies the edge contours of objects in the image data by detecting drastic changes in the grayscale values ​​of image pixels. VGG is a deep convolutional neural network model with a simple structure. Image features are extracted by stacking multiple convolutional layers and pooling layers.In addition, for audio data, you can first extract MFCC (Mel-scale Frequency Cepstral Coefficients) audio features, and then perform audio semantic analysis on them to achieve topic labeling of the audio data.

[0062] Secondly, the text classification algorithm and image classification algorithm are used to intelligently classify the text data and image data in the compiled archive data. For text data, the Bert model trained with historical archive data is used to map the text data into vector representation, and then the vector representation is output to the TextCNN model for classification prediction. For example, the text data is divided into categories such as "personnel files", "financial files", and "project files", so that the category to which the text data belongs can be accurately determined based on the feature vector of the text data, and intelligent automatic classification can be achieved. Among them, Bert is a Transformer-based model that obtains deep language feature representations from large-scale texts through self-supervised learning. TextCNN is a text classification model based on convolutional neural networks that can effectively analyze and complete classification tasks. For image data, the ResNet model trained with historical archive data can be used to extract image features and predict image categories. Among them, ResNet is a model that solves the gradient vanishing and degradation problems in deep networks by introducing residual connections. It can efficiently extract image features and accurately predict image categories. In addition, for audio data, ResNetSE trained with historical archive data extracts audio features and identifies the categories of audio data. The ResNetSE model combines the residual structure Squeeze-and-Excitation (SE) mechanism of the ResNet model, and improves the feature representation capability by adaptively adjusting the weights of feature channels. It can effectively extract audio features and accurately predict audio categories.

[0063] Finally, the compiled archival data is identified according to its category and theme, and archival associations are established, such as archival files of different stages of the same project or related archival records of the same event. By building a complete archival chain, it is integrated into a complete archival project collection, which is convenient for relevant personnel to understand the progress and results of the project. For example, experimental archival data with the same theme and type can be integrated into a research file, and then, according to the experimental start time and end time or date range marked in the experimental archival data, the experimental archival data under the same research file can be time-associated. In addition to the start time and end time of the experiment, the modification time of the experimental archival data can also be used as the basis for establishing the association of the experimental archival data or the basis for judging the core data. If the experimental archival data has been modified many times and the modification time runs through the entire research file cycle, then this experimental archival data may play a key role in the process of project advancement, and its stage may be the core experimental stage or the verification stage. Furthermore, in the overall planning documents of some scientific research projects, the project goals will be decomposed into multiple stage goals. If the experimental purpose described in the experimental archival data is closely matched with the goal of a certain stage, then this part of the experimental archival data can be divided into the corresponding stage. In addition, experimental archive data usually cites the results of previous experimental archive data as background or basis. By analyzing these citation relationships, the order and stage of the cited or cited experimental archive data can be determined. In addition, since the initial stage of the project mainly uses some conventional and basic experimental techniques to preliminarily explore the research object, more advanced and customized technologies will be introduced in the later stage to conduct in-depth research or verify specific hypotheses, then, by analyzing technical associations, the stage sequence of experimental archive data can also be sorted out. Furthermore, since the results and conclusions in the experimental report usually reflect the stage results of the project, the experimental archive data can also be divided into the corresponding stages of the project by identifying the iconic results.

[0064] In this embodiment, intelligent algorithms are used to classify and subject-label the compiled archival data, providing intelligent support for the classification and association of the compiled archival data, reducing manual intervention, and improving the efficiency and accuracy of archival processing.

[0065] Optionally, the method further comprises: During the archival data storage process, real-time monitoring of system resource usage; Dynamically adjust the detection parameters and CPU usage of each part according to the processing progress, data quality and system resource usage of the archive data.

[0066] In this application, in order to achieve transparency and controllability of system resources in the process of archive compilation, the system resource usage will be monitored in real time during the archival data storage process. For example, in the process of archival data preservation, the number and preservation rate of the saved archival data will be recorded; in the process of hash detection, metadata logic detection, format specification detection and personalized detection, the proportion of the detected archival data to the total archival data to be detected and the time consumption of each detection step will be recorded.

[0067] Adjust the working parameters and task allocation of each detection process in real time, and continuously monitor the effect of dynamic adjustment after execution. If it is found that the expected goal is not achieved, analyze and make decisions again according to the new situation to form a closed-loop control process with dynamic feedback. In this process, the cache size parameter in the archive data saving process is modified to control the saving rate, change the CPU occupancy rate of a certain detection process to adapt to archives of different quality levels, and reallocate system resources (such as adjusting the number of CPU cores allocated to each module, memory allocation space, etc.). For example, the disk write speed of the storage server is 50MB / s, the read speed is 20MB / s, the current available disk space is 2TB, the used space accounts for 80% of the total space, the CPU occupancy rate is 80%, the disk read and write rate is lower than the preset read and write threshold, and the CPU occupancy rate is higher than the CPU occupancy threshold, then temporarily reduce the receiving rate of the archive receiving module or suspend part of the detection process to reduce system pressure.

[0068] In addition, data quality is an important part of archive processing. In order to detect the authenticity, integrity, availability and security of archive data in real time, the authenticity of the archive can be judged by comparing with standard data samples, verifying the data format, etc. The integrity of the archive can be ensured by judging whether required fields are missing and whether the files are complete and intact. The availability of the archive can be considered by evaluating whether the archive can be opened normally and whether the content can be clearly identified. The security of the archive data can be guaranteed by detecting whether there is a risk of data leakage and whether it complies with confidentiality regulations.

[0069] In a possible implementation, the storage information of historical archive data (including but not limited to processing progress, data quality, and system resource usage in the corresponding period) is used as a training set to pre-train the long short-term memory network (LSTM), analyze the relationship between each detection task and resource consumption, and predict the changes in the amount of archive data received in the short term in the future, so as to estimate the resource requirements under different task allocations. Among them, LSTM is an improved recurrent neural network that is good at processing time series data and long dependencies. In addition, valuable features can be extracted from the storage information of historical archive data based on linear regression as input to the LSTM model. For example, extract data related to the progress of archive data processing, including but not limited to the number of processed archives, the number of remaining archives, and the average processing time in the current stage; extract archive data quality, including but not limited to the evaluation index values ​​of different quality dimensions and the frequency of specific quality problems; extract system resource status, including but not limited to the fluctuation range of CPU usage, memory idle rate, and disk read and write frequency. Through more targeted feature data, the LSTM model learns the relationship between these features and the amount of archive data received (such as subsequent processing progress, whether data quality problems will occur, whether resources will be overloaded, etc.), and continuously optimizes its own prediction ability. Then, according to the predicted changes in the amount of archival data received, further adjustments can be made. For example, if the predicted amount of archival data received will increase significantly in the next hour and the current network bandwidth resources are relatively tight, the frequency of archival data acquisition can be reduced, or more network resources can be temporarily allocated to the storage server. In addition, if it is found that the data quality of a certain type of archival data frequently has specific problems, the detection focus or detection parameters of this type of archival data will be adjusted to strengthen the investigation of the problem. If the overall CPU utilization is close to the preset threshold, it will trigger the reallocation of resources, give priority to the resource requirements of high-priority detection tasks or storage tasks, and suspend or delay some secondary detection or storage tasks.

[0070] In this embodiment, through intelligent regulation, the detection parameters and CPU occupancy rate of each part are dynamically adjusted according to the processing progress of the archive data, data quality and system resource occupancy, thereby achieving efficient operation of the archive receiving and compiling process.

[0071] Figure 3 Schematic diagram of the structure of the file compilation management according to the embodiment of the present application, such as Figure 3 As shown: In a possible implementation, in order to achieve the security, standardization and traceability of official documents and archives, the archive compilation process can be divided into four parts, including archive reception, archive compilation, archive detection and storage management. In the archive reception stage, the office system that submits the official document archives is used as a third-party archive source to interface with the archive management system. When a new official document archive is drafted, reviewed and published in the office system, the relevant official document archive data file and its metadata will be automatically parsed and transmitted to the archive management system. The system first converts and standardizes the archive files to ensure that they comply with the archive data management specifications, and extracts core information such as file number, publishing department and subject keywords. If it is found that the confidentiality level in the archive file is unclear or lacks key metadata such as the release date, the system will automatically notify the document drafter to supplement and improve it. For meeting minutes archive files that need to be collected manually, the archivist enters the meeting time, location, participants and main content of the meeting into the system according to the specified template. At the same time, during the input process, the archive management system uses the intelligent association function to automatically prompt possible follow-up content based on the information that has been entered, such as prompting possible discussion topics based on the departments and positions of the participants, thereby improving the efficiency and accuracy of archival data input. In the archive compilation stage, the archive data is first saved to the compilation library, and the official documents and meeting minutes are intelligently classified according to the preset archive classification system and subject word list. For example, policy documents related to economic development will be classified into economic policy folders and associated with relevant statistical data, research reports and other archives. For meeting minutes, they are classified according to the theme and level of the meeting. During the compilation process, the system automatically identifies the reference relationship in the file, such as laws and regulations or superior documents cited in policy documents, and links them together for easy access and reference. At the same time, version management is performed on the archive data, recording the revision time, content changes and reasons of different revisions of the same file to ensure the traceability of the archive data. In the archive detection stage, the main focus is on the authenticity, integrity, and security of the archive data. In the authenticity detection, the source of the release of the archive file and the legitimacy of the circulation process are verified by connecting with the authoritative release channel and the digital certificate authentication system. In the integrity test, we check whether the archive file format complies with the official document specifications and whether the content covers the necessary policy points and implementation details. In the security test, we use strict access control and encryption technology to ensure the confidentiality and integrity of the files during storage and transmission. For meeting minutes, we focus on testing the accuracy and completeness of their content to ensure that the key issues discussed at the meeting and the decision results are accurately recorded. In addition, for archive data that fails the test, we not only provide detailed error reports, but also initiate corresponding error correction and approval procedures according to archive management regulations and work processes.For example, for official documents with irregular formats, the archive management system will automatically return them to the submitter, attach a standardized format template and modification suggestions, and require resubmission for review. Finally, in the storage management stage, qualified archives will be stored and managed. In addition, according to the preset archive management authority settings, corresponding access rights are assigned to users of different levels. For example, ordinary staff can only view and download archives related to their own work, while senior managers can access a wider range of archive data resources. At the same time, the archive management system will regularly back up and archive the archive data in the warehouse, and store important archive data in an off-site backup center to prevent local data loss and cause significant losses. In addition, the archive management system also provides management functions for archival data borrowing and returning, and records information such as borrowers, borrowing time and borrowing purpose in detail to ensure the safe circulation and effective use of archival data. Through this series of rigorous management measures, the security, standardization and traceability of archival data are effectively guaranteed, the overall archive management level is improved, and intelligent archival data management is realized.

[0072] According to the embodiments of the present disclosure, the following technical effects are achieved: 1) The acquisition, processing, detection and storage of archive compilation are realized through intelligent means, which reduces labor costs and improves the efficiency and quality of archive compilation.

[0073] 2) Hash detection, metadata logic detection, format specification detection and personalized detection are used to detect the integrity, availability, security and reliability of archival data, and personalized customized services are provided to users.

[0074] 3) Analyze document data through intelligent algorithms and establish correlations between document data, which provides a basis for subsequent integrated analysis of archival data and facilitates researchers to fully understand the progress and achievements of archival data.

[0075] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0076] The above is an introduction to the method embodiment. The following is a further explanation of the scheme described in this application through an apparatus embodiment.

[0077] Figure 4 A block diagram of a file receiving and compiling device according to an embodiment of the present application is shown. Figure 4 Shown include: The acquisition module 401 is used to acquire archival data, identify, parse and uniformly format the archival data, and filter the archival data according to preset rules to acquire pre-processed archival data; The first saving module 402 is used to extract metadata from the pre-processed archive data, match the metadata with the metadata template in the format library, and save the pre-processed archive data to the compilation library if there is no metadata missing in the pre-processed archive data; The detection module 403 is used to perform four-property detection on the compiled archive data in the compilation library, and the four-property detection includes hash detection, metadata logic detection, format specification detection and personalized detection; The second saving module 404 saves the compiled archive data to the archive library through distributed storage if the compiled archive data passes the four-property test; if the compiled archive data fails the four-property test, the error position in the compiled archive data is recorded and repair suggestions are provided.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0079] Figure 5 A schematic diagram of the structure of a terminal device or server suitable for implementing an embodiment of the present application is shown.

[0080] like Figure 5 As shown, the terminal device or server includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the terminal device or server are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0081] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0082] In particular, according to an embodiment of the present application, the above method flow steps can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0083] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0084] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of the code, and the aforementioned module, program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0086] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the method described in the present application.

[0087] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned application concept. For example, the above features are replaced with (but not limited to) technical features with similar functions applied in the present application.

Claims

1. A method for receiving and compiling archives, characterized in that: include: Acquire archival data, identify, parse and uniformly format the archival data, and filter the archival data according to preset rules to acquire pre-processed archival data; Extracting metadata from the preprocessed archive data, matching the metadata with a metadata template in a format library, and if there is no metadata missing from the preprocessed archive data, saving the preprocessed archive data in an editing library; Performing four-property detection on the compiled archive data in the compilation library, wherein the four-property detection includes hash detection, metadata logic detection, format specification detection and personalization detection; If the compiled archive data passes the four-property test, the compiled archive data is saved in the archive library through distributed storage. If the compiled archive data fails the four-property test, the error location in the compiled archive data is recorded and repair suggestions are provided.

2. The file receiving and compiling method according to claim 1, characterized in that: The extracting metadata from the pre-processed archive data, matching the metadata with a metadata template in a format library, and if there is no metadata missing in the pre-processed archive data, saving the pre-processed archive data in an editing library, further includes: If metadata is missing in the pre-processed archive data, relevant personnel are reminded to complete the data, and when the relevant personnel complete the data manually, intelligent prompts are given based on the pre-processed archive data that has been entered.

3. The file receiving and compiling method according to claim 1, characterized in that: Performing hash detection on the compiled archive data in the compilation library includes: Performing a hash operation on the preprocessed archive data to generate a first hash value; Performing a secondary hash operation on the compilation archive data stored in the compilation library to generate a second hash value; The first hash value and the second hash value are compared to determine the integrity of the compiled archive data.

4. The file receiving and compiling method according to claim 1, characterized in that: Performing metadata logic detection and format specification detection on the compiled archive data in the compilation library, including: Detecting the format and logical relationship of the metadata in the compiled archival data; Corresponding format specification detection is performed according to the file type of the compiled archive data.

5. The file receiving and compiling method according to claim 1, characterized in that: Performing personalized detection on the compiled archive data in the compiled library includes: A preset personalized detection data category is obtained, and personalized detection is performed on the compiled file data according to the type of the compiled file data, the detection priority of the preset personalized detection data category, and the preset personalized detection data category.

6. The file receiving and compiling method according to claim 1, characterized in that: The method further comprises: Using a text analysis algorithm and an image recognition algorithm to intelligently classify and subject-label the text data and image data in the compiled archive data; The compiled archival data is identified according to its category and subject, an archival association relationship is established, and a complete archival chain is constructed.

7. The file receiving and compiling method according to claim 1, characterized in that: The method further comprises: During the archival data storage process, real-time monitoring of system resource usage; According to the processing progress of the archive data, the data quality and the system resource occupancy, the detection parameters of each part and the CPU occupancy rate are dynamically adjusted.

8. A file receiving and compiling device, characterized in that: include: An acquisition module is used to acquire archival data, identify, parse and uniformly format the archival data, and filter the archival data according to preset rules to acquire pre-processed archival data; A first saving module is used to extract metadata from the pre-processed archive data, match the metadata with a metadata template in a format library, and if there is no metadata missing in the pre-processed archive data, save the pre-processed archive data in an editing library; A detection module, used for performing four-property detection on the compiled archive data in the compilation library, wherein the four-property detection includes hash detection, metadata logic detection, format specification detection and personalization detection; The second saving module saves the compiled archive data to the archive library through distributed storage if the compiled archive data passes the four-property detection; if the compiled archive data fails to pass the four-property detection, records the location of the error in the compiled archive data and provides repair suggestions.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images