Intelligent file centralized management method, device and equipment in information system and medium

Abstract

This application proposes an intelligent centralized file management method, apparatus, device, and medium in an information system, relating to the field of information management technology. The method includes: a client transmitting file slices to a server; the server sorting the slices by identifier and recalculating hash values ​​to verify integrity; if verification fails, a client retransmission mechanism is triggered; the server restores the verified slice content to text fragments and inputs them into a compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client; compliant slices are integrated and stored in a designated directory; the compliant slice content is input into a topic extraction model to generate metadata tags and store them in a vector library. This application significantly improves the processing efficiency and intelligence level in the centralized file management process.

Description

Technical Field

[0001] This invention relates to the field of information management technology, and in particular to an intelligent centralized file management method, apparatus, device, and medium in an information system. Background Technology

[0002] In enterprise information management, there are a large number of documents of various types, including Word, PDF, Excel, and TXT formats, which are scattered across various dedicated business systems. At the same time, user terminals and business departments are constantly generating new technical documents and standards. Although Large Language Models (LLM) and Enhanced Retrieval (RAG) technologies can handle the ever-increasing number of internal and external documents in enterprises more conveniently due to their semantic understanding, content generation, and information extraction capabilities, their in-depth application in enterprise document management is still limited by the architectural limitations of existing systems.

[0003] Current enterprise information management systems mostly employ traditional file transfer and storage models. Specifically, in the transfer stage, files are typically uploaded to the server in chunks using a streaming, slice-based approach. While this alleviates network pressure, secondary slicing is required after upload for vectorized embedding and knowledge base construction, significantly extending the file processing cycle. In the storage stage, files are often organized in a tree-like directory structure, lacking support for multi-dimensional tag attributes. The same file may belong to multiple categories due to business relevance, but existing systems only support single-directory path retrieval, making it difficult to handle multi-tag joint queries, especially posing a technical bottleneck for accurately locating files with the same name across directories. Furthermore, content compliance checks are lagging; sensitive word detection and format verification can only be performed after the file is fully uploaded, making it impossible to intercept problematic content in real time during transmission, resulting in compliance risks and efficiency losses. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for intelligent centralized file management in information systems, which solves the problems of long time cycles, lack of automatic tag recognition, and inability to perform real-time content inspection during the centralized file management process in information systems.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, a method for intelligent centralized file management in an information system is provided, including:

[0007] The client executes file slicing and transfers it to the server;

[0008] The server sorts the data by slice identifier and recalculates the hash value to verify integrity; if verification fails, the client retransmission mechanism is triggered.

[0009] The server restores the verified slice content into text fragments and inputs them into the compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client.

[0010] Consolidate and store compliant slices in a specified directory;

[0011] The compliant slice content is input into the topic to extract the large model, generate metadata tags, and store them in the vector library.

[0012] In a first possible implementation of the first aspect, the client-side execution of file slice transmission to the server includes:

[0013] The size and number of slices are dynamically determined based on file size, network conditions, readability of slice content, and transmission protocol.

[0014] Generate a unique hash identifier based on the hash algorithm;

[0015] Slices are encapsulated into packets with addresses and identifiers and transmitted concurrently through different paths.

[0016] In the second possible implementation of the first aspect, the statement that if any slice fails, the entire document is discarded and feedback is sent to the client includes:

[0017] Record non-compliant locations and report them to the client to issue a reminder.

[0018] Secondly, an intelligent centralized file management device for an information system is provided, comprising:

[0019] The file slice transfer module is used by the client to perform file slice transfer to the server;

[0020] The integrity verification module is used by the server to sort by slice identifier and recalculate hash value to verify integrity; if the verification fails, the client retransmission mechanism is triggered.

[0021] The compliance check module is used by the server to restore the verified slice content into text fragments, input them into the compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client.

[0022] The storage module is used to integrate and store compliant slices to a specified directory;

[0023] The topic extraction module is used to input compliant slice content into the topic extraction model, generate metadata tags, and store them in the vector library.

[0024] In the first possible implementation of the second aspect, the slice transmission module is specifically used for:

[0025] The size and number of slices are dynamically determined based on file size, network conditions, readability of slice content, and transmission protocol.

[0026] Generate a unique hash identifier based on the hash algorithm;

[0027] Slices are encapsulated into packets with addresses and identifiers and transmitted concurrently through different paths.

[0028] In the second possible implementation of the second aspect, the compliance check module is specifically used for:

[0029] Record non-compliant locations and report them to the client to issue a reminder.

[0030] In the third possible implementation of the second aspect, the processes of the storage module and the topic extraction module are executed synchronously without interference.

[0031] Thirdly, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the intelligent file centralized management method in the information system as described in the first aspect.

[0032] Fourthly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the intelligent file centralized management method in the information system as described in the first aspect.

[0033] The intelligent centralized file management method in the information system of the present invention has the following advantages:

[0034] (1) The file is uploaded, integrated and vectorized and stored in only one slice and integration from the client to the server. There is no need to re-slice and vectorize the file on the client, which saves server resources and time costs.

[0035] (2) The file content has already undergone compliance checks during the upload process. Compliant files are retained, and non-compliant files are returned directly, reducing the consumption of server resources. This achieves real-time checks on file content and quality while avoiding the lag in document verification.

[0036] (3) After the document compliance check is passed, the sliced ​​content undergoes two independent processes. First, the slices are integrated into a complete document and stored in the traditional file upload mode. Second, the theme (metadata) of the sliced ​​content is directly extracted and stored in the vector library, making it convenient for users to quickly retrieve files according to the content theme (metadata).

[0037] The apparatus, electronic device, and readable storage medium corresponding to the intelligent file centralized management method in the information system of the present invention can achieve the same technical effect, and will not be described in detail here to avoid duplication. Attached Figure Description

[0038] Figure 1 A schematic flowchart illustrating an intelligent centralized file management method in an information system, provided as an embodiment of this application;

[0039] Figure 2 A schematic flowchart illustrating another intelligent centralized file management method in an information system provided in an embodiment of this application;

[0040] Figure 3 A schematic diagram of the structure of an intelligent centralized file management device in an information system provided in this application embodiment;

[0041] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions in the embodiments of this application are clearly described. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.

[0043] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0044] The steps described in the specification and the flowcharts in the accompanying drawings of this invention are not necessarily strictly executed according to the step numbers; the execution order of the method steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0045] The following detailed description, in conjunction with the accompanying drawings and preferred embodiments, describes the intelligent centralized file management method, apparatus, device, and medium in the information system provided in this application.

[0046] With the development of artificial intelligence technologies such as large language models, leveraging their powerful semantic understanding, text generation, and content extraction capabilities, it is possible to more conveniently process the ever-increasing volume of internal and external documents within enterprises. In particular, augmented retrieval technology (RAG) has significantly improved the ability of large language models to process the private domain of internal enterprise information.

[0047] However, since large language models require a certain amount of computing power, how to compress the file processing cycle as much as possible, discover relevant compliance issues in the file in advance, and further improve the processing efficiency of the relevant files has become the focus of this invention.

[0048] Traditional enterprise information management system software mostly provides file upload functionality, but these files are mostly transmitted as streams and stored as file directories, which mainly presents the following problems:

[0049] (1) The same file needs to be repeatedly sliced ​​in order to transfer it from the local client to the server's knowledge base: During the file upload process, streaming slicing is usually used to reduce the pressure of file upload. After the upload is completed, it is often necessary to slice it again to perform operations such as vectorization of the text content, which lengthens the file processing cycle.

[0050] (2) Files are stored according to directories and lack tag attributes. Often, a file may belong to multiple categories from a classification perspective. However, during the file retrieval process, files can usually only be returned according to a tree structure and cannot be retrieved simultaneously from multiple tag attributes, especially for files with the same file name in different directories.

[0051] (3) The content and quality of the file cannot be checked in real time. Especially in scenarios where the requirements for sensitive words are high, the content can only be checked after the file is uploaded, which is often delayed.

[0052] The above situations have resulted in problems such as long time cycles, lack of automatic tag recognition, and inability to perform real-time content checks during the centralized management of files in information systems.

[0053] Based on this, the present invention aims to study a method for realizing the vectorized storage, automatic tagging, and real-time content inspection of centralized file management in an information system through artificial intelligence means such as large models, thereby improving the efficiency of centralized file management and providing more convenient file resources for the intelligent upgrading of information systems and the construction of internal knowledge bases.

[0054] This invention adds a segment parsing function to the streaming data receiving end on the basis of traditional file streaming segment network transmission, realizing real-time parsing and vectorized embedding of streaming segments. Relying on artificial intelligence such as large language models, it realizes automatic extraction of file content segment tags, thereby improving the efficiency of file content inspection.

[0055] Please see Figure 1-2 This application provides an intelligent centralized file management method in an information system, such as... Figure 1-2 As shown, it includes:

[0056] Step S1: The client performs file slicing and transfer to the server.

[0057] The client program will slice the uploaded file according to specified rules and then transmit it concurrently over the network. Specifically, this includes:

[0058] Step S11: Dynamically determine the size and number of slices based on file size, network conditions, readability of slice content, and transmission protocol.

[0059] Step S12: Generate a unique hash identifier according to the hash algorithm.

[0060] Step S13: Encapsulate the slice into a packet with address and identifier, and transmit it concurrently through different paths.

[0061] In this application, file slice transmission is a process that coordinates multiple technologies. Before transmission, the system determines the slice size and number based on file size, network conditions, slice content readability, and transmission protocol, generally between 1MB and 100MB, balancing transmission efficiency and management pressure. During slicing, hash algorithms such as MD5 and SHA-256 generate unique identifiers for the receiving end to verify integrity and order. During transmission, the TCP / IP protocol stack encapsulates the slices into packets with addresses and identifiers, transmitting them concurrently through different paths, with the TCP sliding window mechanism adjusting the rate according to network congestion.

[0062] In step S2, the server sorts the data by slice identifier and recalculates the hash value to verify integrity; if the verification fails, the client retransmission mechanism is triggered.

[0063] This application requires only one slice to meet the needs of transmission and subsequent processing, avoiding the time cost of secondary slicing. The server sorts the slices by their identifiers and recalculates their hash values, using double verification to ensure the integrity and correct order of the slices. Only compliant slices proceed to subsequent processing, avoiding unnecessary resource consumption.

[0064] In step S3, the server restores the verified slice content into text fragments and inputs them into the compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client.

[0065] In this application, after a slice passes verification, it is not directly concatenated; instead, the slice content is directly restored and fed into a large model for compliance checking. After the slice hash value verification is successful, the content is restored using a corresponding algorithm and input into the compliance checking model for inspection. Subsequent steps are executed only after all slices of the same document have been checked. If any slice fails the compliance check, all slices of the document are discarded, the non-compliant location is recorded, and a notification is sent to the client to alert the user. This step completes the check during transmission, avoiding the risk of delays. Furthermore, only compliant documents consume storage and processing resources, reducing the accumulation of invalid data.

[0066] This step utilizes a large-scale model for content compliance checks. Through advanced techniques such as deep semantic understanding, multi-task learning, and rule-model fusion, it efficiently and intelligently completes the compliance checks of the content slices. The deployment of the large-scale model for compliance checks consists of the following five steps:

[0067] a. Data preprocessing and annotation: Clean and segment the documents, and prepare high-quality labeled data for model training.

[0068] b. Model training and optimization: Train large models using labeled data and improve model performance by adjusting hyperparameters and optimizing loss functions.

[0069] c. Real-time detection and evaluation: Deploy the trained model to real-world applications, perform real-time compliance checks on the input documents, and periodically evaluate the model's performance.

[0070] d. Feedback loop and continuous optimization: Collect user feedback and new compliance requirements to continuously optimize the model and form a closed-loop continuous improvement process.

[0071] e. Integration and Deployment: Integrate the compliance inspection model into the workflow of this invention to achieve seamless integration and efficient application.

[0072] Finally, compliance documents are synchronized via a dual-path process, including:

[0073] Step S41: Integrate and store the compliant slices to the specified directory.

[0074] Slices that pass the compliance check are sequentially assembled into a complete document. The server stores the document in a designated location and stores the location information in the database. This step retains a tree-structured directory storage model to adapt to existing system requirements, and the path information is stored in a structured manner to support basic retrieval.

[0075] Step S42: Input the compliant slice content into the topic extraction big model, generate metadata tags and store them in the vector library.

[0076] Content slices that pass compliance checks are processed by extracting topics (metadata) from a large model and then directly stored in a vector library. This step constructs a multi-dimensional tagging system using metadata such as topics and entities, supporting semantic retrieval; the vector library supports rapid similarity matching, overcoming the limitations of traditional directory retrieval.

[0077] The content topic (metadata) extraction technology in this step, through the integration of deep semantic understanding, topic modeling, named entity recognition, and other technologies, can achieve efficient and accurate automated content topic (metadata) extraction. The deployment process of the large-scale content extraction model and the large-scale content compliance check model both consist of five steps. Except for the third step, the other steps are identical. The third step in the deployment process of the large-scale content extraction model is real-time content extraction, which applies the trained model to actual documents to automatically extract the document's topic and key metadata.

[0078] Based on the above technical solution, this application has the following advantages:

[0079] (1) When slicing files, readability of the sliced ​​content is considered. Complete paragraphs are preserved during slicing to avoid incomplete document slice content. This improves the accuracy of the large model for compliance inspection of sliced ​​content and ensures the accuracy of the large model for extracting the theme (metadata) of sliced ​​content. The file upload, integration and vectorization storage are completed in only one slicing and integration from the client to the server. There is no need to re-slice and vectorize the file on the client, which saves server resources and time costs.

[0080] (2) After the document slices are restored, they are not directly integrated into a complete document. Instead, a batch compliance check is performed on the file slices. Once a non-compliant item is detected, the entire document is returned, and no more server resources are consumed. Only after all slices of a document have passed the compliance check will the subsequent process proceed. In other words, the file content has already undergone compliance checks during the upload process. Compliant files are retained, and non-compliant files are directly returned, reducing the consumption of server resources. This achieves real-time checking of file content and quality while avoiding the lag in document verification.

[0081] (3) After the document compliance check is passed, the sliced ​​content undergoes two independent processes. First, the slices are integrated into a complete document and stored according to the traditional file upload model. Second, the theme (metadata) of the sliced ​​content is directly extracted and stored in the vector library, making it convenient for users to quickly retrieve files according to the content theme (metadata). By directly extracting the theme (metadata) of the sliced ​​content and storing it in the vector library, the server can quickly return the corresponding file set as a tag during the file retrieval process, improving the document retrieval efficiency.

[0082] See Figure 3Corresponding to the above-mentioned intelligent file centralized management method embodiment in the information system, this application embodiment provides an intelligent file centralized management device in the information system, including:

[0083] The slice transmission module 1001 is used for transmitting file slices from the client to the server.

[0084] The integrity verification module 1002 is used by the server to sort by slice identifier and recalculate hash value to verify integrity; if the verification fails, the client retransmission mechanism is triggered.

[0085] The compliance check module 1003 is used by the server to restore the verified slice content into text fragments, input them into the compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client.

[0086] Storage module 1004a is used to integrate and store compliant slices to a specified directory;

[0087] The topic extraction module 1004b is used to input compliant slice content into the topic extraction model, generate metadata tags, and store them in the vector library.

[0088] Furthermore, the slice transmission module is specifically used for:

[0089] The size and number of slices are dynamically determined based on file size, network conditions, readability of slice content, and transmission protocol.

[0090] Generate a unique hash identifier based on the hash algorithm;

[0091] Slices are encapsulated into packets with addresses and identifiers and transmitted concurrently through different paths.

[0092] Furthermore, the compliance check module is specifically used for:

[0093] Record non-compliant locations and report them to the client to issue a reminder.

[0094] Furthermore, the processes of the storage module and the topic extraction module are executed synchronously without interference.

[0095] The intelligent file centralized management device in the above information system implements the steps and processes of the embodiment of the intelligent file centralized management method in the above information system, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0096] See Figure 4Corresponding to the above-mentioned intelligent file centralized management method embodiment in the information system, this application embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps and processes of the above-mentioned intelligent file centralized management method embodiment in the information system and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0097] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback function, image playback function, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0098] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0099] Corresponding to the above-mentioned intelligent file centralized management method embodiment in the information system, this application embodiment also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the steps and processes of the above-mentioned intelligent file centralized management method embodiment in the information system and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0100] The processor is the processor in the electronic device described in the above embodiments of this application. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0101] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0103] It is understood that the embodiments of this application have been described above in conjunction with the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. As those skilled in the art will know, various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, those skilled in the art, under the guidance or instruction of this application, can modify these features and embodiments to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, this invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this invention.

Claims

1. A method for centralized intelligent file management in an information system, characterized in that, include: The client executes file slicing and transfers it to the server; The server sorts the data by slice identifier and recalculates the hash value to verify integrity. If verification fails, the client retransmission mechanism is triggered; The server restores the verified slice content into text fragments and inputs them into the compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client. Consolidate and store compliant slices in a specified directory; The compliant slice content is input into the topic to extract the large model, generate metadata tags and store them in the vector library; The client-side execution of file slice transmission to the server includes: The size and number of slices are dynamically determined based on file size, network conditions, readability of slice content, and transmission protocol. Generate a unique hash identifier based on the hash algorithm; Slices are encapsulated into packets with addresses and identifiers and transmitted concurrently through different paths.

2. The intelligent centralized file management method in an information system according to claim 1, characterized in that, The provision that if any slice fails, the entire document is discarded and feedback is sent to the client includes: Record non-compliant locations and report them to the client to issue a reminder.

3. An intelligent centralized file management device in an information system, characterized in that, include: The file slice transfer module is used by the client to perform file slice transfer to the server; The integrity verification module is used by the server to sort by slice identifier and recalculate hash value to verify integrity. If verification fails, the client retransmission mechanism is triggered; The compliance check module is used by the server to restore the verified slice content into text fragments, input them into the compliance check model for real-time checking; if any slice fails, the entire document is discarded and feedback is sent to the client. The storage module is used to integrate and store compliant slices to a specified directory; The topic extraction module is used to input compliant slice content into the topic extraction model, generate metadata tags, and store them in the vector library; The slice transmission module is specifically used for: The size and number of slices are dynamically determined based on file size, network conditions, readability of slice content, and transmission protocol. Generate a unique hash identifier based on the hash algorithm; Slices are encapsulated into packets with addresses and identifiers and transmitted concurrently through different paths.

4. The intelligent centralized file management device in the information system according to claim 3, characterized in that, The aforementioned compliance check module is specifically used for: Record non-compliant locations and report them to the client to issue a reminder.

5. The intelligent centralized file management device in the information system according to claim 3, characterized in that, The processes of the storage module and the topic extraction module do not interfere with each other and are executed synchronously.

6. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the intelligent file centralized management method in the information system as described in any one of claims 1 to 2.

7. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the intelligent file centralized management method in the information system as described in any one of claims 1 to 2.

Images