Intelligent file centralized management method and device in information system, equipment and medium

Abstract

The invention provides an intelligent file centralized management method and device in an information system, equipment and a medium, and relates to the technical field of information management, and the method comprises the steps that a client side executes file slicing and transmits the file slicing to a server side; the server side ranks according to the slice identifiers and recalculates the hash value, and verifies the integrity; if the verification fails, triggering a client retransmission mechanism; the server side restores the slice content passing verification into text fragments, and the text fragments are input into a compliance check large model for real-time check; if any slice does not pass, abandoning the whole document and feeding back the whole document to the client; integrating and storing the compliance slices to a specified directory; and inputting the compliance slice content into the topic extraction large model, generating a metadata tag, and storing the metadata tag into a vector library. According to the method, the processing efficiency and the intelligent level in the file centralized management process are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of information management technology, and in particular to an intelligent file centralized management method, device, equipment and medium in an information system. Background Art

[0002] Enterprise information management involves a vast array of documents and materials, stored in various formats, including Word, PDF, Excel, and TXT, scattered across specialized business systems. Simultaneously, various user terminals and business departments are constantly generating new technical documents, standards, and specifications. While Large Language Models (LLMs) and Retrieval Enhancement (RAG) technologies, with their semantic understanding, content generation, and information extraction capabilities, can more conveniently process the ever-increasing volume of internal and external documents within an enterprise, their in-depth application in enterprise document management is still limited by the architectural constraints of existing systems.

[0003] Current enterprise information management systems mostly adopt traditional file transfer and storage modes. Specifically, in the transmission process, files are usually uploaded to the server in blocks in a streaming slicing manner. Although this can alleviate network pressure, they need to be sliced ​​again after uploading to achieve vectorized embedding and knowledge base construction, which significantly prolongs the file processing cycle. In the storage process, files are mostly organized in a tree-like directory structure and lack support for multi-dimensional label attributes. The same file may belong to multiple categories due to business relevance, but the existing system only supports single directory path retrieval, which makes it difficult to cope with multi-label joint query requirements, especially for the precise positioning of files with the same name across directories. There is a technical bottleneck. In addition, the content compliance check mechanism lags behind, and operations such as sensitive word detection and format verification need to wait until the file is fully uploaded before they can be executed. It is impossible to intercept problematic content in real time during the transmission process, resulting in compliance risks and efficiency losses. Summary of the Invention

[0004] The present invention provides an intelligent centralized file management method, device, equipment and medium in an information system, which solves the problems of long time period, lack of automatic tag recognition and inability to perform real-time content inspection during the centralized file management process in the information system.

[0005] To achieve the above objectives, this application adopts the following technical solutions: In a first aspect, a method for intelligent centralized file management in an information system is provided, comprising: The client executes file slicing and transfers it to the server; The server sorts by slice identifier and recalculates the hash value to verify the integrity; if the verification fails, the client retransmission mechanism is triggered; The server converts the verified slices into text fragments and inputs them into the compliance check model for real-time verification. If any slice fails, the entire document is discarded and fed back to the client. Integrate and store compliant slices into a specified directory; Input the compliant slice content into the subject extraction model, generate metadata tags and store them in the vector library.

[0006] In a first possible implementation of the first aspect, the client executing file slicing and transmitting it to the server includes: Dynamically determine the slice size and quantity based on file size, network conditions, slice content readability, and transmission protocol; Generate a unique hash identifier based on a hash algorithm; Encapsulate the slices into addressed and identified packets and transmit them concurrently through different paths.

[0007] In a second possible implementation of the first aspect, if any slice fails, the entire document is discarded and fed back to the client, including: Record non-compliant locations and report them to the client to issue a reminder.

[0008] In a second aspect, an intelligent centralized file management device in an information system is provided, comprising: Slice transfer module, used by the client to execute file slice transfer to the server; The integrity verification module is used by the server to sort by slice identifier and recalculate the hash value to verify the integrity; if the verification fails, the client retransmission mechanism is triggered; The compliance check module is used by the server to convert the verified slice content into text fragments and input them into the compliance check model for real-time inspection. If any slice fails, the entire document is discarded and fed back to the client. The storage module is used to integrate and store compliant slices into a specified directory; The topic extraction module is used to input the compliant slice content into the topic extraction model, generate metadata tags and store them in the vector library.

[0009] In a first possible implementation manner of the second aspect, the slice transmission module is specifically configured to: Dynamically determine the slice size and quantity based on file size, network conditions, slice content readability, and transmission protocol; Generate a unique hash identifier based on a hash algorithm; Encapsulate the slices into addressed and identified packets and transmit them concurrently through different paths.

[0010] In a second possible implementation manner of the second aspect, the compliance checking module is specifically configured to: Record non-compliant locations and report them to the client to issue a reminder.

[0011] In a third possible implementation manner of the second aspect, the processes of the storage module and the topic extraction module are executed synchronously without interfering with each other.

[0012] In a third aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the intelligent file centralized management method in the information system as described in the first aspect.

[0013] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the intelligent file centralized management method in the information system as described in the first aspect are implemented.

[0014] The intelligent centralized file management method in the information system of the present invention has the following advantages: (1) The file is uploaded, integrated, and vectorized stored only once from the client to the server. There is no need to re-slice and re-vectorize the file on the client, saving server resources and time costs.

[0015] (2) The compliance of the file content has been checked during the upload process. Compliant files are retained and non-compliant files are directly returned, reducing the consumption of server resources. While achieving real-time inspection of file content and quality, it avoids the lag of document verification.

[0016] (3) After the document compliance check is passed, the slice content will go through two independent processes. The first is to integrate the slices into a complete document and store it in the traditional file upload mode. The second is to directly extract the subject (metadata) of the slice content and store it in the vector library, so that users can quickly search for files according to the content subject (metadata).

[0017] The device, electronic device and readable storage medium corresponding to the intelligent centralized file management method in the information system of the present invention can achieve the same technical effects. To avoid repetition, they will not be described here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of a method for intelligent centralized file management in an information system provided in an embodiment of the present application; Figure 2 A schematic flow chart of another intelligent file centralized management method in an information system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an intelligent centralized file management device in an information system provided in an embodiment of the present application; Figure 4A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] To further illustrate the technical means and effects of the present invention to achieve its intended purpose, the technical solutions in the embodiments of this application are clearly described. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of this application are within the scope of protection of this application.

[0020] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0021] The description of the method flow in the specification of this application and the steps in the flowcharts in the drawings of the specification of this application do not necessarily need to be strictly executed according to the step numbers. The method steps can be executed in a different order. In addition, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0022] The following is a detailed description of the intelligent file centralized management method, device, equipment and medium in the information system provided by the embodiments of the present application in combination with the accompanying drawings and preferred embodiments.

[0023] The development of AI technologies like large language models, leveraging their powerful semantic understanding, text generation, and content extraction capabilities, has made it easier to process the ever-increasing volume of internal and external documents within an enterprise. In particular, enhanced retrieval technology (RAG) has significantly improved the large language model's ability to process the private domain of internal enterprise information.

[0024] However, since large language models require a certain amount of computing power, how to compress the file processing cycle as much as possible, discover relevant compliance and other related issues in the files in advance, and further improve the processing efficiency of related files has become the focus of this invention.

[0025] Most traditional enterprise information management system software provides file upload functions, but most of these files are transmitted in a stream manner and stored in a file directory format, which mainly has the following problems: (1) The same file needs to be sliced ​​repeatedly to be transferred from the local client to the server's knowledge base: During the file upload process, streaming slicing is usually used to reduce the pressure of file upload. After the upload is completed, it is often necessary to slice again to perform operations such as vectorization of the text content, which prolongs the file processing cycle.

[0026] (2) Files are stored in directories and lack tag attributes. Often, a file can belong to multiple categories from a classification perspective. However, during file retrieval, files can usually only be returned according to the tree structure, and it is impossible to retrieve files from multiple tag attributes simultaneously, especially for files with the same file name in different directories.

[0027] (3) The content and quality of files cannot be checked in real time, especially in scenarios where sensitive words are required. The content can only be verified after the file is uploaded, which often lags behind.

[0028] The above situation has resulted in problems such as long time periods, lack of automatic tag recognition, and inability to conduct real-time content inspection in the process of centralized file management in information systems.

[0029] Based on this, the present invention aims to study a method for realizing content vectorized storage, automatic tagging, real-time content inspection and other processing of centralized file management in an information system through artificial intelligence means such as large models, thereby improving the efficiency of centralized file management, providing more convenient file resources for intelligent upgrades of information systems, and building internal knowledge bases.

[0030] Based on the traditional file streaming slicing network transmission, the present invention adds a fragment parsing function at the streaming data receiving end to realize real-time parsing and vectorized embedding of streaming fragments. Relying on artificial intelligence such as large language models, it realizes the automatic extraction of file content fragment labels and improves the efficiency of file content inspection.

[0031] See Figure 1-2 , the embodiment of the present application provides an intelligent file centralized management method in an information system, such as Figure 1-2 As shown, including: Step S1: The client executes file slicing and transmits it to the server.

[0032] The client program will slice the uploaded file according to the specified rules and then transmit it concurrently over the network. Specifically: Step S11, dynamically determine the slice size and quantity based on the file size, network conditions, slice content readability, and transmission protocol.

[0033] Step S12: Generate a unique hash identifier according to a hash algorithm.

[0034] Step S13: encapsulate the slices into packets with addresses and identifiers, and transmit them concurrently through different paths.

[0035] In this application, file slicing transmission is a collaborative process involving multiple technologies. Before transmission, the system determines the size and number of slices based on file size, network conditions, slice content readability, and transmission protocol. The size and number are generally between 1MB and 100MB, balancing transmission efficiency and management pressure. During slicing, hash algorithms such as MD5 and SHA-256 generate unique identifiers for the receiving end to verify integrity and sorting. During transmission, the TCP / IP protocol stack encapsulates the slices into addressed and identified packets, which are transmitted concurrently through different paths. The TCP sliding window mechanism adjusts the rate based on network congestion.

[0036] In step S2, the server sorts by slice identifier and recalculates the hash value to verify the integrity; if the verification fails, the client retransmission mechanism is triggered.

[0037] This application only requires one slice to meet the transmission and subsequent processing requirements, avoiding the time cost of secondary slicing. The server sorts by slice identifier and recalculates the hash value. Through double verification, it ensures the integrity and correct order of the slices. Only compliant slices enter the subsequent processing to avoid invalid resource usage.

[0038] In step S3, the server converts the verified slice content into text fragments and inputs them into the compliance check model for real-time inspection. If any slice fails, the entire document is discarded and fed back to the client.

[0039] In this application, after the slices are verified, they are not directly spliced ​​together. Instead, the slice contents are directly restored and passed into the large model for compliance checking. After the slice hash value is verified, the content is restored using the corresponding algorithm, and the content is input into the compliance check large model for inspection. Subsequent steps are executed after all slices of the same document have been checked. If any slice fails the compliance check, all slices of the document are discarded, the non-compliant locations are recorded, and the reports are sent to the client to alert the user. This step completes the inspection during the transmission process to avoid the risk of lag. At the same time, only compliant documents occupy storage and processing resources, reducing the accumulation of invalid data.

[0040] This step relies on a large model to perform slice content compliance checking technology. Through advanced means such as deep semantic understanding, multi-task learning, and rule and model fusion, it can efficiently and intelligently complete the compliance check of slice content. The deployment of the large model for compliance checking is divided into the following five steps: a. Data preprocessing and annotation: Perform preprocessing on documents, such as cleaning and word segmentation, and prepare high-quality annotated data for model training.

[0041] b. Model training and optimization: Use labeled data to train large models and improve model performance by adjusting hyperparameters, optimizing loss functions, and other methods.

[0042] c. Real-time detection and evaluation: Deploy the trained model to actual applications, perform real-time compliance checks on input documents, and regularly evaluate the model's effectiveness.

[0043] d. Feedback loop and continuous optimization: Collect user feedback and new compliance requirements, continuously optimize the model, and form a closed-loop continuous improvement process.

[0044] e. Integration and deployment: Integrate the compliance check model into the workflow of the present invention to achieve seamless connection and efficient application.

[0045] Finally, dual-path processing is performed synchronously for compliant documents, including: Step S41: Integrate and store the compliant slices into the specified directory.

[0046] Slices that pass the compliance check are sequentially stitched together to form a complete document. The server then stores the document in a designated location and stores the location information in the database. This step retains the tree-like directory storage model to accommodate existing system requirements, with structured storage of path information to support basic retrieval.

[0047] Step S42: Input the compliant slice content into the subject extraction model, generate metadata tags and store them in the vector library.

[0048] Slices that pass compliance checks are extracted using a large model to extract topics (metadata) and then directly stored in a vector library. This step constructs a multi-dimensional tagging system based on metadata such as topics and entities, supporting semantic search. The vector library enables fast similarity matching, breaking through the limitations of traditional directory search.

[0049] The sliced ​​content topic (metadata) extraction technology in this step integrates deep semantic understanding, topic modeling, named entity recognition, and other technologies to achieve efficient and accurate automated content topic (metadata) extraction. The deployment process for the sliced ​​content extraction model is divided into five steps, similar to the content compliance check model. The third step is different, and the remaining steps are identical. The third step of the sliced ​​content extraction model deployment process is real-time content extraction, which applies the trained model to actual documents to automatically extract the document's topic and key metadata.

[0050] Based on the above technical solution, this application has the following advantages: (1) When slicing files, the readability of the sliced ​​content is taken into consideration. Complete paragraphs are retained during slicing to avoid incomplete document slice content. This not only improves the accuracy of the large model for checking the compliance of the sliced ​​content, but also ensures the accuracy of the large model for extracting the theme (metadata) of the sliced ​​content. The file is uploaded, integrated, and vectorized for storage in just one slicing and integration from the client to the server. There is no need to re-slice and re-vectorize the file on the client, saving server resources and time costs.

[0051] (2) After the document slices are restored, they are not directly integrated into a complete document. Instead, the file slices are first batch checked for compliance. Once non-compliant items are found, the entire document is returned and no longer occupies server resources. Only when all the slices of a document pass the compliance check, the subsequent process is carried out. In other words, the file content has been checked for compliance during the upload process. The compliant files are retained and the non-compliant files are directly returned, reducing the consumption of server resources. While achieving real-time inspection of file content and quality, it also avoids the lag of document verification.

[0052] (3) After the document compliance check is passed, the slice content executes two independent processes. The first is to integrate the slices into a complete document and store it in the traditional file upload mode. The second is to directly extract the subject (metadata) from the slice content and store it in the vector library, so that users can quickly search for files according to the content subject (metadata). Directly extracting the subject (metadata) from the slice content and storing it in the vector library, during the file retrieval process, the server can use the subject (metadata) as a label to quickly return the corresponding file set, thereby improving the efficiency of document retrieval.

[0053] See also Figure 3 Corresponding to the above-mentioned embodiment of the intelligent centralized file management method in an information system, the embodiment of the present application provides an intelligent centralized file management device in an information system, including: Slice transmission module 1001, used for the client to execute file slice transmission to the server; Integrity verification module 1002, used by the server to sort by slice identifier and recalculate the hash value to verify the integrity; if the verification fails, the client retransmission mechanism is triggered; The compliance check module 1003 is used by the server to convert the verified slice content into text fragments and input them into the compliance check model for real-time checking. If any slice fails, the entire document is discarded and fed back to the client. Storage module 1004a, used to integrate and store compliant slices into a specified directory; The subject extraction module 1004b is used to input the compliant slice content into the subject extraction model, generate metadata tags and store them in the vector library.

[0054] Furthermore, the slice transmission module is specifically used to: Dynamically determine the slice size and quantity based on file size, network conditions, slice content readability, and transmission protocol; Generate a unique hash identifier based on a hash algorithm; Encapsulate the slices into addressed and identified packets and transmit them concurrently through different paths.

[0055] Furthermore, the compliance checking module is specifically used to: Record non-compliant locations and report them to the client to issue a reminder.

[0056] Furthermore, the processes of the storage module and the theme extraction module are executed synchronously without interfering with each other.

[0057] The intelligent centralized file management device in the above-mentioned information system implements the steps and various processes of the embodiment of the intelligent centralized file management method in the above-mentioned information system, and can achieve the same technical effects. To avoid repetition, they are not described here.

[0058] See also Figure 4 , corresponding to the embodiment of the intelligent file centralized management method in the above-mentioned information system, the embodiment of the present application provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, the steps and various processes of the embodiment of the intelligent file centralized management method in the above-mentioned information system are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0059] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0060] Processor 1010 may include one or more processing units. Optionally, processor 1010 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1010.

[0061] Corresponding to the embodiment of the intelligent file centralized management method in the above-mentioned information system, the embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, the steps and various processes of the embodiment of the intelligent file centralized management method in the above-mentioned information system are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0062] The processor is the processor in the electronic device described in the embodiment of the present application. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0063] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0064] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0065] It will be appreciated that the embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative and not restrictive, and those skilled in the art will appreciate that various changes or equivalent replacements may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, those of ordinary skill in the art may modify these features and embodiments to adapt to specific circumstances and materials under the inspiration or guidance of this application without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application fall within the scope protected by the present invention.

Claims

1. An intelligent centralized file management method in an information system, characterized in that: include: The client executes file slicing and transfers it to the server; The server sorts by slice ID and recalculates the hash value to verify integrity; If the verification fails, the client retransmission mechanism is triggered; The server converts the verified slices into text fragments and inputs them into the compliance check model for real-time verification. If any slice fails, the entire document is discarded and fed back to the client. Integrate and store compliant slices into a specified directory; Input the compliant slice content into the subject extraction model, generate metadata tags and store them in the vector library.

2. The intelligent centralized file management method in an information system according to claim 1, characterized in that: The client executes file slicing and transmits it to the server, including: Dynamically determine the slice size and quantity based on file size, network conditions, slice content readability, and transmission protocol; Generate a unique hash identifier based on a hash algorithm; Encapsulate the slices into addressed and identified packets and transmit them concurrently through different paths.

3. The intelligent centralized file management method in an information system according to claim 1, characterized in that: If any slice fails, the entire document will be discarded and fed back to the client, including: Record non-compliant locations and report them to the client to issue a reminder.

4. An intelligent centralized file management device in an information system, characterized in that: include: Slice transfer module, used by the client to execute file slice transfer to the server; The integrity verification module is used by the server to sort by slice identifier and recalculate the hash value to verify the integrity; If the verification fails, the client retransmission mechanism is triggered; The compliance check module is used by the server to convert the verified slice content into text fragments and input them into the compliance check model for real-time inspection. If any slice fails, the entire document is discarded and fed back to the client. The storage module is used to integrate and store compliant slices into a specified directory; The topic extraction module is used to input the compliant slice content into the topic extraction model, generate metadata tags and store them in the vector library.

5. The intelligent centralized file management device in the information system according to claim 4, characterized in that: The slice transmission module is specifically used for: Dynamically determine the slice size and quantity based on file size, network conditions, slice content readability, and transmission protocol; Generate a unique hash identifier based on a hash algorithm; Encapsulate the slices into addressed and identified packets and transmit them concurrently through different paths.

6. The intelligent centralized file management device in the information system according to claim 4, characterized in that: The compliance checking module is specifically used to: Record non-compliant locations and report them to the client to issue a reminder.

7. The intelligent centralized file management device in the information system according to claim 4, characterized in that: The processes of the storage module and the subject extraction module are executed synchronously without interfering with each other.

8. An electronic device, characterized in that: The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the intelligent file centralized management method in the information system as claimed in any one of claims 1 to 3 are implemented.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the intelligent file centralized management method in the information system according to any one of claims 1 to 3 are implemented.

Images