Multi-storage protocol document management method, system and device and storage medium

Abstract

The invention relates to the technical field of document management, and particularly provides a multi-storage protocol document management method, system and device and a storage medium, and the method comprises the steps: receiving a document through an expansion interface supporting a plurality of storage protocols; identifying semantic information of the document by using a large model, and generating an abstract for the document based on the semantic information; obtaining the type of a document, distributing a target storage medium for the document according to the type, and obtaining a storage address of the document in the target storage medium; and generating an index of the document according to the abstract and the storage address, wherein the index is used for supporting a retrieval service. According to the method and the device, unified management of various storage protocol documents is realized, the document management quality is improved, and subsequent retrieval application of the documents is facilitated.

Description

Technical Field

[0001] The present invention belongs to the technical field of document management, and particularly relates to a management method, system, device and storage medium for multi-storage protocol documents. Background Art

[0002] In modern society, individuals and organizations often need to manage a large amount of document data, which may be stored in different cloud storage services, local storage devices or different file servers, using different storage protocols. Traditional document management systems often focus on specific storage protocols, resulting in users needing to use multiple different tools to manage documents in different storage locations, with low efficiency. Summary of the Invention

[0003] In view of the above deficiencies of the prior art, the present invention provides a management method, system, device and storage medium for multi-storage protocol documents to solve the above technical problems.

[0004] In a first aspect, the present invention provides a management method for multi-storage protocol documents, including: Receiving a document through an extended interface that supports multiple storage protocols; Using a large model to identify the semantic information of the document and generating an abstract for the document based on the semantic information; Obtaining the type of the document, allocating a target storage medium for the document according to the type, and obtaining the storage address of the document in the target storage medium; Generating an index for the document according to the abstract and the storage address, and the index is used to support the retrieval service.

[0005] In an optional embodiment, the method further includes: Designing a unified transport layer, protocol adaptation layer and protocol implementation layer for the interface. The unified transport layer is used to externally expose standard RESTful APIs, process document upload and download requests, and perform format conversion; the protocol adaptation layer is used to define abstract interface specifications; the protocol implementation layer includes specific implementation modules for each storage protocol; Defining a core interface; Implementing a dynamic extension mechanism for the interface using the Java SPI or OSGi framework; Defining a JSON Schema to describe the parameter structure for each storage protocol.

[0006] In an optional embodiment, using a large model to identify the semantic information of the document and generating an abstract for the document based on the semantic information includes: Fusing a vision-language joint encoder to extract text encoding and image encoding from the document, and using adaptive gating to fuse the text encoding and image encoding to obtain a fused feature vector; Using a graph attention network, based on the fused feature vector and the pre-constructed knowledge graph, to generate a semantic topological structure containing dynamic knowledge graph information; Using a reinforcement learning model to generate an abstract based on a preset abstract control dimension and the semantic topological structure.

[0007] In an optional implementation manner, the method further includes: During text encoding, the document can be split into multiple chunks, each chunk is encoded using a chunk_encoder, and then a cognitive map of the document is generated through a DiffusionTransformer, and the cognitive map is used to assist the encoder in semantic understanding.

[0008] In an optional implementation manner, using a reinforcement learning model to generate an abstract based on a preset abstract control dimension and the semantic topological structure includes: Presetting an abstract control dimension, where the abstract control dimension includes information density, professionalism, and sentiment tendency; Generating an abstract for the semantic topological structure according to the abstract control dimension based on a controllable generation method of reinforcement learning, and optimizing the generation process according to a reward function; Confirming that there is an associated document for the document, using the cross_doc_summary function to first mine the relationship between documents through build_relation_graph to construct a document relationship graph, and then using a GraphTransformer to process the document relationship graph to assist in generating a comparative abstract.

[0009] In an optional implementation manner, obtaining the type of the document, and allocating a target storage medium for the document according to the type, and obtaining the storage address of the document in the target storage medium includes: Identifying the type of the document according to a custom type rule; Allocating a target storage medium for the document according to a predefined correspondence between the type and the storage medium and the type of the document; Storing the document in the target storage medium, and obtaining the storage address of the document.

[0010] In an optional implementation manner, identifying the type of the document according to a custom type rule includes: Predefining a plurality of dimension indicators and corresponding dimension weights, where the plurality of dimension indicators include source, application field heat index, file size, file format, and whether it is confidential; Parsing each dimension indicator of the document, calculating the weighted sum of each dimension indicator, and determining the type of the document according to the weighted sum; The application field heat index is the current retrieval heat of the application field to which the document belongs.

[0011] In a second aspect, the present invention provides a management system for multi-storage protocol documents, including: A receiving module, configured to receive documents through an extended interface that supports multiple storage protocols; An identification module, configured to use a large model to identify the semantic information of the document and generate an abstract for the document based on the semantic information; A storage module, configured to obtain the type of the document, allocate a target storage medium for the document according to the type, and obtain the storage address of the document in the target storage medium; An indexing module, configured to generate an index for the document according to the abstract and the storage address, and the index is used to support retrieval services.

[0012] In a third aspect, a device is provided, including: A memory, configured to store a management program for multi-storage protocol documents; A processor, configured to implement the steps of the management method for multi-storage protocol documents provided in the first aspect when executing the management program for multi-storage protocol documents.

[0013] In a fourth aspect, a computer-readable storage medium is provided, on which a management program for multi-storage protocol documents is stored. When the management program for multi-storage protocol documents is executed by a processor, the steps of the management method for multi-storage protocol documents provided in the first aspect are implemented.

[0014] The beneficial effects of the present invention are as follows. The management method, system, device, and storage medium for multi-storage protocol documents provided by the present invention realize the reception of documents with multiple storage protocols by constructing an extended interface that supports multiple storage protocols, and then perform semantic recognition on the received documents and generate an abstract to achieve semantic-level parsing of the documents; classify and store the documents according to the custom document type to achieve orderly storage management of the documents; at the same time, construct an index of the abstract and the storage address to facilitate subsequent retrieval services. The present invention realizes the unified management of multi-storage protocol documents, improves the quality of document management, and facilitates the subsequent retrieval and application of documents.

[0015] In addition, the design principle of the present invention is reliable, the structure is simple, and it has a very wide application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a schematic flow chart of the method according to an embodiment of the present invention.

[0018] Figure 2 It is another schematic flow chart of the method according to an embodiment of the present invention.

[0019] Figure 3 It is a schematic block diagram of the system according to an embodiment of the present invention.

[0020] Figure 4 It is a schematic structural diagram of a device provided by an embodiment of the present invention. Detailed implementation manners

[0021] In order to enable those skilled in the art of the present technology to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

[0023] The method for managing multi-storage protocol documents provided by the embodiments of the present invention is executed by a computer device. Correspondingly, the multi-storage protocol document management system runs in the computer device.

[0024] Figure 1 It is a schematic flow chart of the method according to an embodiment of the present invention. Among them, Figure 1 The execution subject can be a multi-storage protocol document management system. According to different requirements, the order of the steps in this flow chart can be changed, and some can be omitted.

[0025] Such as Figure 1 shown, the method includes: S1. Receiving a document through an extended interface that supports multiple storage protocols; S2. Using a large model to identify the semantic information of the document and generating an abstract for the document based on the semantic information; S3. Obtaining the type of the document, allocating a target storage medium for the document according to the type, and obtaining the storage address of the document in the target storage medium; S4. Generate an index for the document based on the abstract and the storage address, where the index is used to support the retrieval service.

[0026] In one embodiment of the present invention, based on step S1, a possible embodiment will be given below to non - restrictively elaborate on its specific implementation scheme.

[0027] By designing a unified extension interface and implementing a dynamic extension mechanism, flexible access to different storage protocols and unified management are achieved. Meanwhile, a friendly user interface is provided to simplify the process of users configuring storage protocol parameters and managing documents.

[0028] 1. Build an extension interface that supports multiple storage protocols.

[0029] As the interface layer of the system to the outside world, it is responsible for exposing standard RESTful APIs, processing document upload and download requests, and performing necessary format conversions.

[0030] API Design: Design a series of standard RESTful API interfaces. For example, / upload is used for document upload and / download is used for document download. Each API follows the HTTP protocol specification and uses appropriate HTTP methods (such as POST for upload and GET for download).

[0031] Request Processing: When receiving a document upload request, first verify the legality of the request, including the format of the request header and request body. Then, preliminarily process the uploaded document data, such as checking the file type and size. For a download request, locate the corresponding document according to the request parameters and return it to the client in an appropriate format.

[0032] Format Conversion: During the process of handling document upload and download, perform necessary format conversions according to the requirements of different storage protocols. For example, convert the uploaded document from one format to another to meet the storage requirements of a specific storage protocol.

[0033] Define an abstract interface specification to provide a unified interface standard for the implementation of different storage protocols, enabling the protocol implementation layer to perform specific implementations according to these specifications.

[0034] Abstract Interface Definition: Define a series of abstract interfaces, such as the StorageProtocol interface, which contains abstract methods for basic operations such as upload, download, and deletion. These interfaces do not contain specific implementation logic and only specify the method signatures and return value types.

[0035] Interface Specification: Provide detailed specifications for each abstract interface, including the meaning of method parameters, the significance of return values, exception handling, etc. For example, the parameters of the upload method may include the content of the document, the storage path, etc., and the return value may be an indication of successful storage.

[0036] The protocol implementation layer contains specific implementation modules for each storage protocol. According to the abstract interface specifications defined by the protocol adaptation layer, it implements the specific operations of different storage protocols.

[0037] Storage Protocol Implementation Classes: For different storage protocols such as S3, NFS, FTP, etc., implement the corresponding storage protocol classes respectively. These classes implement the abstract interfaces defined by the protocol adaptation layer. For example, the S3 StorageProtocol class implements the StorageProtocol interface and specifically implements operations such as uploading and downloading of the S3 storage protocol.

[0038] Internal Logic Implementation: In each storage protocol implementation class, implement the corresponding operation logic according to the specific storage protocol specifications. For example, in the upload method of the S3StorageProtocol class, use the AWS SDK to call the S3 API for document upload operations.

[0039] Define core interfaces to clarify the core functional interfaces of the system. These interfaces are the key to the system's interaction with the outside world and also the basis for implementing the dynamic extension mechanism.

[0040] Core Interface List: Define interfaces such as the StorageManager interface, which includes operations such as adding, deleting, and querying storage protocols; the DocumentManager interface, which includes operations such as uploading, downloading, and viewing documents.

[0041] Interface Association: Ensure that these core interfaces are associated with the interfaces of the unified transport layer, protocol adaptation layer, and protocol implementation layer to form a complete interface system. For example, the upload and download operations of the DocumentManager interface can call the corresponding APIs of the unified transport layer.

[0042] Adopt the Java SPI or OSGi framework to implement the dynamic extension mechanism of interfaces, enabling the system to dynamically load and unload different storage protocol implementation modules at runtime.

[0043] Java SPI Implementation: Service Provider Configuration: Create a file named with the fully qualified name of the core interface under the META - INF / services directory. The content of the file is the fully qualified name of the specific class that implements the interface. For example, for the StorageProtocol interface, list specific implementation classes such as com.example.S3StorageProtocol in the com.example.StorageProtocol file under META - INF / services.

[0044] Service Loading: At system startup, use the ServiceLoader class to load all service providers that implement the core interface.

[0045] OSGi Implementation: Bundle Development: Package each storage protocol implementation module as an OSGi Bundle. Each Bundle contains the classes that implement the core interface and necessary dependencies.

[0046] Service Registration and Discovery: When the Bundle starts, register the implemented core interface service to the OSGi service registry. Other modules can discover and use these services by querying the service registry.

[0047] Define a JSON Schema to describe the parameter structure for each storage protocol, which is used to describe the parameter structure of the storage protocol, facilitating user configuration and system verification.

[0048] JSON Schema Definition: For each storage protocol, define a corresponding JSON Schema file. For example, for the S3 storage protocol, define a JSON Schema that contains parameters such as accessKey, secretKey, and bucketName.

[0049] 2. Design and Implementation of Storage Configuration Interface Users can configure the relevant parameters of different storage protocols through this page to achieve unified management of the docking storage protocols. At the same time, provide intelligent prompts and auto - completion functions to simplify the user configuration process.

[0050] Interface Layout: Design a simple and intuitive interface, including a drop - down box for storage protocol selection, parameter input boxes, save and cancel buttons, etc.

[0051] Intelligent Prompts and Auto - Completion: According to the storage protocol selected by the user, load the corresponding JSON Schema and provide intelligent prompts and auto - completion functions based on the Schema. For example, when the user enters accessKey, prompt the format requirements of this parameter.

[0052] Parameter Validation: After the user enters parameters, use JSON Schema to perform real-time validation on the parameters. If the parameters are illegal, give corresponding error prompts.

[0053] Data Storage: After the user completes the configuration, save the configuration parameters to the system database for subsequent use.

[0054] Unified Document Management Page: Through this page, users can achieve unified uploading, downloading, and viewing of documents under different storage protocols without switching different applications or tools. At the same time, according to the user's operation habits and historical behaviors, adaptively adjust the interface layout and function display.

[0055] Interface Layout: Design an interface that includes elements such as a document list, upload button, download button, and view button. The document list displays document information under different storage protocols, including file name, storage protocol, upload time, etc.

[0056] Operation Implementation: When the user clicks the upload button, call the / uploadAPI in the unified transport layer to upload the document; when clicking the download button, call the / downloadAPI to download the document; when clicking the view button, display the document content in a suitable way according to the document type and storage protocol.

[0057] Adaptive Adjustment: The system records the user's operation habits and historical behaviors, such as the document types that the user often views and the operation frequency. According to this information, adaptively adjust the interface layout and function display, such as placing the function buttons frequently used by the user in a more prominent position.

[0058] In an embodiment of the present invention, based on step S2, a possible embodiment will be given below to non-restrictively elaborate on its specific implementation.

[0059] S201. The fusion vision-language joint encoder extracts text encoding and image encoding from the document, and uses adaptive gating to fuse the text encoding and image encoding to obtain a fused feature vector; when encoding the text, the document can be split into multiple blocks, and each block is encoded using the chunk_encoder, and then the cognitive map of the document is generated through the DiffusionTransformer, and the cognitive map is used to assist the encoder in semantic understanding.

[0060] 1. Vision-Language Joint Encoder Document Chunking: First, split the input text document into multiple text blocks according to certain rules. The splitting rules can be based on the number of sentences, character length, etc. For example, it can be split into one text block for every 10 sentences.

[0061] Block Encoding: Each text block is encoded using the chunk_encoder. The chunk_encoder can be a pre-trained language model such as BERT, DeBERTa, etc. For each text block, it is input into the chunk_encoder, and through the forward propagation process of the model, the encoded vector of each text block is obtained.

[0062] Cognitive Map Generation: The encoded vectors of all text blocks are input into the DiffusionTransformer. The DiffusionTransformer is a neural network structure based on the diffusion model. It processes the encoded vectors of text blocks through a series of diffusion steps to generate the cognitive map of the document. The cognitive map can be represented as a high-dimensional vector that contains the semantic relationships and context information between various parts of the document.

[0063] Image Encoding: The images in the document are encoded using a pre-trained vision model (such as CLIP-ViT-L). The image is input into the vision model, and through the feature extraction layer of the model, the encoded vector of the image is obtained.

[0064] 2. Adaptive Gating Fusion Gating Mechanism: Design an adaptive gating unit that receives the text encoded vector and the image encoded vector as inputs. The gating unit automatically adjusts the weights of the text and the image in the fusion process by learning the correlation between the text and the image.

[0065] Fusion Calculation: According to the weights output by the gating unit, the text encoded vector and the image encoded vector are weighted and summed to obtain the fused feature vector. The specific calculation formula is as follows: Fused Feature = Gate t × Text Encoding + (1 - Gate t ) × ImageEncoding where Gate t is the weight output by the gating unit at time t, and its value range is between [0, 1].

[0066] Output Data: Fused Feature Vector: A high-dimensional vector that fuses the feature information of the text and the image.

[0067] Cognitive Map of the Document: A high-dimensional vector that contains the semantic relationships and context information between various parts of the document, and is used to assist subsequent semantic understanding tasks.

[0068] S202. Generate a semantic topology structure containing dynamic knowledge graph information using a graph attention network based on the fused feature vector and the pre-constructed knowledge graph.

[0069] Input data: Fused feature vector: The fused feature vector generated in step S201.

[0070] Pre-constructed knowledge graph: A graph structure containing entities and the relationships between entities, such as a medical knowledge graph, an encyclopedia knowledge graph, etc.

[0071] Processing means: 1. Node embedding.

[0072] Feature vector mapping: Map the fused feature vector into the node space of the knowledge graph. A linear transformation or a non-linear mapping function can be used to convert the fused feature vector into a vector with the same feature dimension as the knowledge graph nodes.

[0073] Node initialization: Use the mapped fused feature vector as the initial feature of some nodes in the knowledge graph. These nodes can be entity nodes related to the document content.

[0074] 2. Graph attention network (GAT).

[0075] Attention mechanism: GAT uses an attention mechanism to calculate the weights between nodes. For each node, it calculates the attention weights with each neighbor node according to the features of its neighbor nodes and its own features. The attention weights represent the correlation between nodes.

[0076] Message passing: According to the calculated attention weights, message passing is carried out between nodes. Each node receives information from its neighbor nodes and fuses this information with its own features. The specific calculation formula is as follows:

[0077] where, is the attention weight between node i and node j, h i and h j are the feature vectors of node i and node j respectively, W is a learnable weight matrix, a is the parameter vector of the attention mechanism, is the set of neighbor nodes of node i, is the activation function.

[0078] To capture more complex node relationships, multiple GAT layers can be stacked together. The output of each layer is used as the input of the next layer. After multiple layers of message passing and feature fusion, the final node features are obtained.

[0079] 3. Dynamic Knowledge Graph Update.

[0080] Knowledge Fusion: During the GAT processing, the information in the document is fused with the pre - constructed knowledge graph. According to the entities and relationships in the document, the features of the nodes and the weights of the relationships in the knowledge graph are updated.

[0081] Dynamic Generation: Based on the updated knowledge graph and node features, a semantic topological structure containing dynamic knowledge graph information is generated. The semantic topological structure can be represented as a graph, where nodes represent entities, edges represent relationships between entities, and the features of nodes and edges contain information from both the document and the knowledge graph.

[0082] Output Data: Semantic Topological Structure Containing Dynamic Knowledge Graph Information: A graph structure where nodes represent entities, edges represent relationships between entities, and the features of nodes and edges contain information from both the document and the knowledge graph.

[0083] S203. Use a reinforcement learning model to generate a summary based on a preset summary control dimension and the semantic topological structure.

[0084] (1) Preset the summary control dimension, which includes information density, professionalism, and sentiment tendency.

[0085] Input: In the system initialization stage or according to the specific needs of the user, manually set the relevant parameters of the summary control dimension.

[0086] Processing Means: For Information Density: Parameter Definition: Set the range of information density. For example, it can be divided into 1 - 5 levels from low to high. Level 1 represents extremely low information density, only containing a small amount of the most core information; Level 5 represents extremely high information density, containing a large amount of detailed information in the document.

[0087] Granularity Setting: Define the measurement granularity of information density. For example, taking sentences as the unit, judge its contribution to information density according to the number and importance of key information contained in the sentence.

[0088] For Professionalism: Professional Vocabulary Library: Establish a professional vocabulary library, where different fields of documents correspond to different sets of professional vocabulary. For example, medical documents correspond to medical professional vocabulary, and financial documents correspond to financial professional vocabulary.

[0089] Level Division: Divide professionalism into different levels, such as basic, intermediate, and advanced. Basic professionalism may only contain a small number of professional vocabulary, while advanced professionalism uses a large number of professional terms and complex professional concepts.

[0090] For Sentiment Tendency: Sentiment Lexicon: Use a predefined sentiment lexicon, such as the Harbin Institute of Technology sentiment lexicon, which contains positive, negative, and neutral words.

[0091] Quantitative Representation: Quantify the sentiment tendency, for example, a numerical value between -1 and 1, where -1 represents completely negative, 1 represents completely positive, and 0 represents neutral.

[0092] Output: A data structure storing parameters related to the summary control dimension (information density level, professionalism level, quantified sentiment tendency value) for use in the subsequent summary generation process.

[0093] (2) The controllable generation method based on reinforcement learning generates a summary for the semantic topology structure according to the summary control dimension and optimizes the generation process according to the reward function.

[0094] Input: A semantic topology structure containing dynamic knowledge graph information; predefined summary control dimension parameters.

[0095] Processing Means: Define the reinforcement learning environment: State Representation: Combine the features of nodes and edges in the semantic topology structure, summary control dimension parameters, etc. into a high-dimensional vector as the state. For example, the semantic features of nodes, the relationship strength between nodes, information density level, professionalism level, quantified sentiment tendency value, etc.

[0096] Action Space: Define possible summary generation actions, such as selecting the information of a certain node to add to the summary, simplifying or expanding a certain sentence, etc.

[0097] Reward Function: Coherence Score (coherence): Use text coherence measurement methods, such as word embedding similarity, semantic coherence models between sentences, etc., to calculate the coherence score within the generated summary. For example, use a pre-trained language model to calculate the semantic similarity between adjacent sentences, and the higher the similarity, the higher the coherence score.

[0098] Key Information Coverage (coverage): Calculate the proportion of key information included in the generated summary according to the key nodes and edges in the semantic topology structure. For example, mark the key nodes in the semantic topology structure as important information and count the number of key node information included in the summary.

[0099] Redundancy: Detect the proportion of duplicate information in the summary by calculating the similarity between sentences to determine whether there is redundancy. Sentences with high similarity are considered redundant information.

[0100] Comprehensive Reward: The reward function is the weighted sum of the coherence score, key information coverage, and redundancy, with the weights being dynamically adjustable weights.

[0101] Policy Network Training: Initialize the policy network: Use a neural network (such as a recurrent neural network RNN, long short-term memory network LSTM, etc.) as the policy network and randomly initialize the network parameters.

[0102] Interactive Learning: In the reinforcement learning environment, the policy network selects actions based on the current state. After executing the actions, new states and rewards are obtained. Update the parameters of the policy network according to the rewards, so that the policy network can learn the policy of selecting optimal actions in different states. Policy gradient algorithms (such as REINFORCE, A2C, etc.) can be used for training.

[0103] Output: The preliminary summary generated according to the summary control dimension, and the optimized policy network for subsequent possible summary generation tasks.

[0104] (3) Confirm that there are associated documents for the said document. Use the cross_doc_summary function to first mine the relationships between documents through build_relation_graph to construct a document relationship graph, and then use GraphTransformer to process the document relationship graph to assist in generating a comparative summary.

[0105] Input: The current document and its generated preliminary summary; the set of confirmed associated documents.

[0106] Processing Means: Document Relationship Mining: Feature Extraction: Extract features from the current document and associated documents, including text features (such as word frequency, word vectors, etc.) and structural features (such as paragraph structure, sentence length, etc.).

[0107] Relationship Construction: Use the build_relation_graph function to construct a document relationship graph based on the feature similarity, citation relationship, topic relevance, etc. between documents. For example, calculate the cosine similarity between documents, and establish an edge connection between documents with a similarity higher than a certain threshold.

[0108] GraphTransformer Processing: Node Embedding: Embed the nodes (documents) and edges (document relationships) in the document relationship graph and convert them into low-dimensional vectors. Graph embedding algorithms (such as DeepWalk, Node2Vec, etc.) can be used for node embedding.

[0109] Transformer Architecture: Use GraphTransformer to process the document relationship graph. GraphTransformer is a graph neural network based on the Transformer architecture, which captures the complex relationships between documents through the multi-head attention mechanism. During the processing, the model updates the features of the nodes and edges in the document relationship graph and learns the association information between documents.

[0110] Comparative Abstract Generation: Information Fusion: Integrate the preliminary abstract of the current document and the associated document information learned through GraphTransformer. For example, identify important information related to the current document's theme in the associated documents but not included in the current abstract, and add it to the abstract.

[0111] Presentation of Comparative Information: Highlight the comparative information between the current document and the associated documents in the abstract, such as differences and similarities. Specific markers or expressions can be used to present the comparative information, such as using words like "In contrast", "Different from...", etc.

[0112] Output: The final abstract containing comparative information, which synthesizes the information of the current document and the associated documents and highlights the comparative relationship between the documents.

[0113] In an embodiment of the present invention, based on step S3, a possible embodiment will be given below to non-restrictively elaborate on its specific implementation.

[0114] S301. Identify the type of the document according to the custom type rules.

[0115] Pre-define multiple dimension indicators and corresponding dimension weights. The multiple dimension indicators include source, application field popularity indicator, file size, file format, and whether it is confidential. Analyze each dimension indicator of the document and calculate the weighted sum of each dimension indicator. Determine the type of the document according to the weighted sum. The application field popularity indicator is the current search popularity of the application field to which the document belongs.

[0116] Input Data: The document whose type is to be identified, and the document contains information such as text, metadata (such as file name, creation time, etc.).

[0117] Define Dimension Indicators and Weights: Source: Refers to the origin of the document, such as internally generated, externally purchased, downloaded from the network, etc. Different weights are assigned to different sources. For example, the weight of an internally generated document is 0.2, the weight of an externally purchased document is 0.3, and the weight of a document downloaded from the network is 0.1.

[0118] Application Domain Heat Index: By monitoring channels such as search engines and industry forums in real time, obtain the current search heat of the application domain to which the document belongs. Quantify the heat, for example, divide it into three levels: low, medium, and high, corresponding to different values respectively (such as low heat is 1, medium heat is 2, and high heat is 3), and assign a certain weight to this dimension, such as 0.2.

[0119] File Size: Measure the size of the document in bytes. The file size can be divided into different intervals, such as less than 1MB, 1MB - 10MB, greater than 10MB, etc. Each interval corresponds to a different value, and at the same time, assign a certain weight to this dimension, such as 0.1.

[0120] File Format: Common file formats include PDF, DOCX, TXT, etc. Different file formats may represent different types of documents. Assign corresponding weights to each file format, such as the weight of PDF format is 0.15, the weight of DOCX format is 0.1, and the weight of TXT format is 0.05.

[0121] Whether Classified: Determine whether the document contains sensitive information, which is divided into two cases: classified and unclassified, corresponding to different values respectively (such as classified is 1, unclassified is 0), and assign a certain weight to this dimension, such as 0.25.

[0122] Store each dimension index and its weight in a data structure, such as a dictionary.

[0123] Parse each dimension index of the document: Source Parsing: Determine the source of the document by checking information such as the metadata and file path of the document. For example, if the document is stored in a specific directory on the internal server, it is judged to be internally generated; if the document is obtained from an external supplier, it is judged to be externally purchased.

[0124] Obtain Application Domain Heat Index: Use web crawler technology to obtain the current search heat of the application domain to which the document belongs from channels such as search engines and industry databases, and convert it into the corresponding quantified value.

[0125] Obtain File Size: Use the file operation functions provided by the operating system to obtain the size of the document and classify it into the corresponding interval to obtain the corresponding value.

[0126] Parse File Format: Determine the file format of the document by checking the file extension.

[0127] Judge Whether Classified: Use natural language processing technology to analyze the text content of the document to identify whether it contains sensitive words or information, so as to judge whether the document is classified.

[0128] Calculate the weighted sum of the indicators for each dimension: Multiply the values of the indicators for each dimension obtained through parsing by their corresponding weights, and then sum them to obtain the comprehensive score of the document.

[0129] Determine the type of the document. Different predefined score ranges correspond to different document types. For example: Score between 0 - 1: General document; Score between 1 - 2: Important document; Score between 2 - 3: Confidential document. Determine the type to which the document belongs based on the calculated comprehensive score.

[0130] Output data: The type of the document, such as general document, important document, confidential document, etc.

[0131] S302. According to the predefined correspondence between types and storage media, and the type of the document, allocate a target storage media for the document.

[0132] Input data: The type of the document determined in step S301.

[0133] The predefined correspondence between types and storage media is stored in a data structure, such as a dictionary: type_storage_mapping = { "General document": "Local disk", "Important document": "Enterprise - level storage array", "Confidential document": "Encrypted tape library"}.

[0134] Based on the type of the document, find the corresponding storage media from the correspondence between types and storage media. For example, if the document type is "Important document", then the corresponding storage media found is "Enterprise - level storage array".

[0135] Output data: The target storage media of the document, such as local disk, enterprise - level storage array, encrypted tape library, etc.

[0136] S303. Store the document in the target storage media and obtain the storage address of the document.

[0137] Input data: The document to be stored; The target storage media determined in step S302.

[0138] Processing means: Connect to the target storage media: According to the type of the target storage media, use the corresponding interface or protocol to connect to the storage media. For example, if the target storage media is a local disk, use the file operation interface provided by the operating system; if it is an enterprise - level storage array, use the relevant protocols of Storage Area Network (SAN) or Network - Attached Storage (NAS) for connection.

[0139] Store the document: Transfer the document to the target storage medium and store it according to certain rules. For example, on a local disk, the document can be stored in a specified folder; in an enterprise-level storage array, it can be classified and stored according to information such as the type of the document and the creation time.

[0140] Obtain the storage address: After the storage is completed, obtain the storage address of the document on the target storage medium. For a local disk, the storage address can be the path of the file; for an enterprise-level storage array, the storage address can be the logical volume number, file identifier, etc.

[0141] Output data: The storage address of the document on the target storage medium.

[0142] In an embodiment of the present invention, based on step S4, a possible embodiment will be given below to non-restrictively elaborate on its specific implementation scheme.

[0143] Input data: The document abstract generated in the previous steps, which summarizes the key information of the document and contains text content; the storage address of the document on the target storage medium, which is an important identifier for locating the document.

[0144] Processing means: 1. Abstract preprocessing.

[0145] Text cleaning: Remove special characters in the abstract, such as punctuation marks, HTML tags (if the abstract contains web content), etc. Convert all text to lowercase to unify the character case and avoid affecting subsequent indexing and retrieval due to different cases.

[0146] Word segmentation processing: For a Chinese abstract, a Chinese word segmentation tool such as jieba can be used to split the abstract into individual words. For an English abstract, a simple space splitting method can be used to split the sentence into words.

[0147] 2. Index optimization.

[0148] Stop word removal: Define a stop word list containing common words with no practical meaning, such as "de" "shi" "zai" etc. (Chinese) or "the" "and" "is" etc. (English). Remove the index entries corresponding to these stop words from the inverted index to reduce the size of the index and improve the retrieval efficiency.

[0149] 3. Index storage.

[0150] Store the constructed indexes (inverted index and metadata index) in a suitable storage medium, such as a database (e.g., MySQL, Elasticsearch, etc.) or a file system. If Elasticsearch is used to store the indexes, its Python client library can be used for operations.

[0151] Output data: The document index stored in the specified storage medium, which can be used for subsequent retrieval services. Users can quickly locate the storage addresses of relevant documents through methods such as keyword queries.

[0152] Please refer to Figure 2 , the following is a specific management process: 1. Design a flexible multi-protocol adapter module using a microservices architecture. Each adapter is specifically responsible for docking a storage protocol (such as FTP, SFTP, NFS, object storage, etc.) and abstracting a unified outer interface to ensure that the internal storage protocol differences are transparent to the outside. Include a dynamic loading mechanism for instantaneously loading newly developed storage protocol adapters to ensure the scalability of the system.

[0153] 2. Deploy and integrate the large model service as an intelligent service center, responsible for intelligent classification, content analysis, and retrieval optimization of documents, and understanding user intentions.

[0154] 3. Establish a configuration management database to store configuration templates of various protocols and user-defined configurations, and record user personalization information based on user operations.

[0155] 4. Extract key information of the document during document upload, analyze the document content using the large model, and automatically classify the document into a suitable storage medium in combination with the classification rules set by the user.

[0156] 5. Design a feedback mechanism to collect user evaluations of the classification results and feedback them to the large model training process to continuously optimize the classification algorithm.

[0157] 6. Apply the large model for text mining and knowledge graph construction to establish an efficient document index system.

[0158] 7. Provide a document retrieval service for the managed documents. Integrate the document index system according to the extracted document content, support functions such as natural language queries, tag filtering, and time range selection, and use the large model to parse the query intention to provide accurate retrieval results.

[0159] In some embodiments, the management system for multi-storage protocol documents may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the management system for multi-storage protocol documents can be stored in the memory of the computer device and executed by at least one processor to execute (see detailsFigure 1 Function for managing multi - storage protocol documents (description).

[0160] In this embodiment, the multi - storage protocol document management system can be divided into multiple functional modules according to the functions it performs, as Figure 3 shown. The module referred to in the present invention means a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and are stored in the memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0161] A receiving module, configured to receive documents through an extended interface that supports multiple storage protocols; An identification module, configured to identify the semantic information of the document using a large - model and generate an abstract for the document based on the semantic information; A storage module, configured to obtain the type of the document, allocate a target storage medium for the document according to the type, and obtain the storage address of the document in the target storage medium; An indexing module, configured to generate an index for the document according to the abstract and the storage address, and the index is used to support retrieval services.

[0162] Figure 4 The multi - storage protocol document management method provided in the embodiments of the present application can be applied to a device. Those skilled in the art can understand that the device structure involved in the embodiments of the present invention does not constitute a limitation on the device. The device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. In the embodiments of the present invention, the device includes but is not limited to laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown in the figure, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the embodiments of the present application described herein and / or claimed.

[0163] Among them, the device 400 may include: a processor 410, a memory 420, and a communication unit 430. These components communicate through one or more buses. Those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation on the present invention. It can be a bus - shaped structure, a star - shaped structure, and may also include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0164] Among them, the memory 420 can be used to store the execution instructions of the processor 410. The memory 420 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disc. When the execution instructions in the memory 420 are executed by the processor 410, the device 400 can execute some or all of the steps in the above method embodiments.

[0165] The processor 410 is the control center of the storage device, connecting various parts of the entire electronic device through various interfaces and lines. By running or executing the software programs and / or modules stored in the memory 420, and by calling the data stored in the memory, it executes various functions of the electronic device and / or processes data. The processor can be composed of an integrated circuit (IC). For example, it can be composed of a single packaged IC, or can be composed of multiple packaged ICs with the same or different functions connected together. For example, the processor 410 can include only a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single arithmetic core or can include multiple arithmetic cores.

[0166] The communication unit 430 is used to establish a communication channel so that the storage device can communicate with other devices. It receives user data sent by other devices or sends user data to other devices.

[0167] The present invention also provides a computer storage medium. Among them, the computer storage medium can store a program, and when the program is executed, it can include some or all of the steps in the various embodiments provided by the present invention. The storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), etc.

[0168] Those skilled in the art can clearly understand that the technologies in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc, etc., various media that can store program codes, including several instructions for causing a computer device (which can be a personal computer, a server, or a second device, a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0169] For the same or similar parts among the various embodiments in this specification, reference can be made to each other. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the descriptions in the method embodiments.

[0170] In several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there can be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the systems or modules can be in electrical, mechanical, or other forms.

[0171] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they can be located in one place, or they can be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] In addition, in each embodiment of the present invention, the various functional modules can be integrated in a processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0173] Although the present invention has been described in detail by reference to the accompanying drawings and in conjunction with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, those of ordinary skill in the art can make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions should all be within the scope of the present invention / Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, and they should all be covered within the protection scope of the present invention.

Claims

1. A management method for multi-storage protocol documents, characterized in that including: receiving a document through an extended interface that supports multiple storage protocols; using a large model to identify the semantic information of the document and generating an abstract for the document based on the semantic information; obtaining the type of the document, allocating a target storage medium for the document according to the type, and obtaining the storage address of the document in the target storage medium; generating an index for the document according to the abstract and the storage address, where the index is used to support the retrieval service.

2. The method according to claim 1, characterized in that, The method further includes: designing a unified transport layer, protocol adaptation layer, and protocol implementation layer for the interface. The unified transport layer is used to externally expose a standard RESTful API, process document upload and download requests, and perform format conversion; the protocol adaptation layer is used to define an abstract interface specification; the protocol implementation layer includes specific implementation modules for each storage protocol; defining a core interface; implementing a dynamic extension mechanism for the interface using the Java SPI or OSGi framework; defining a JSON Schema to describe the parameter structure for each storage protocol.

3. The method according to claim 1, wherein using a large model to identify the semantic information of the document and generating an abstract for the document based on the semantic information, including: fusing a vision-language joint encoder to extract text encoding and image encoding from the document, and using an adaptive gate to fuse the text encoding and image encoding to obtain a fused feature vector; using a graph attention network to generate a semantic topological structure containing dynamic knowledge graph information based on the fused feature vector and a pre-constructed knowledge graph; using a reinforcement learning model to generate an abstract based on a preset abstract control dimension and the semantic topological structure.

4. The method according to claim 3, wherein The method further includes: during text encoding, the document can be split into multiple blocks, each block is encoded using chunk_encoder, and then a cognitive map of the document is generated through DiffusionTransformer, where the cognitive map is used to assist the encoder in semantic understanding.

5. The method according to claim 3, wherein using a reinforcement learning model to generate an abstract based on a preset abstract control dimension and the semantic topological structure, including: presetting an abstract control dimension, where the abstract control dimension includes information density, professionalism, and sentiment tendency; generating an abstract for the semantic topological structure according to the abstract control dimension based on a controllable generation method of reinforcement learning, and optimizing the generation process according to a reward function; confirming that there are associated documents for the document, using the cross_doc_summary function to first mine the relationships between documents through build_relation_graph to construct a document relationship graph, and then using GraphTransformer to process the document relationship graph to assist in generating a comparative abstract.

6. The method according to claim 1, characterized in that, obtaining the type of the document, allocating a target storage medium for the document according to the type, and obtaining the storage address of the document in the target storage medium, including: identifying the type of the document according to a custom type rule; allocating a target storage medium for the document according to the predefined correspondence between the type and the storage medium and the type of the document; storing the document in the target storage medium and obtaining the storage address of the document.

7. The method according to claim 1, wherein identifying the type of the document according to a custom type rule, including: Pre-define multiple dimension indicators and corresponding dimension weights. The multiple dimension indicators include source, application domain popularity indicator, file size, file format, and whether it is confidential. Analyze each dimension indicator of the document, calculate the weighted sum of each dimension indicator, and determine the type of the document according to the weighted sum. The application domain popularity indicator is the current retrieval popularity of the application domain to which the document belongs.

8. A management system for multi-storage protocol documents, characterized in that, It includes: A receiving module for receiving a document through an extended interface that supports multiple storage protocols. An identification module for using a large model to identify the semantic information of the document and generating an abstract for the document based on the semantic information. A storage module for obtaining the type of the document, allocating a target storage medium for the document according to the type, and obtaining the storage address of the document in the target storage medium. An indexing module for generating an index of the document according to the abstract and the storage address, and the index is used to support retrieval services.

9. A device, characterized in that, It includes: A memory for storing a management program for multi-storage protocol documents. [[ID=IO]]A processor for implementing the steps of the management method for multi-storage protocol documents as described in any one of claims 1-7 when executing the management program for multi-storage protocol documents.

10. A computer-readable storage medium storing a computer program, characterized in that, The management program for multi-storage protocol documents is stored on the readable storage medium, and when the management program for multi-storage protocol documents is executed by a processor, the steps of the management method for multi-storage protocol documents as described in any one of claims 1-7 are implemented.

Images