User file management method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a user file management method and device, electronic equipment and storage medium, relates to the technical field of data processing, and through analyzing and converting a target file uploaded by a user into a unified first semantic vector, the semantic vector is stored into a dynamic knowledge base together with metadata and is synchronized in real time with newly added content through an incremental updating mechanism, a cross-modal knowledge graph is constructed based on a label classification rule and associated logic, a second semantic vector is generated after receiving a natural language query to perform joint retrieval of the semantic vector and the metadata, the relevance of the optimization result is optimized with the aid of the cross-modal knowledge graph, and finally a multi-modal answer containing text, chart links and the like is generated through a pre-training model and is returned through a standardized interface, so that the problems that heterogeneous file retrieval is difficult due to the lack of a unified semantic processing mechanism in the prior art, the knowledge base is static and cannot timely incorporate newly added content, and the retrieval result is fragmented due to the lack of cross-modal knowledge association can be solved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a user file management method and apparatus, electronic device and storage medium. Background Technology

[0002] With the deep integration of cloud computing and artificial intelligence technologies, cloud-based file management systems, as crucial infrastructure for enterprise-level data storage and knowledge management, have been widely applied in areas such as document collaboration, remote work, and multimodal data storage. Among related technologies, intelligent question-answering systems tailored to specific scenarios have been built through the collaborative operation of OCR, CV, and NLP technologies. Specifically, this technology system covers the entire process from document parsing to knowledge retrieval, including key aspects such as structured processing of legal provisions, construction of knowledge graphs of regulations and systems, and associated storage of Markdown documents and audio-visual resources. Among these, the question-answering system based on a large language model achieves interactive natural language retrieval through a combination of BERT intent classification and the LLaMA-7B generation engine. However, existing solutions mainly focus on single-modal document processing and lack a unified parsing framework for multimodal files such as PDFs, scanned documents, videos, and images.

[0003] However, existing cloud drive file management methods directly employ keyword retrieval and manually constructed knowledge bases, failing to establish a unified semantic representation system for multimodal data. This can lead to inconsistent parsing results across file types, delayed knowledge base updates, and a lack of semantic association capabilities. Specifically, existing systems rely on OCR to extract text in text file processing but cannot parse chart structures; in video file processing, they only support timestamp retrieval and lack keyframe semantic recognition. Furthermore, knowledge bases built using traditional search engines like Elasticsearch lack a hybrid storage architecture integrated with the Faiss vector database, resulting in manual annotation of new files and update delays of 24-72 hours. In addition, existing technologies typically employ single-file in-search strategies, failing to construct a multi-dimensional network of relationships between files, charts, terms, and entities using knowledge graphs. This impacts the response efficiency for complex semantic queries such as "Q3 objective in the contract," limiting the evolution of cloud drive systems towards cognitive intelligence. Summary of the Invention

[0004] This disclosure provides a user file management method, apparatus, electronic device, and storage medium. Its main objective is to at least partially address one of the technical problems in the related art.

[0005] According to a first aspect of this disclosure, a user file management method is provided, comprising: The target file uploaded by the user is parsed, and the parsing result is converted into a unified first semantic vector; The first semantic vector and the metadata of the target file are stored together in a dynamic knowledge base. The newly added file content in the target file is synchronized in real time through an incremental update mechanism, and a cross-modal knowledge graph is constructed based on predefined tag classification rules and association logic. It receives natural language queries input by users, parses the query intent and generates a second semantic vector, performs joint retrieval of semantic vectors and metadata in a dynamic knowledge base, and optimizes the cross-modal relevance of retrieval results based on a cross-modal knowledge graph; The search results and query intent are input into a pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information, which is then returned to the user through a standardized interface.

[0006] Optionally, the target file uploaded by the user is parsed, and the parsing result is converted into a unified first semantic vector, including: Based on the file type of the target file, the corresponding parsing strategy is matched. The target file includes files of multiple modalities. According to the parsing strategy, the target file is parsed, and the parsing result includes at least one of the following: text content, image features, and keyframe information.

[0007] Optionally, based on the file type of the target file, a corresponding parsing strategy is matched, including: The dynamic conditional encoding module maps the file format of the target file to a predefined parsing strategy. Text files are mapped to text extraction and chart parsing strategies, video files are mapped to keyframe extraction and audio-to-text strategies, and table files are mapped to table structuring and cell semantic vectorization strategies.

[0008] Optionally, the first semantic vector representation and the metadata of the target file are stored together in a dynamic knowledge base, including: Full-text indexes are built based on a full-text search engine to support keyword retrieval, and vector indexes are built based on a vector database to support semantic similarity retrieval. A three-tiered storage architecture of text, vectors, and metadata is formed by using a metadata database to store file attributes, parse metadata, and associate document identifiers.

[0009] Optionally, a joint retrieval of semantic vectors and metadata in the dynamic knowledge base can be performed, including: The query intent is classified twice based on a natural language processing model, and a differentiated retrieval context is constructed based on the classification results. By analyzing the relationships between files, charts, terms, and entities through a pre-built cloud disk knowledge graph, documents containing entity associations are prioritized in the search results to achieve cross-modal chained retrieval.

[0010] Optionally, the search results and query intent are input into a pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information, including: When the search results contain charts or keyframe information, a structured response output is achieved by embedding images, hyperlinks, and timestamps in rich text format. For table-based search results, a table parsing model is used to parse the semantic relationships between cells and generate a formatted answer that includes a description of the table structure and key data fields.

[0011] Optional, also includes: Maintain multi-turn dialogue context information, store historical questions, temporary variables and user preference settings, and call the context data in subsequent question-and-answer processes to adjust the parameter weights of the intent recognition model in order to optimize the semantic understanding accuracy of complex queries.

[0012] According to a second aspect of this disclosure, a user file management device is provided, comprising: The parsing unit is used to parse the target file uploaded by the user and convert the parsing result into a unified first semantic vector; The building unit is used to store the first semantic vector and the metadata of the target file together in the dynamic knowledge base, synchronize the newly added file content in the target file in real time through the incremental update mechanism, and build a cross-modal knowledge graph based on predefined tag classification rules and association logic; The retrieval unit is used to receive natural language queries input by users, parse the query intent and generate a second semantic vector, perform joint retrieval of semantic vectors and metadata in the dynamic knowledge base, and optimize the cross-modal relevance of the retrieval results based on the cross-modal knowledge graph. The generation unit is used to input the retrieval results and query intent into the pre-trained model, generate a multimodal natural language response containing text, chart links, or keyframe information, and return it to the user through a standardized interface.

[0013] Optionally, the parsing unit is also used for: Based on the file type of the target file, the corresponding parsing strategy is matched. The target file includes files of multiple modalities. According to the parsing strategy, the target file is parsed, and the parsing result includes at least one of the following: text content, image features, and keyframe information.

[0014] Optionally, based on the file type of the target file, a corresponding parsing strategy is matched, including: The dynamic conditional encoding module maps the file format of the target file to a predefined parsing strategy. Text files are mapped to text extraction and chart parsing strategies, video files are mapped to keyframe extraction and audio-to-text strategies, and table files are mapped to table structuring and cell semantic vectorization strategies.

[0015] Optionally, building blocks are also used for: Full-text indexes are built based on a full-text search engine to support keyword retrieval, and vector indexes are built based on a vector database to support semantic similarity retrieval. A three-tiered storage architecture of text, vectors, and metadata is formed by using a metadata database to store file attributes, parse metadata, and associate document identifiers.

[0016] Optionally, the retrieval unit is also used for: The query intent is classified twice based on a natural language processing model, and a differentiated retrieval context is constructed based on the classification results. By analyzing the relationships between files, charts, terms, and entities through a pre-built cloud disk knowledge graph, documents containing entity associations are prioritized in the search results to achieve cross-modal chained retrieval.

[0017] Optionally, the generating unit is also used for: When the search results contain charts or keyframe information, a structured response output is achieved by embedding images, hyperlinks, and timestamps in rich text format. For table-based search results, a table parsing model is used to parse the semantic relationships between cells and generate a formatted answer that includes a description of the table structure and key data fields.

[0018] Optional, also includes: The maintenance unit is used to maintain multi-turn dialogue context information, store historical questions, temporary variables and user preference settings, and call the context data in subsequent question-and-answer processes to adjust the parameter weights of the intent recognition model in order to optimize the semantic understanding accuracy of complex queries.

[0019] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0020] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect above.

[0021] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0022] This disclosure provides a user file management method, apparatus, electronic device, and storage medium. It parses user-uploaded target files and converts them into a unified first semantic vector. This semantic vector, along with metadata, is stored in a dynamic knowledge base, and new content is updated in real-time via an incremental update mechanism. Simultaneously, a cross-modal knowledge graph is constructed based on tag classification rules and association logic. Upon receiving a natural language query, a second semantic vector is generated for joint retrieval of the semantic vector and metadata. The cross-modal knowledge graph optimizes the relevance of results. Finally, a pre-trained model generates a multimodal answer containing text, chart links, etc., and returns it through a standardized interface. Therefore, it solves the problems in existing technologies, such as difficulty in retrieving heterogeneous files due to the lack of a unified semantic processing mechanism, the inability to incorporate new content into a static knowledge base, fragmented search results due to the lack of cross-modal knowledge association, and insufficient semantic understanding and inability to generate multimodal answers due to reliance on keyword matching. This achieves unified semantic management and dynamic updates of cloud drive files, improves the accuracy and cross-modal relevance of user query results, generates multi-dimensional, highly relevant natural language answers, and optimizes the comprehensiveness of the knowledge base and the user experience.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 A flowchart illustrating a user file management method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a user file management device provided in an embodiment of the present disclosure; Figure 3 A schematic block diagram of an example electronic device provided for embodiments of this disclosure. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] The user file management method, apparatus, electronic device, and storage medium of this disclosure are described below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating a user file management method provided in an embodiment of the present disclosure.

[0028] like Figure 1 As shown, the method includes the following steps: Step 101: Parse the target file uploaded by the user and convert the parsing result into a unified first semantic vector.

[0029] In this embodiment, to achieve unified semantic management of various file types within the cloud drive, this step first performs a parsing operation on the target file uploaded by the user to extract various semantic information contained in the file. Then, the extracted semantic information is converted into a first semantic vector with a unified format or dimension, thereby eliminating the differences in semantic representation between different types of target files and providing a standardized data foundation for subsequent knowledge base storage and retrieval. Specifically, the parsing of the target file can be adapted to the specific file type to ensure the completeness and accuracy of semantic information extraction. The unified processing of the first semantic vector needs to be achieved through preset semantic coding rules or models, so that parsing results from different sources and formats can be mapped to the same semantic space. As one implementation method, target files can be parsed by integrating a multimodal parsing module. For example, optical character recognition technology can be used to extract text content from text files such as PDFs and scanned documents; audio and video processing algorithms can be used to extract keyframe features and audio-to-text information from video files; and object detection algorithms can be used to identify entity features from image files. Then, a semantic coding model is used to convert the above parsing results into a semantic vector with a unified dimension.

[0030] By parsing the target file and performing unified semantic vector conversion, the problem of poor compatibility in subsequent processing caused by the heterogeneity of semantic representations of multiple file types in existing technologies is effectively solved. This provides a prerequisite for the unified storage and efficient semantic retrieval of target files in a dynamic knowledge base. At the same time, it improves the semantic processing capabilities of the cloud disk file management system for multimodal files and ensures the consistency and availability of semantic information of different types of files.

[0031] Step 102: Store the first semantic vector and the metadata of the target file together in the dynamic knowledge base, synchronize the newly added file content in the target file in real time through the incremental update mechanism, and construct a cross-modal knowledge graph based on predefined label classification rules and association logic.

[0032] In this embodiment, to achieve dynamic management and cross-modal association of cloud disk file knowledge, this step first stores the aforementioned first semantic vector and the target file's metadata together in a dynamic knowledge base, enabling the semantic and attribute information of the file to be stored in association, providing data support for subsequent retrieval. Then, an incremental update mechanism is used to synchronize newly added file content in the target file in real time, ensuring that the information in the dynamic knowledge base remains consistent with the actual file storage state of the cloud disk, avoiding knowledge lag. Simultaneously, based on predefined tag classification rules, the stored file-related information is classified and organized, and association relationships between different files and different modal information are established according to preset association logic, thereby constructing a cross-modal knowledge graph to achieve structured and associated knowledge management. As one implementation method, the dynamic knowledge base can adopt an architecture that supports hybrid storage of semantics and metadata. The incremental update mechanism can be implemented by periodically scanning newly added files in the cloud disk or responding to user-defined update trigger conditions. Predefined tag classification rules may include classification by file type and business scenario, and association logic may include subordinate relationships between files, content association relationships, etc.

[0033] By associating semantic vectors with metadata, performing real-time incremental updates, and constructing cross-modal knowledge graphs, this technology effectively solves the problems of reliance on manual maintenance leading to delayed updates and lack of cross-modal information association in existing technologies. It not only ensures the timeliness and integrity of the knowledge base but also provides a structured knowledge association foundation for subsequent cross-modal semantic retrieval, thereby improving the intelligence and efficiency of cloud disk file knowledge management.

[0034] Step 103: Receive the natural language query input by the user, parse the query intent and generate a second semantic vector, perform joint retrieval of the semantic vector and metadata in the dynamic knowledge base, and optimize the cross-modal relevance of the retrieval results based on the cross-modal knowledge graph.

[0035] In this embodiment, to respond to user queries for cloud drive file knowledge and achieve accurate and relevant search results, this step first receives the user's natural language query. Through a preset intent parsing logic, it identifies the core needs and target direction of the user's query. Then, based on semantic encoding rules consistent with the first semantic vector, it converts the parsed query intent into a second semantic vector that can be compared with data in the dynamic knowledge base, ensuring that the query and stored information are related within the same semantic space. Subsequently, based on the second semantic vector, it performs semantic-level retrieval on the first semantic vector stored in the dynamic knowledge base, while simultaneously performing attribute-level filtering based on the target file's metadata, achieving joint retrieval of semantics and metadata to narrow the search scope and improve accuracy. Finally, it calls the constructed cross-modal knowledge graph, and based on the relationships between different modal information recorded therein, optimizes the relevance of the preliminary search results to ensure that the output results cover cross-modal content related to the query. As one implementation method, intent parsing can be implemented using an intent classification model, the second semantic vector can be generated through a semantic encoding model, joint retrieval can combine full-text retrieval and vector retrieval technologies, and the optimization of the cross-modal knowledge graph can be completed based on the preset relationships between files, charts, and entities within it.

[0036] By employing intent parsing, joint retrieval, and cross-modal knowledge graph optimization, this approach effectively addresses the issues of low accuracy in single-keyword retrieval and lack of relevance in cross-modal retrieval results in existing technologies. It not only improves the matching accuracy between user query needs and cloud drive file knowledge but also ensures the cross-modal integrity of retrieval results, providing high-quality data support for generating comprehensive multimodal answers in the future.

[0037] Step 104: Input the search results and query intent into the pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information, and return it to the user through a standardized interface.

[0038] In this embodiment, to transform the retrieved cloud drive file knowledge into intuitive information output that meets user needs, this step first obtains the aforementioned search results and the parsed query intent, and passes them as collaborative inputs to a pre-trained model. The pre-trained model, based on preset semantic generation logic, combines the core content of the files in the search results with the key requirements of the query intent to generate a multimodal natural language response containing text explanations and chart links or video keyframe information for related display. This ensures that the response conveys accurate semantics and has multi-dimensional information presentation capabilities. Subsequently, the generated multimodal response is pushed to the user interaction terminal through a standardized interface, ensuring that different types of user access platforms (such as web interfaces and mobile applications) can stably receive and correctly display the response content. As one implementation method, the pre-trained model can adopt a model that supports Retrieval Augmentation (RAG) technology, such as a model fine-tuned based on LLaMA-7B. The generated multimodal response can embed chart links and keyframe preview information in rich text format, and the standardized interface can follow common API communication specifications to adapt to multi-terminal access.

[0039] By generating multimodal answers through pre-trained models and transmitting them via standardized interfaces, this technology effectively solves the problems of existing question-answering systems that can only output single text, cannot intuitively present charts or keyframe information, and have poor interface adaptability leading to unstable multi-terminal display. This not only improves the completeness and intuitiveness of the answers but also ensures compatibility across multiple devices, further optimizing the efficiency and experience for users to obtain knowledge from cloud drive files.

[0040] This disclosure provides a user file management method. It parses user-uploaded target files and converts them into a unified first semantic vector. This semantic vector, along with metadata, is stored in a dynamic knowledge base, and new content is updated in real-time via an incremental update mechanism. Simultaneously, a cross-modal knowledge graph is constructed based on tag classification rules and association logic. Upon receiving a natural language query, a second semantic vector is generated for joint retrieval of the semantic vector and metadata. The cross-modal knowledge graph optimizes the relevance of results. Finally, a pre-trained model generates multimodal answers containing text, chart links, etc., and returns them through a standardized interface. Therefore, this method solves the problems in existing technologies, such as difficulty in retrieving heterogeneous files due to the lack of a unified semantic processing mechanism, the inability to incorporate new content into a static knowledge base, fragmented search results due to a lack of cross-modal knowledge association, and insufficient semantic understanding and inability to generate multimodal answers due to reliance on keyword matching. This achieves unified semantic management and dynamic updates of cloud drive files, improves the accuracy and cross-modal relevance of user query results, generates multi-dimensional, highly relevant natural language answers, and optimizes the comprehensiveness of the knowledge base and the user experience.

[0041] Within the technical framework disclosed in step 101, the target file uploaded by the user is parsed, and the parsing result is converted into a unified first semantic vector. This is further specified as follows: based on the file type of the target file, a corresponding parsing strategy is matched, and the target file includes files of multiple modalities; according to the parsing strategy, the target file is parsed, and the parsing result includes at least one of text content, image features, and keyframe information.

[0042] Furthermore, in this embodiment of the disclosure, the corresponding parsing strategy is matched based on the file type of the target file, specifically including: mapping the file format of the target file to a predefined parsing strategy through a dynamic conditional encoding module, wherein text files are mapped to a text extraction and chart parsing strategy, video files are mapped to a keyframe extraction and audio-to-text strategy, and table files are mapped to a table structuring and cell semantic vectorization strategy.

[0043] Specifically, when performing parsing operations on user-uploaded target files, the dynamic conditional encoding module first completes the identification of file formats and matching of parsing strategies: this module first reads the extension of the target file (such as .pdf, .mp4, .xlsx) and MIME type to determine the file category (text, video, table), and then associates the file category with the corresponding parsing strategy according to the preset mapping rules. For text files (such as PDF and DOCX), a text extraction and chart parsing strategy is adopted. PaddleOCR technology is used to extract the text content within the file, while a chart recognition submodule is activated to perform contour detection and element extraction on embedded charts (such as bar charts and flowcharts) in the file, obtaining chart titles, data labels, and axis information. For video files (such as MP4 and AVI), a keyframe extraction and audio-to-text strategy is executed. FFmpeg is used to extract video keyframes at 5-second intervals, and the YOLOv8 object detection algorithm is used to identify entities (such as product images and scene scenes) in the keyframes to generate image features. Then, the Whisper audio-to-text model is used to convert the video audio track into text. For table files (such as XLSX and CSV), a table structuring and cell semantic vectorization strategy is adopted. The regular expression rule engine is used to parse the row / column structure and cell data of the table, and the TAPAS pre-trained model is used to semantically encode the content of each cell to generate cell-level semantic features. The final output parsing result contains at least one of the following: text content, image features, or keyframe information of the corresponding file.

[0044] By using a dynamic conditional encoding module, the file type and parsing strategy are accurately matched, avoiding the problem of insufficient adaptation of a single parsing method to multimodal files. This ensures that the core information (such as text, charts, and keyframes) of different file types can be completely extracted. At the same time, the dedicated parsing methods for various file types improve the accuracy of semantic information extraction, providing high-quality and complete raw data for subsequent conversion into a unified first semantic vector, effectively reducing subsequent retrieval biases caused by insufficient parsing.

[0045] Under the technical solution framework revealed in step 102, the first semantic vector representation and the metadata of the target file are jointly stored in a dynamic knowledge base, which is further specified as follows: a full-text index is established based on a full-text search engine to support keyword retrieval, and a vector index is established based on a vector database to support semantic similarity retrieval; a metadata database is used to store file attributes, parse metadata and associated document identifiers, forming a three-layer storage architecture of text, vector and metadata.

[0046] Specifically, when storing the first semantic vector and the target file's metadata together in the dynamic knowledge base, a three-layer storage architecture of text, vectors, and metadata is adopted for structured management: The full-text index layer is built on the Elasticsearch full-text search engine, creating an inverted index of the text content parsed from the target file (such as document body and audio-to-text results) and keyword-related attributes in the metadata (such as file name and tags) to support precise retrieval of keywords such as "Q3 target in the contract"; the vector index layer is built on the Faiss vector database, storing the converted first semantic vector (including text semantic vectors, image feature vectors, and table cell semantic vectors) in the vector index, through... The calculation of cosine similarity between vectors supports semantic similarity retrieval such as "contracts semantically similar to file A"; the meta-database layer uses a MySQL relational database or a MongoDB unstructured database to store the target file's file attributes (such as author, file size, creation time), parsed metadata (such as the index ID of the text vector in Faiss, the association identifier between the chart and the file), and associated document identifiers (such as the attachment ID corresponding to the main document, the cross-modal associated file ID). Each record in the meta-database is associated with the Elasticsearch full-text index entry and the Faiss vector index entry through a unique file identifier, ensuring that the three layers of data can be traced and accessed from each other.

[0047] By leveraging the collaborative efforts of full-text indexing, vector indexing, and metadata database, the problem of a single storage method being unable to simultaneously satisfy keyword retrieval and semantic retrieval is solved. Furthermore, the metadata database enables structured management of file attributes and parsed information, preventing data chaos. At the same time, the correlation mapping between the three layers of data ensures that data can be quickly accessed across layers during subsequent joint retrieval, significantly improving the retrieval efficiency and data management standardization of the dynamic knowledge base.

[0048] Under the technical solution framework revealed in step 103, the semantic vectors and metadata in the dynamic knowledge base are jointly retrieved, which is further specified as follows: the query intent is classified twice based on the natural language processing model, and a differentiated retrieval context is constructed according to the classification results; the relationship between files, charts, terms and entities is analyzed through the pre-built cloud disk knowledge graph, and documents containing entity associations are matched first in the retrieval results to realize cross-modal chain retrieval.

[0049] Specifically, when jointly retrieving semantic vectors and metadata in a dynamic knowledge base, two intent classifications are first performed based on a natural language processing model to construct a differentiated retrieval context: The first classification uses the BERT intent classification model to initially classify the user's natural language query and determine the core query type, such as "query", "generation", and "classification"; the second classification further refines the results of the initial classification. For example, if the initial classification is "query", it is further subdivided into subtypes such as "text content query", "chart data query", and "keyframe information query". Then, a differentiated retrieval context is constructed based on the subdivided intent type. For "text content query", text fragments with high semantic similarity to the second semantic vector are selected from the knowledge base as context. For "chart data query", metadata containing information related to the target chart (such as chart ID and file identifier) ​​is extracted first as context. Simultaneously, a pre-built cloud disk knowledge graph is invoked. This knowledge graph uses files, charts, terms, and entities as nodes and "contains," "associates," and "references" as edges to store the relationships between nodes (e.g., a file contains a chart, or an entity appears in a video keyframe). During the retrieval process, the knowledge graph analyzes the relationships between entities (e.g., product names, data metrics) involved in the query intent and other nodes. Among the candidate documents obtained in the initial retrieval, documents containing the associated entity are prioritized for matching. For example, when a user queries "quarterly sales of product A," in addition to returning a text file containing sales data, a sales chart file associated with the sales data and a video file recording the introduction of product A (including relevant keyframes) are also prioritized for return, realizing cross-modal chained retrieval.

[0050] By constructing differentiated search contexts through two intent classifications, the problem of insufficient context adaptability caused by single intent classification is avoided, significantly improving the matching accuracy between search context and query intent. At the same time, relying on cloud disk knowledge graph to parse entity relationships and realize cross-modal chain search, the problem of fragmented results in traditional search is effectively solved, ensuring that users can obtain cross-modal information that is fully related to their query needs, thus improving the completeness and usability of search results.

[0051] Within the technical framework revealed in step 104, the search results and query intent are input into the pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information. Specifically, when the search results contain chart or keyframe information, images, hyperlinks, and timestamps are embedded in rich text format to achieve a structured response output. For table-type search results, a table parsing model is used to parse the semantic relationships of cells and generate a formatted response containing table structure descriptions and key data fields.

[0052] Specifically, when inputting search results and query intent into a pre-trained model to generate multimodal natural language responses, differentiated processing is applied to different types of search results: When search results contain charts or keyframe information, the pre-trained model (such as a fine-tuned model) first obtains the storage path of the chart (such as a file identifier in a distributed object storage system), the video source file path corresponding to the keyframe, and the timestamp information (such as the minute and second position of the keyframe in the video) from the metadata database. Then, it embeds this information into the response in a structured manner using rich text formats such as Markdown. Among these, the chart is displayed as a thumbnail preview and associated with a hyperlink, the hyperlink pointing to the access address of the original chart in the storage system, and the keyframe is marked with the corresponding timestamp (such as "..."). The keyframe at 00:02:15 in the video is used to link to the corresponding playback position in the video file. For table-type search results, the TAPAS table parsing model is first called to process the retrieved table data. This model analyzes the row and column structure of the table and the relationship between cell data (such as the correspondence between row headers and column data) to extract the core structural description of the table (such as "the table contains three columns: 'quarter', 'sales', and 'growth rate,' covering data from Q1 to Q4 of 2023") and key data fields (such as "sales in Q4 of 2023 were 1.2 million units, with a year-on-year growth rate of 15%)." Then, the pre-trained model generates a formatted answer containing the table structure description and key data fields based on these parsing results, ensuring that the table information is clear and easy to understand.

[0053] By embedding multimodal elements in rich text format, the problem of traditional text answers being unable to intuitively present charts and keyframe information is solved, allowing users to quickly locate and view related non-text content. At the same time, relying on a table parsing model to generate formatted answers avoids the structural chaos of directly outputting table data, significantly improving the readability of table-type information and further optimizing the practicality of multimodal answers and the efficiency of users obtaining information.

[0054] Within the framework of the technical solutions disclosed in the foregoing embodiments, the embodiments of this application further include: maintaining multi-turn dialogue context information, storing historical questions, temporary variables and user preference settings, and calling the context data to adjust the parameter weights of the intent recognition model in subsequent question-and-answer processes, so as to optimize the semantic understanding accuracy of complex queries.

[0055] Specifically, in the technical solution of this application embodiment, the process of maintaining multi-turn dialogue context information relies on the dialogue management module and history management module of the user interaction layer to work together: After the user initiates the first query, the dialogue management module will assign a unique identifier to the user session and establish a context storage unit based on the identifier to store the historical questions in each round of question and answer in real time (such as the user's previous "query the sales contract of Q3 2023" and "the product specifications in the contract") and temporary variables (such as the user's temporary filter condition "only view data in East China"); at the same time, the history management module will extract user preference settings from the user operation log (such as the user often prefers to view results in chart form and defaults to searching for files in the last 6 months), and store these preferences in the metadata database after associating them with the user's unique identifier. In subsequent question-and-answer processes, when a user initiates a new complex query (such as "quarterly sales volume corresponding to this product specification"), the system first reads historical context data from the dialogue management module through the user's session identifier, and then combines it with the user's preference settings in the metadata database to generate parameter adjustment coefficients for the intent recognition model. For example, if the historical context shows that the user has repeatedly paid attention to data related to "quarterly sales volume", the system will increase the weight of feature parameters related to "sales volume" and "quarter" in the intent recognition model (such as the BERT model) by 15%-20%. If the user prefers chart results, the recognition weight of intents related to "chart data query" will be increased simultaneously. By dynamically adjusting the model parameter weights, the system optimizes the semantic understanding accuracy of complex queries.

[0056] By maintaining the context of multi-turn dialogues and dynamically adjusting the parameters of the intent recognition model, the problems of traditional single-turn question-answering systems being unable to associate historical interaction information and having low semantic understanding accuracy for complex queries that depend on context (such as referential queries and progressive queries) are effectively solved. At the same time, by adjusting parameters in conjunction with user preferences, the intent recognition results are further ensured to match user habits, which not only improves the coherence and accuracy of question-answering, but also reduces the operation of users repeatedly entering filtering conditions, and significantly optimizes the user interaction experience in complex scenarios.

[0057] It should be noted that the embodiments of this disclosure may include multiple steps. For ease of description, these steps are numbered, but these numbers are not a limitation on the execution time slots or execution order between the steps; these steps can be implemented in any order, and the embodiments of this disclosure do not limit this.

[0058] Corresponding to the user file management method described above, this disclosure also proposes a user file management device. Since the device embodiments of this disclosure correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to the method embodiments described above, and will not be repeated here.

[0059] Figure 2 This is a schematic diagram of the structure of a user file management device provided in an embodiment of the present disclosure, such as... Figure 2 As shown, it includes: Parsing unit 21 is used to parse the target file uploaded by the user and convert the parsing result into a unified first semantic vector; The construction unit 22 is used to store the first semantic vector and the metadata of the target file together in the dynamic knowledge base, synchronize the newly added file content in the target file in real time through the incremental update mechanism, and construct a cross-modal knowledge graph based on predefined tag classification rules and association logic; The retrieval unit 23 is used to receive natural language queries input by users, parse the query intent and generate a second semantic vector, perform joint retrieval of semantic vectors and metadata in the dynamic knowledge base, and optimize the cross-modal relevance of the retrieval results based on the cross-modal knowledge graph. The generation unit 24 is used to input the retrieval results and query intent into the pre-trained model, generate a multimodal natural language answer containing text, chart links or keyframe information, and return it to the user through a standardized interface.

[0060] This disclosure provides a user file management device that parses user-uploaded target files and converts them into a unified first semantic vector. This semantic vector, along with metadata, is stored in a dynamic knowledge base, and new content is updated in real time through an incremental update mechanism. Simultaneously, a cross-modal knowledge graph is constructed based on tag classification rules and association logic. Upon receiving a natural language query, a second semantic vector is generated for joint retrieval of the semantic vector and metadata. The cross-modal knowledge graph optimizes the relevance of results. Finally, a pre-trained model generates multimodal answers containing text, chart links, etc., and returns them through a standardized interface. Therefore, this device solves the problems in existing technologies, such as difficulty in retrieving heterogeneous files due to the lack of a unified semantic processing mechanism, the inability to incorporate new content into a static knowledge base, fragmented search results due to the lack of cross-modal knowledge association, and insufficient semantic understanding and inability to generate multimodal answers due to reliance on keyword matching. This achieves unified semantic management and dynamic updates of cloud drive files, improves the accuracy and cross-modal relevance of user query results, generates multi-dimensional, highly relevant natural language answers, and optimizes the comprehensiveness of the knowledge base and the user experience.

[0061] Furthermore, in one possible implementation of this embodiment, the parsing unit 21 is also used for: Based on the file type of the target file, the corresponding parsing strategy is matched. The target file includes files of multiple modalities. According to the parsing strategy, the target file is parsed, and the parsing result includes at least one of the following: text content, image features, and keyframe information.

[0062] Furthermore, in one possible implementation of this embodiment, matching the corresponding parsing strategy based on the file type of the target file includes: The dynamic conditional encoding module maps the file format of the target file to a predefined parsing strategy. Text files are mapped to text extraction and chart parsing strategies, video files are mapped to keyframe extraction and audio-to-text strategies, and table files are mapped to table structuring and cell semantic vectorization strategies.

[0063] Furthermore, in one possible implementation of this embodiment, the construction unit 22 is also used for: Full-text indexes are built based on a full-text search engine to support keyword retrieval, and vector indexes are built based on a vector database to support semantic similarity retrieval. A three-tiered storage architecture of text, vectors, and metadata is formed by using a metadata database to store file attributes, parse metadata, and associate document identifiers.

[0064] Furthermore, in one possible implementation of this embodiment, the retrieval unit 23 is also used for: The query intent is classified twice based on a natural language processing model, and a differentiated retrieval context is constructed based on the classification results. By analyzing the relationships between files, charts, terms, and entities through a pre-built cloud disk knowledge graph, documents containing entity associations are prioritized in the search results to achieve cross-modal chained retrieval.

[0065] Furthermore, in one possible implementation of this embodiment, the generation unit 24 is also used for: When the search results contain charts or keyframe information, a structured response output is achieved by embedding images, hyperlinks, and timestamps in rich text format. For table-based search results, a table parsing model is used to parse the semantic relationships between cells and generate a formatted answer that includes a description of the table structure and key data fields.

[0066] Furthermore, in one possible implementation of this embodiment, such as Figure 2 As shown, it also includes: Maintenance unit 25 is used to maintain multi-turn dialogue context information, store historical questions, temporary variables and user preference settings, and call context data in subsequent question-and-answer processes to adjust the parameter weights of the intent recognition model in order to optimize the semantic understanding accuracy of complex queries.

[0067] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of this embodiment, and the principle is the same, so it is not limited in this embodiment.

[0068] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0069] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0070] like Figure 3As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 302 or a computer program loaded from storage unit 308 into RAM (Random Access Memory) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O (Input / Output) interface 305 is also connected to the bus 304.

[0071] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0072] The computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as user file management methods. For example, in some embodiments, the user file management method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the aforementioned user file management method by any other suitable means (e.g., by means of firmware).

[0073] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0074] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0075] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0076] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0077] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0078] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0079] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0080] The various numerical designations such as "first," "second," etc., used in this disclosure are merely for ease of description and are not intended to limit the scope of the embodiments of this disclosure, nor do they indicate a sequential order.

[0081] At least one of the features described in this disclosure can also be described as one or more, and multiple features can be two, three, four or more, and this disclosure does not impose any limitations. In the embodiments of this disclosure, for a technical feature, the technical features in that technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no sequential order or size order among the technical features described by "first", "second", "third", "A", "B", "C" and "D".

[0082] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0083] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A user file management method, characterized in that, include: The target file uploaded by the user is parsed, and the parsing result is converted into a unified first semantic vector; The first semantic vector and the metadata of the target file are stored together in a dynamic knowledge base. The newly added file content in the target file is synchronized in real time through an incremental update mechanism, and a cross-modal knowledge graph is constructed based on predefined tag classification rules and association logic. The system receives natural language queries input by users, parses the query intent and generates a second semantic vector, performs joint retrieval of the first semantic vector and metadata in the dynamic knowledge base, and optimizes the cross-modal relevance of the retrieval results based on the cross-modal knowledge graph. The joint retrieval of the first semantic vector and metadata in the dynamic knowledge base includes: classifying the query intent twice based on a natural language processing model, and constructing a differentiated retrieval context based on the classification results; parsing the relationships between files, charts, terms and entities through a pre-built cloud disk knowledge graph, and prioritizing the matching of documents containing entity associations in the retrieval results to achieve cross-modal chain retrieval; The search results and query intent are input into a pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information, and then returned to the user through a standardized interface. This process includes: when the search results contain chart or keyframe information, embedding images, hyperlinks, and timestamps in rich text format to achieve a structured response output; for table-type search results, using a table parsing model to parse the semantic relationships between cells and generate a formatted response containing a table structure description and key data fields.

2. The user file management method according to claim 1, characterized in that, The process of parsing the user-uploaded target file and converting the parsing result into a unified first semantic vector includes: Based on the file type of the target file, a corresponding parsing strategy is matched. The target file includes files with multiple modalities. The target file is parsed according to the parsing strategy, and the parsing result includes at least one of text content, image features, and keyframe information.

3. The user file management method according to claim 2, characterized in that, The parsing strategy based on the file type of the target file includes: The dynamic conditional encoding module maps the file format of the target file to a predefined parsing strategy. Text files are mapped to text extraction and chart parsing strategies, video files are mapped to keyframe extraction and audio-to-text strategies, and table files are mapped to table structuring and cell semantic vectorization strategies.

4. The user file management method according to claim 1, characterized in that, The step of storing the first semantic vector and the metadata of the target file together in a dynamic knowledge base includes: Full-text indexes are built based on a full-text search engine to support keyword retrieval, and vector indexes are built based on a vector database to support semantic similarity retrieval. A three-tiered storage architecture of text, vectors, and metadata is formed by using a metadata database to store file attributes, parse metadata, and associate document identifiers.

5. The user file management method according to claim 1, characterized in that, Also includes: Maintain multi-turn dialogue context information, store historical questions, temporary variables and user preference settings, and call the context data in subsequent question-and-answer processes to adjust the parameter weights of the intent recognition model in order to optimize the semantic understanding accuracy of complex queries.

6. A user file management device, characterized in that, include: The parsing unit is used to parse the target file uploaded by the user and convert the parsing result into a unified first semantic vector; The construction unit is used to store the first semantic vector and the metadata of the target file together in a dynamic knowledge base, synchronize the newly added file content in the target file in real time through an incremental update mechanism, and construct a cross-modal knowledge graph based on predefined tag classification rules and association logic; The retrieval unit is used to receive natural language queries input by users, parse the query intent and generate a second semantic vector, perform joint retrieval of the first semantic vector and metadata in the dynamic knowledge base, and optimize the cross-modal relevance of the retrieval results based on the cross-modal knowledge graph. The joint retrieval of the first semantic vector and metadata in the dynamic knowledge base includes: classifying the query intent twice based on a natural language processing model, and constructing a differentiated retrieval context based on the classification results; parsing the relationships between files, charts, terms and entities through a pre-built cloud disk knowledge graph, and prioritizing the matching of documents containing entity associations in the retrieval results to achieve cross-modal chain retrieval; The generation unit is used to input the search results and the query intent into a pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information, and return it to the user through a standardized interface. The step of inputting the search results and the query intent into the pre-trained model to generate a multimodal natural language response containing text, chart links, or keyframe information includes: when the search results contain chart or keyframe information, embedding images, hyperlinks, and timestamps in rich text format to achieve a structured response output; for table-type search results, using a table parsing model to parse the semantic relationships of cells and generate a formatted response containing a table structure description and key data fields.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images