An archive data intelligent retrieval method and system based on a multi-modal large model

Abstract

The application belongs to the technical field of intelligent retrieval, and discloses a file data intelligent retrieval method and system based on a multi-modal large model. The method comprises the following steps: a cloud data center, building a file data intelligent retrieval platform, deploying a blockchain network, and constructing a file data intelligent retrieval engine; the cloud data center, performing data processing on real-time multi-modal file big data; the cloud data center, generating a knowledge graph according to a plurality of real-time retrieval themes; the cloud data center, performing data clustering; the cloud data center, using the blockchain network to perform distributed storage; a user terminal, uploading real-time query data; the cloud data center, using the file data intelligent retrieval engine to perform intelligent retrieval; and the cloud data center, visually displaying target file data. The application solves the problems of low retrieval efficiency, insufficient accuracy, weak multi-modal data processing capability and insufficient security in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of intelligent retrieval technology, specifically relating to an intelligent retrieval method and system for archival data based on a multimodal large model. Background Technology

[0002] With the rapid development of information technology, archival digitization has become an irreversible trend. Digitization not only improves the efficiency of archival storage, retrieval, and utilization, but also makes archival information easier to share and disseminate. The explosive growth of archival data places higher demands on archival data retrieval technology to ensure that users are provided with more efficient, convenient, and accurate archival information services.

[0003] Existing archival data retrieval technologies have the following shortcomings:

[0004] 1) Low retrieval efficiency: Traditional archival data retrieval mainly relies on keyword matching. Faced with massive amounts of archival data, the retrieval speed is slow and inefficient, making it difficult to quickly locate specific archival information, especially when the data volume is huge.

[0005] 2) Insufficient accuracy: Keyword retrieval is easily affected by factors such as vocabulary diversity and semantic ambiguity, resulting in inaccurate search results. It is difficult to effectively handle complex linguistic phenomena such as polysemous words, synonyms, and antonyms, causing retrieval errors.

[0006] 3) Weak multimodal data processing capabilities: Most existing technologies are designed for single-modal data (such as text or images) and lack the ability to comprehensively process multimodal data (such as text, images, audio, etc.), making it difficult to fully utilize the rich information in multimodal data and limiting the in-depth understanding and mining of archival data.

[0007] 4) Insufficient security: Existing technologies are vulnerable to security threats during data storage and transmission, such as data leakage and tampering. They lack effective data security mechanisms and cannot ensure the integrity and credibility of archival data. Summary of the Invention

[0008] To address the problems of low retrieval efficiency, insufficient accuracy, weak multimodal data processing capabilities, and inadequate security in existing technologies, the present invention aims to provide an intelligent retrieval method and system for archival data based on a multimodal large model.

[0009] The technical solution adopted in this invention is as follows:

[0010] A method for intelligent retrieval of archival data based on a multimodal large model includes the following steps:

[0011] The cloud data center will build an intelligent retrieval platform for archival data, deploy a blockchain network, use multimodal large model algorithms to construct an intelligent retrieval engine for archival data, and connect it to the intelligent retrieval platform for archival data.

[0012] The cloud data center uses an intelligent retrieval engine for archival data to process the collected real-time multimodal archival big data, resulting in several standard real-time multimodal archival data sets with real-time retrieval tags.

[0013] The cloud data center uses an intelligent retrieval engine for archival data to generate a knowledge graph based on several real-time retrieval topics, resulting in a real-time topic knowledge graph for several real-time retrieval topics.

[0014] In the cloud data center, based on several real-time topic knowledge graphs, several standard real-time multimodal archive data with real-time retrieval tags are clustered to obtain several real-time topic clustered archive data clusters.

[0015] The cloud data center uses a blockchain network to distribute and store several real-time topic clustered archive data clusters and corresponding real-time retrieval topics.

[0016] User terminals access the intelligent retrieval platform for archival data and upload the user's real-time query data to the cloud data center through the intelligent retrieval platform for archival data.

[0017] The cloud data center uses an intelligent retrieval engine for archival data based on real-time query data to perform intelligent retrieval in several real-time topic clusters of archival data and corresponding real-time retrieval topics in the blockchain network to obtain the target archival data.

[0018] The cloud data center uses an intelligent data retrieval platform to visualize target archival data on user terminals.

[0019] Furthermore, in the cloud data center, an intelligent archival data retrieval platform is built, a blockchain network is deployed, a multimodal large model algorithm is used to construct an intelligent archival data retrieval engine, and this engine is connected to the intelligent archival data retrieval platform, including the following steps:

[0020] In the cloud data center, a framework for an intelligent archival data retrieval platform is built. Within this framework, a user login module, a query data input module, and an archival data visualization module are constructed, resulting in the intelligent archival data retrieval platform.

[0021] By distributing and connecting several data servers in a cloud data center, and setting up an IPFS system and smart contracts, a blockchain network is obtained.

[0022] Based on the large dataset of multimodal training samples, a multimodal large model algorithm is used to build an intelligent retrieval engine for archival data, and the intelligent retrieval engine for archival data is connected to the intelligent retrieval platform for archival data.

[0023] Furthermore, based on the large dataset of multimodal training samples, a multimodal large model algorithm is used to construct an intelligent retrieval engine for archival data, and the intelligent retrieval engine for archival data is connected to the intelligent retrieval platform for archival data, including the following steps:

[0024] Collect large amounts of multimodal training samples and historical knowledge data on different preset retrieval topics, and preprocess them to obtain several preprocessed multimodal training samples and several preprocessed historical knowledge data.

[0025] Based on several preprocessed multimodal archive data, a multimodal data processing model is constructed using multimodal data processing algorithms, and several standard multimodal training samples are generated.

[0026] Based on several preprocessed historical knowledge data, a knowledge graph generation model is constructed using a fusion algorithm of natural language processing and deep learning.

[0027] Based on several standard multimodal training samples, a multimodal data feature extraction model is constructed using a large model algorithm, and features of several multimodal training samples are generated.

[0028] Based on the features of several multimodal training samples, a multimodal feature fusion algorithm is used to construct an archive data retrieval tag generation model and generate historical retrieval tags for each standard multimodal training sample.

[0029] Based on the historical retrieval labels of several standard multimodal training samples and randomly generated historical query data, a deep learning algorithm is used to construct an intelligent retrieval model for archival data.

[0030] By integrating multimodal data processing models, multimodal data feature extraction models, knowledge graph generation models, archival data retrieval tag generation models, and archival data intelligent retrieval models, an archival data intelligent retrieval engine is obtained, and the archival data intelligent retrieval engine is connected to the archival data intelligent retrieval platform.

[0031] Furthermore, the multimodal data processing model is constructed based on the PPO-DPA algorithm;

[0032] The knowledge graph generation model is built based on the BERT-CRF-SVM-cGAN algorithm;

[0033] The multimodal data feature extraction model is built based on the BERT-FPN-CNN-Transformer algorithm;

[0034] The archival data retrieval tag generation model is built based on the RF-MLP algorithm;

[0035] The intelligent retrieval model for archival data is built based on the LSTM-DBN-MLP algorithm.

[0036] Furthermore, the cloud data center uses an intelligent archival data retrieval engine to process the collected real-time multimodal archival big data, obtaining several real-time standard multimodal archival data sets with real-time retrieval tags, including the following steps:

[0037] Collect real-time multimodal archive big data and preprocess the real-time multimodal archive big data to obtain several preprocessed real-time multimodal archive data;

[0038] Using the multimodal data processing model of the intelligent retrieval engine for archival data, data processing is performed on each preprocessed real-time multimodal archival data to obtain the corresponding standard real-time multimodal archival data.

[0039] The multimodal data feature extraction model of the intelligent archival data retrieval engine is used to extract real-time multimodal archival data features from standard real-time multimodal archival data.

[0040] Based on the characteristics of real-time multimodal archival data, the archival data retrieval tag generation model of the intelligent archival data retrieval engine is used to generate intelligent retrieval tags and obtain the corresponding real-time retrieval tags.

[0041] By traversing all preprocessed real-time multimodal archive data, several real-time standard multimodal archive data sets with real-time retrieval tags are obtained.

[0042] Furthermore, in the cloud data center, using an intelligent retrieval engine for archival data, a knowledge graph is generated based on several real-time retrieval topics, resulting in a real-time topic knowledge graph for those topics. This process includes the following steps:

[0043] The cloud data center collects several real-time knowledge data for different real-time retrieval topics, and preprocesses the real-time knowledge data to obtain several preprocessed real-time knowledge data for different real-time retrieval topics.

[0044] Based on several preprocessed real-time knowledge data of the same real-time retrieval topic, a knowledge graph generation model of the archive data intelligent retrieval engine is used to generate a real-time topic knowledge graph.

[0045] By traversing several preprocessed real-time knowledge data of all real-time retrieval topics, a real-time topic knowledge graph of several real-time retrieval topics is obtained.

[0046] Furthermore, in the cloud data center, based on several real-time topic knowledge graphs, data clustering is performed on several standard real-time multimodal archive data sets with real-time retrieval tags to obtain several real-time topic clustered archive data clusters, including the following steps:

[0047] The cloud data center obtains the real-time retrieval tags of each standard real-time multimodal archive data and the similarity of all real-time topic knowledge graphs;

[0048] Standard real-time multimodal archive data is divided into real-time topic knowledge graphs with the highest similarity to real-time retrieval tags, resulting in several standard real-time multimodal archive data belonging to different real-time topic knowledge graphs;

[0049] Several standard real-time multimodal archives belonging to the same real-time topic knowledge graph are used as real-time topic clustering archive data clusters for the corresponding real-time topic knowledge graph.

[0050] Furthermore, the cloud data center uses a blockchain network to distribute and store several real-time topic clustering archive data clusters and corresponding real-time retrieval topics, including the following steps:

[0051] Several real-time topic clustering archive data clusters are stored in the IPFS system of the blockchain network to obtain several real-time data hash values;

[0052] Based on the real-time data hash value and real-time timestamp of each real-time topic cluster archive data cluster, the corresponding real-time transaction data is generated using the smart contract of the blockchain network;

[0053] Several data servers connected in a distributed manner using a blockchain network reach a consensus on several real-time transaction data. After the consensus is successful, several real-time consensus records and corresponding real-time storage records are generated.

[0054] The real-time consensus record, real-time storage record, and several real-time search tags and corresponding real-time search topics of each real-time topic cluster archive data cluster are synchronized to the distributed ledger of the blockchain network.

[0055] Furthermore, the cloud data center, based on real-time query data, uses an intelligent archival data retrieval engine to perform intelligent retrieval within several real-time topic-clustered archival data clusters and corresponding real-time retrieval topics on the blockchain network to obtain the target archival data, including the following steps:

[0056] The cloud data center obtains the similarity between real-time query data and real-time topic knowledge graphs of several real-time retrieval topics;

[0057] The real-time retrieval topic corresponding to the real-time topic knowledge graph with the highest similarity is used as the target real-time retrieval topic for real-time query data.

[0058] Extract several alternative real-time search tags corresponding to the target real-time search topic from the distributed ledger of the blockchain network;

[0059] The intelligent retrieval model of the archive data intelligent retrieval engine performs intelligent retrieval based on several alternative real-time retrieval tags and real-time query data to obtain the target real-time retrieval tags.

[0060] Extract the target real-time storage record of the target real-time retrieval tag from the distributed ledger of the blockchain network, and extract the target real-time data hash value from several data servers of the blockchain network based on the target real-time storage record.

[0061] Based on the target's real-time data hash value, the corresponding target file data is extracted from the IPFS system of the blockchain network.

[0062] An intelligent archival data retrieval system based on a multimodal large model is provided to implement an intelligent archival data retrieval method. The system includes a cloud data center and several user terminals, all of which are communicatively connected to the cloud data center. The cloud data center is equipped with an intelligent archival data retrieval platform and an intelligent archival data retrieval engine. The cloud data center includes an initialization unit, a data processing unit, a knowledge graph generation unit, a data clustering unit, a distributed storage unit, an intelligent retrieval unit, and a visualization unit, which are connected in sequence.

[0063] The beneficial effects of this invention are as follows:

[0064] This invention provides an intelligent retrieval method and system for archival data based on a multimodal large-scale model. Through the multimodal large-scale model algorithm and efficient data processing mechanism of the intelligent archival data retrieval engine, it achieves rapid retrieval, significantly shortens retrieval time, effectively handles massive amounts of archival data, quickly locates target materials, and improves work efficiency. Utilizing multimodal feature extraction and deep learning technology from the intelligent archival data retrieval engine, it accurately understands query intent, reduces errors caused by semantic ambiguity, effectively handles complex linguistic phenomena such as polysemous words and synonyms, and improves the accuracy of retrieval results. By comprehensively processing multiple modalities of data such as text, images, and audio through the intelligent archival data retrieval engine, it enhances multimodal data processing capabilities, fully utilizes the rich content in multimodal information, deeply explores the potential value of archival data, and improves data utilization efficiency. Deploying a blockchain network ensures the security and integrity of data during storage and transmission, and preventing data leakage and tampering through distributed ledger and hash value verification. A cloud data center provides an intelligent archival data retrieval platform, supporting real-time data updates and processing to meet the retrieval needs of real-time archival data.

[0065] Other beneficial effects of the present invention will be further explained in the specific embodiments. Attached Figure Description

[0066] Figure 1 This is a flowchart of the intelligent retrieval method for archival data based on a multimodal large model in this invention.

[0067] Figure 2 This is a structural block diagram of the intelligent archival data retrieval system based on a multimodal large model in this invention. Detailed Implementation

[0068] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0069] Example 1:

[0070] like Figure 1 As shown, this embodiment provides an intelligent retrieval method for archival data based on a multimodal large model, including the following steps:

[0071] S1: Cloud data center, build an intelligent archival data retrieval platform, deploy a blockchain network, use multimodal large model algorithms to construct an intelligent archival data retrieval engine, and connect it to the intelligent archival data retrieval platform, including the following steps:

[0072] S1-1: Cloud data center, build the framework of intelligent archival data retrieval platform, and build user login module, query data input module and archival data visualization module in the framework of intelligent archival data retrieval platform to obtain intelligent archival data retrieval platform;

[0073] The user login module is used to collect user login information and verify the user's login based on the login information. After successful login verification, the query data input module and the archive data visualization module of the intelligent archive data retrieval platform are activated. The query data input module is used to collect the user's real-time query data and upload the user's real-time query data to the cloud data center. The archive data visualization module is used to visualize the target archive data sent by the cloud data center on the user terminal.

[0074] S1-2: Distribute and connect several data servers in the cloud data center, and set up the InterPlanetary File System (IPFS) and smart contracts to obtain a blockchain network;

[0075] S1-3: Based on the large dataset of multimodal training samples, a multimodal large model algorithm is used to construct an intelligent retrieval engine for archival data, and the intelligent retrieval engine for archival data is connected to the intelligent retrieval platform for archival data, including the following steps:

[0076] S1-3-1: Collect large data of multimodal training samples and some historical knowledge data of different preset retrieval topics, and preprocess them to obtain some preprocessed multimodal training samples and some preprocessed historical knowledge data.

[0077] Preprocessing includes data cleaning, error filtering, and format standardization of raw data to improve data quality and provide data support for subsequent model building.

[0078] S1-3-2: Based on several preprocessed multimodal archive data, use multimodal data processing algorithms to construct a multimodal data processing model and generate several standard multimodal training samples;

[0079] The multimodal data processing model is built based on the Proximal Policy Optimization (MOPPO)-Data Processing Algorithm (DPA) algorithm. The multimodal data processing model includes a multimodal archive information parsing module, a data processing strategy generation module based on the PPO algorithm, and a data processing algorithm library based on several DPA algorithms, which are connected in sequence. The data processing strategy generation module includes an experience replay pool, an Actor network, a Critic network, and an agent. The agent is connected to the experience replay pool, the Actor network, and the Critic network, respectively.

[0080] The multimodal archive information parsing module is used to parse the multimodal archive information in the multimodal archive data, obtaining data information for all modalities included in the multimodal archive data, such as image modal archive data, audio modal archive data, and text modal archive data. The Actor network is responsible for outputting the probability distribution of the action to be taken in a given state. Its goal is to learn an optimal policy, i.e., maximizing the long-term cumulative reward. In the continuous action space, the Actor network typically outputs a mean and an optional variance parameter to describe the probability distribution of actions. The Critic network is responsible for evaluating the value of a given state, i.e., predicting the expected reward obtainable from that state and following the current policy. The system outputs a scalar value representing the value of a state or a state-action value. An experience replay pool stores historical experience for reuse during training. The agent generates a data processing strategy based on the probability distribution of actions (data processing algorithm calls) to be taken in a given state (the state of the multimodal data file) output by the Actor network. The data processing algorithm library stores several DPA algorithms, including grayscale conversion, resizing, and Gaussian denoising for image modal data; windowing and filtering for audio modal data; and magnitude normalization and data format conversion for text modal data. The aim is to convert the raw multimodal data into a standard format so that subsequent models can process it more effectively.

[0081] S1-3-3: Based on several preprocessed historical knowledge data, a knowledge graph generation model is constructed using a fusion algorithm of natural language processing and deep learning;

[0082] The knowledge graph generation model is built on the Bidirectional Encoder Representations from Transformers (BERT) - Conditional Random Fields (CRF) - Support Vector Machine (SVM) - Conditional Generative Adversarial Network (cGAN) algorithm. The knowledge graph generation model includes a semantic feature extraction module based on the BERT algorithm, a named entity recognition module based on the CRF algorithm, an entity relationship recognition module based on the SVM algorithm, and a knowledge graph generation module based on the cGAN algorithm, which are connected in sequence. The knowledge graph generation module includes a generator and a discriminator connected in sequence.

[0083] The BERT semantic feature extraction module can capture deep semantic information in knowledge text, which is very useful for identifying different types of named entities. The CRF named entity recognition module can consider the dependency relationship between adjacent named entity labels, which can help the model learn the sequence dependency of entity labels, thereby improving the accuracy of named entity annotation and realizing the extraction of named entities. The SVM of the entity relationship recognition module is mainly used to classify the extracted named entity pairs, determine whether there is a specific relationship between them, and the type of relationship. It converts the named entities and their context information into high-dimensional feature vectors. These vectors can effectively represent the relationship between entities. By combining the deep semantic information extracted by BERT and the sequence dependency considered by CRF, it can handle complex entity relationships more effectively. The generator's task is to generate the corresponding topic knowledge graph based on the named entities identified by the named entity recognition module and the entity relationships identified by the entity relationship recognition module. The discriminator's task is to judge whether the topic knowledge graph output by the generator is real, that is, whether it matches the real topic knowledge graph. Through this adversarial training, the generator continuously optimizes its generation ability until it can generate high-quality topic knowledge graphs.

[0084] S1-3-4: Based on several standard multimodal training samples, use the large model algorithm to construct a multimodal data feature extraction model and generate several multimodal training sample features;

[0085] The multimodal data feature extraction model is built based on the BERT-Feature Pyramid Networks (FPN)-Convolutional Neural Networks (CNN)-Transformer algorithm. The multimodal data feature extraction model includes a text feature extraction module based on the BERT algorithm, an image feature extraction module based on the FPN algorithm, an audio feature extraction module based on the CNN algorithm, and a multimodal data feature combination module based on the Transformer algorithm. The text feature extraction module, image feature extraction module, and audio feature extraction module are all connected to the multimodal data feature combination module.

[0086] The image feature extraction module utilizes the top-down and bottom-up paths of FPN to effectively extract image features at different scales, ensuring accurate detection of various details in archival images. It achieves feature fusion at different levels, enhancing feature expressiveness. By collecting external textual big data from sources such as the internet, databases, and file systems—covering various topics, domains, and language styles—the text feature extraction module is pre-trained, giving it strong generalization capabilities. Through a bidirectional attention mechanism, it captures contextual information within the text, resulting in richer and more accurate extracted text features, better adapting to different textual data and tasks. Key information in the text, such as keywords, phrases, and sentence structure, is used to form high-dimensional text feature vectors. The audio feature extraction module captures time-frequency features in audio, such as pitch, volume, and rhythm, through convolution and pooling operations. Convolution and pooling operations can effectively capture time-frequency information in audio, which is beneficial for audio data classification and retrieval. The multimodal data feature combination module captures the correlation between features of different modalities through self-attention and multi-head attention mechanisms, enhancing the model's semantic understanding ability. It unifies and associates archival data of different modalities such as text, images, and audio. The fused multimodal features have richer information, which helps to improve the comprehensiveness and accuracy of archival data retrieval.

[0087] S1-3-5: Based on the features of several multimodal training samples, a multimodal feature fusion algorithm is used to construct an archive data retrieval tag generation model and generate historical retrieval tags for each standard multimodal training sample.

[0088] The archival data retrieval label generation model is built based on the Random Forest (RF)-Multilayer Perceptron (MLP) algorithm, and the archival data retrieval label generation model includes a key feature extraction module built based on the RF algorithm and a retrieval label generation module built based on the MLP algorithm, which are connected in sequence.

[0089] The key feature extraction module improves prediction accuracy by constructing multiple decision trees and summarizing their predictions. Each decision tree is trained on a random subset of the training set, which can evaluate the importance of each feature to the prediction result, thereby identifying key features from the multimodal fusion features. It jointly encodes data such as images, text, and audio, learns cross-modal semantic representations, and reduces the dimensionality of the data by extracting key features, simplifying the complexity of subsequent models. Furthermore, the extraction of key features helps to remove noise and irrelevant information, improving the prediction accuracy of the model. The retrieval label generation module generates retrieval labels by learning the weights and biases between input features and output labels. The number of layers and neurons in the MLP can be adjusted as needed to adapt to different complexity requirements.

[0090] S1-3-6: Based on the historical retrieval tags of several standard multimodal training samples and randomly generated historical query data, a deep learning algorithm is used to construct an intelligent retrieval model for archival data;

[0091] The intelligent archival data retrieval model is built based on the Long Short-Term Memory (LSTM)-Deep Belief Network (DBN)-MLP algorithm. The intelligent archival data retrieval model includes a query data feature extraction module based on the LSTM algorithm, a retrieval label matrix feature extraction module based on the DBN algorithm, and an intelligent archival data retrieval module based on the MLP algorithm. The query data feature extraction module and the retrieval label matrix feature extraction module are both connected to the intelligent archival data retrieval module.

[0092] The query data feature extraction module utilizes a Long Short-Term Memory (LSTM) network to extract sequence features from user-input query data, capturing the time dependencies and sequence information within the query data to form a high-dimensional query feature vector. It excels at processing sequence data, effectively capturing time dependencies and ensuring the integrity of sequence information. Accurate query feature representation helps the model better understand the user's query intent, improving retrieval relevance. The retrieval label matrix feature extraction module learns deep-level features of the label matrix, enhancing feature expressiveness. It supports unsupervised pre-training and can initialize network parameters using large amounts of unlabeled data, improving the model's generalization ability. Accurate label feature representation helps the model more accurately identify and match retrieval labels, improving retrieval efficiency. The archive data intelligent retrieval module uses a multilayer perceptron to fuse and make decisions on query feature vectors and label feature vectors. It has powerful non-linear mapping capabilities, learning the complex relationship between query data and retrieval labels, improving retrieval accuracy, and outputting the most relevant retrieval labels to the query data.

[0093] S1-3-7: Integrate the multimodal data processing model, multimodal data feature extraction model, knowledge graph generation model, archival data retrieval tag generation model, and archival data intelligent retrieval model to obtain the archival data intelligent retrieval engine, and connect the archival data intelligent retrieval engine to the archival data intelligent retrieval platform;

[0094] S2: Cloud Data Center, using an intelligent archival data retrieval engine, processes the collected real-time multimodal archival big data to obtain several standard real-time multimodal archival data sets with real-time retrieval tags, including the following steps:

[0095] S2-1: Collect real-time multimodal archive big data and preprocess the real-time multimodal archive big data to obtain several preprocessed real-time multimodal archive data.

[0096] S2-2: Using the multimodal data processing model of the intelligent archival data retrieval engine, data processing is performed on each preprocessed real-time multimodal archival data to obtain the corresponding standard real-time multimodal archival data, including the following steps:

[0097] S2-2-1: The multimodal archive information parsing module of the multimodal data processing model using the intelligent archive data retrieval engine parses the preprocessed real-time multimodal archive data to obtain real-time multimodal archive information;

[0098] S2-2-2: Based on the real-time multimodal archive information, the data processing strategy generation module of the multimodal data processing model is used to generate the data processing strategy and obtain the real-time data processing strategy.

[0099] S2-2-3: Based on the real-time data processing strategy, extract several target DPA algorithms from the data processing algorithm library of the multimodal data processing model;

[0100] S2-2-4: Based on several target DPA algorithms, perform data processing on the preprocessed real-time multimodal archive data to obtain the corresponding standard real-time multimodal archive data;

[0101] S2-2-5: Traverse all preprocessed real-time multimodal archive data to obtain several standard real-time multimodal archive data;

[0102] S2-3: Using the multimodal data feature extraction model of the intelligent archival data retrieval engine, extract real-time multimodal archival data features from standard real-time multimodal archival data, including the following steps:

[0103] S2-3-1: Text feature extraction module of multimodal data feature extraction model using intelligent archival data retrieval engine, extracting real-time text features of real-time text modal archival data in standard real-time multimodal archival data;

[0104] S2-3-2: Image feature extraction module using multimodal data feature extraction model to extract real-time image features from real-time image modality archive data in standard real-time multimodal archive data;

[0105] S2-3-3: Audio feature extraction module using a multimodal data feature extraction model to extract real-time audio features from standard real-time multimodal archive data;

[0106] S2-3-4: The multimodal data feature combination module of the multimodal data feature extraction model combines real-time text features, real-time image features, and real-time audio features to obtain real-time multimodal archive data features;

[0107] S2-4: Based on the characteristics of real-time multimodal archival data, use the archival data intelligent retrieval engine's archival data retrieval tag generation model to generate intelligent retrieval tags and obtain the corresponding real-time retrieval tags, including the following steps:

[0108] S2-4-1: The key feature extraction module of the archive data retrieval tag generation model using the archive data intelligent retrieval engine extracts several real-time key features of real-time multimodal archive data features.

[0109] S2-4-2: Based on several real-time key features, the retrieval tag generation module of the archive data retrieval tag generation model is used to generate intelligent retrieval tags and obtain the corresponding real-time retrieval tags.

[0110] S2-5: Traverse all preprocessed real-time multimodal archive data to obtain several real-time standard multimodal archive data with real-time retrieval tags set;

[0111] S3: Cloud Data Center, using an intelligent retrieval engine for archival data, generates a knowledge graph based on several real-time retrieval topics, resulting in a real-time topic knowledge graph for several real-time retrieval topics, including the following steps:

[0112] S3-1: Cloud data center, collects several real-time knowledge data for different real-time retrieval topics, and preprocesses several real-time knowledge data to obtain several preprocessed real-time knowledge data for different real-time retrieval topics;

[0113] Preprocessing includes data cleaning, error filtering, and format standardization of raw data to improve data quality and provide data support for subsequent model building.

[0114] S3-2: Based on several preprocessed real-time knowledge data of the same real-time retrieval topic, a knowledge graph generation model of the intelligent archival data retrieval engine is used to generate a real-time topic knowledge graph, including the following steps:

[0115] S3-2-1: Semantic feature extraction module for knowledge graph generation model using intelligent retrieval engine for archival data, extracting real-time semantic features of preprocessed real-time knowledge data;

[0116] S3-2-2: Based on real-time semantic features, the named entity recognition module of the knowledge graph generation model is used to perform named entity recognition and obtain several knowledge named entities.

[0117] S3-2-3: Real-time semantic features of several knowledge named entities. The entity relationship recognition module of the knowledge graph generation model is used to identify entity relationships and obtain several knowledge entity relationships.

[0118] S3-2-4: Traverse several preprocessed real-time knowledge data of the same real-time retrieval topic to obtain several knowledge named entities and several knowledge entity relationships of several preprocessed real-time knowledge data;

[0119] S3-2-5: The generator of the knowledge graph generation module using the knowledge graph generation model generates a knowledge graph based on several named entities and several knowledge entity relationships of several preprocessed real-time knowledge data, and obtains the real-time topic knowledge graph of the corresponding real-time retrieval topic.

[0120] S3-3: Traverse several preprocessed real-time knowledge data of all real-time retrieval topics to obtain real-time topic knowledge graphs of several real-time retrieval topics;

[0121] S4: Cloud Data Center, based on several real-time topic knowledge graphs, performs data clustering on several standard real-time multimodal archive data sets with real-time retrieval tags to obtain several real-time topic clustered archive data clusters, including the following steps:

[0122] S4-1: Cloud data center, obtains the real-time retrieval tags of each standard real-time multimodal archive data and the similarity of all real-time topic knowledge graphs;

[0123] S4-2: Divide the standard real-time multimodal archive data into the real-time topic knowledge graph with the highest similarity to the real-time retrieval tags to obtain several standard real-time multimodal archive data belonging to different real-time topic knowledge graphs;

[0124] S4-3: Several standard real-time multimodal archive data belonging to the same real-time topic knowledge graph are used as real-time topic clustering archive data clusters of the corresponding real-time topic knowledge graph.

[0125] S5: Cloud data center, using a blockchain network, distributes and stores several real-time topic-clustered archive data clusters and corresponding real-time retrieval topics, including the following steps:

[0126] S5-1: Store several real-time topic clustering archive data clusters into the IPFS system of the blockchain network to obtain several real-time data hash values;

[0127] S5-2: Based on the real-time data hash value and real-time timestamp of each real-time topic cluster archive data cluster, use the smart contract of the blockchain network to generate the corresponding real-time transaction data;

[0128] S5-3: Several data servers connected in a distributed manner using a blockchain network reach a consensus on several real-time transaction data. After the consensus is successful, several real-time consensus records and corresponding real-time storage records are generated.

[0129] S5-4: Synchronize the real-time consensus record, real-time storage record, and several real-time search tags and corresponding real-time search topics of each real-time topic clustering archive data cluster to the distributed ledger of the blockchain network.

[0130] S6: User terminal, accesses the intelligent retrieval platform for archival data, and uploads the user's real-time query data to the cloud data center through the intelligent retrieval platform for archival data;

[0131] S7: Cloud Data Center, based on real-time query data, uses an intelligent archival data retrieval engine to intelligently search several real-time topic-clustered archival data clusters and corresponding real-time retrieval topics in the blockchain network to obtain the target archival data, including the following steps:

[0132] S7-1: Cloud data center, obtain the similarity between real-time query data and real-time topic knowledge graphs of several real-time retrieval topics;

[0133] S7-2: Use the real-time retrieval topic corresponding to the real-time topic knowledge graph with the highest similarity as the target real-time retrieval topic for real-time query data;

[0134] S7-3: Extract several alternative real-time search tags corresponding to the target real-time search topic from the distributed ledger of the blockchain network;

[0135] S7-4: Using the intelligent archival data retrieval model of the intelligent archival data retrieval engine, intelligent retrieval is performed based on several alternative real-time retrieval tags and real-time query data to obtain the target real-time retrieval tags, including the following steps:

[0136] S7-4-1: Convert several candidate real-time search tags into a real-time candidate search tag matrix, and input the real-time candidate search tag matrix into the archive data intelligent search model of the archive data intelligent search engine;

[0137] S7-4-2: The feature extraction module of the search tag matrix using the intelligent retrieval model of archival data extracts the real-time search tag matrix features of the real-time candidate search tag matrix;

[0138] S7-4-3: The query data feature extraction module using the intelligent retrieval model for archival data extracts real-time query data features.

[0139] S7-4-4: Based on the characteristics of the real-time retrieval tag matrix and the characteristics of the real-time query data, the intelligent retrieval module of the intelligent retrieval model of archive data is used to perform intelligent retrieval and obtain the target real-time retrieval tags;

[0140] S7-5: Extract the target real-time storage record of the target real-time retrieval tag from the distributed ledger of the blockchain network, and extract the target real-time data hash value from several data servers of the blockchain network based on the target real-time storage record.

[0141] S7-6: Extract the corresponding target file data from the IPFS system of the blockchain network based on the target's real-time data hash value;

[0142] S8: Cloud Data Center, which uses an intelligent data retrieval platform to visualize target archive data on user terminals.

[0143] Example 2:

[0144] like Figure 2 As shown, this embodiment provides an intelligent archival data retrieval system based on a multimodal large model, used to implement an intelligent archival data retrieval method. The system includes a cloud data center and several user terminals. The user terminals are all connected to the cloud data center. The cloud data center is equipped with an intelligent archival data retrieval platform and an intelligent archival data retrieval engine. The cloud data center includes an initialization unit, a data processing unit, a knowledge graph generation unit, a data clustering unit, a distributed storage unit, an intelligent retrieval unit, and a visualization display unit connected in sequence.

[0145] The intelligent archival data retrieval platform includes a user login module, a query data input module, and an archival data visualization module.

[0146] The intelligent archival data retrieval engine is equipped with a multimodal data processing model, a multimodal data feature extraction model, a knowledge graph generation model, an archival data retrieval tag generation model, and an intelligent archival data retrieval model.

[0147] The user terminal is used to access the intelligent retrieval platform for archival data and upload the user's real-time query data to the cloud data center through the intelligent retrieval platform for archival data.

[0148] The initialization unit is used to build an intelligent archival data retrieval platform, deploy a blockchain network, use a multimodal large model algorithm to construct an intelligent archival data retrieval engine, and connect to the intelligent archival data retrieval platform.

[0149] The data processing unit is used to process the collected real-time multimodal archival big data using the archival data intelligent retrieval engine, and obtain several standard real-time multimodal archival data with real-time retrieval tags.

[0150] The knowledge graph generation unit is used to generate a real-time topic knowledge graph based on several real-time search topics using the intelligent retrieval engine for archival data.

[0151] The data clustering unit is used to cluster several standard real-time multimodal archive data with real-time retrieval tags based on several real-time topic knowledge graphs, so as to obtain several real-time topic clustered archive data clusters.

[0152] Distributed storage units are used to distribute and store several real-time topic clustered archive data clusters and corresponding real-time retrieval topics using a blockchain network.

[0153] The intelligent retrieval unit is used to perform intelligent retrieval of target archival data in several real-time topic clusters of archival data and corresponding real-time retrieval topics in the blockchain network based on real-time query data and using the intelligent retrieval engine of archival data.

[0154] The visualization unit is used to visualize target archival data on the user terminal through the intelligent archival data retrieval platform.

[0155] This invention provides an intelligent retrieval method and system for archival data based on a multimodal large-scale model. Through the multimodal large-scale model algorithm and efficient data processing mechanism of the intelligent archival data retrieval engine, it achieves rapid retrieval, significantly shortens retrieval time, effectively handles massive amounts of archival data, quickly locates target materials, and improves work efficiency. Utilizing multimodal feature extraction and deep learning technology from the intelligent archival data retrieval engine, it accurately understands query intent, reduces errors caused by semantic ambiguity, effectively handles complex linguistic phenomena such as polysemous words and synonyms, and improves the accuracy of retrieval results. By comprehensively processing multiple modalities of data such as text, images, and audio through the intelligent archival data retrieval engine, it enhances multimodal data processing capabilities, fully utilizes the rich content in multimodal information, deeply explores the potential value of archival data, and improves data utilization efficiency. Deploying a blockchain network ensures the security and integrity of data during storage and transmission, and preventing data leakage and tampering through distributed ledger and hash value verification. A cloud data center provides an intelligent archival data retrieval platform, supporting real-time data updates and processing to meet the retrieval needs of real-time archival data.

[0156] This invention is not limited to the optional embodiments described above, and anyone can derive other various forms of products based on the inspiration of this invention. The specific embodiments described above should not be construed as limiting the scope of protection of this invention; the scope of protection of this invention should be determined by the claims, and the specification can be used to interpret the claims.

Claims

1. A method for intelligent retrieval of archival data based on a multimodal large model, characterized in that: Includes the following steps: The cloud data center will build an intelligent retrieval platform for archival data, deploy a blockchain network, use multimodal large model algorithms to construct an intelligent retrieval engine for archival data, and connect it to the intelligent retrieval platform for archival data. The aforementioned intelligent archival data retrieval engine includes a multimodal data processing model, a multimodal data feature extraction model, a knowledge graph generation model, an archival data retrieval tag generation model, and an intelligent archival data retrieval model. The multimodal data processing model described above is constructed based on the PPO-DPA algorithm; The knowledge graph generation model is constructed based on the BERT-CRF-SVM-cGAN algorithm; The multimodal data feature extraction model described above is constructed based on the BERT-FPN-CNN-Transformer algorithm; The aforementioned archival data retrieval tag generation model is constructed based on the RF-MLP algorithm; The aforementioned intelligent archival data retrieval model is built based on the LSTM-DBN-MLP algorithm; The cloud data center uses an intelligent archival data retrieval engine to process the collected real-time multimodal archival big data, obtaining several standard real-time multimodal archival data sets with real-time retrieval tags. This process includes the following steps: Collect real-time multimodal archive big data and preprocess the real-time multimodal archive big data to obtain several preprocessed real-time multimodal archive data; Using the multimodal data processing model of the intelligent retrieval engine for archival data, data processing is performed on each preprocessed real-time multimodal archival data to obtain the corresponding standard real-time multimodal archival data. The multimodal data feature extraction model of the intelligent archival data retrieval engine is used to extract real-time multimodal archival data features from standard real-time multimodal archival data. Based on the characteristics of real-time multimodal archival data, the archival data retrieval tag generation model of the intelligent archival data retrieval engine is used to generate intelligent retrieval tags and obtain the corresponding real-time retrieval tags. Traverse all preprocessed real-time multimodal archive data to obtain several real-time standard multimodal archive data with real-time retrieval tags set; The cloud data center uses an intelligent retrieval engine for archival data to generate a knowledge graph based on several real-time retrieval topics, resulting in a real-time topic knowledge graph for several real-time retrieval topics. In the cloud data center, based on several real-time topic knowledge graphs, several standard real-time multimodal archive data with real-time retrieval tags are clustered to obtain several real-time topic clustered archive data clusters. The cloud data center uses a blockchain network to distribute and store several real-time topic clustered archive data clusters and corresponding real-time retrieval topics. User terminals access the intelligent retrieval platform for archival data and upload the user's real-time query data to the cloud data center through the intelligent retrieval platform for archival data. The cloud data center, based on real-time query data, uses an intelligent archival data retrieval engine to intelligently search several real-time topic-clustered archival data clusters and corresponding real-time retrieval topics within the blockchain network to obtain the target archival data. This process includes the following steps: The cloud data center obtains the similarity between real-time query data and real-time topic knowledge graphs of several real-time retrieval topics; The real-time retrieval topic corresponding to the real-time topic knowledge graph with the highest similarity is used as the target real-time retrieval topic for real-time query data. Extract several alternative real-time search tags corresponding to the target real-time search topic from the distributed ledger of the blockchain network; The intelligent retrieval model of the archive data intelligent retrieval engine performs intelligent retrieval based on several alternative real-time retrieval tags and real-time query data to obtain the target real-time retrieval tags. Extract the target real-time storage record of the target real-time retrieval tag from the distributed ledger of the blockchain network, and extract the target real-time data hash value from several data servers of the blockchain network based on the target real-time storage record. Based on the target's real-time data hash value, the corresponding target archive data is extracted from the IPFS system of the blockchain network; The cloud data center uses an intelligent data retrieval platform to visualize target archival data on user terminals.

2. The intelligent retrieval method for archival data based on a multimodal large model according to claim 1, characterized in that: The cloud data center establishes an intelligent archival data retrieval platform, deploys a blockchain network, uses multimodal large-scale model algorithms to build an intelligent archival data retrieval engine, and connects it to the intelligent archival data retrieval platform, including the following steps: In the cloud data center, a framework for an intelligent archival data retrieval platform is built. Within this framework, a user login module, a query data input module, and an archival data visualization module are constructed, resulting in the intelligent archival data retrieval platform. By distributing and connecting several data servers in a cloud data center, and setting up an IPFS system and smart contracts, a blockchain network is obtained. Based on the large dataset of multimodal training samples, a multimodal large model algorithm is used to build an intelligent retrieval engine for archival data, and the intelligent retrieval engine for archival data is connected to the intelligent retrieval platform for archival data.

3. The intelligent retrieval method for archival data based on a multimodal large model according to claim 2, characterized in that: Based on large datasets of multimodal training samples, a multimodal large-scale model algorithm is used to construct an intelligent retrieval engine for archival data. This engine is then connected to an intelligent archival data retrieval platform, including the following steps: Collect large amounts of multimodal training samples and historical knowledge data on different preset retrieval topics, and preprocess them to obtain several preprocessed multimodal training samples and several preprocessed historical knowledge data. Based on several preprocessed multimodal archive data, a multimodal data processing model is constructed using multimodal data processing algorithms, and several standard multimodal training samples are generated. Based on several preprocessed historical knowledge data, a knowledge graph generation model is constructed using a fusion algorithm of natural language processing and deep learning. Based on several standard multimodal training samples, a multimodal data feature extraction model is constructed using a large model algorithm, and features of several multimodal training samples are generated. Based on the features of several multimodal training samples, a multimodal feature fusion algorithm is used to construct an archive data retrieval tag generation model and generate historical retrieval tags for each standard multimodal training sample. Based on the historical retrieval labels of several standard multimodal training samples and randomly generated historical query data, a deep learning algorithm is used to construct an intelligent retrieval model for archival data. By integrating multimodal data processing models, multimodal data feature extraction models, knowledge graph generation models, archival data retrieval tag generation models, and archival data intelligent retrieval models, an archival data intelligent retrieval engine is obtained, and the archival data intelligent retrieval engine is connected to the archival data intelligent retrieval platform.

4. The intelligent retrieval method for archival data based on a multimodal large model according to claim 3, characterized in that: In the cloud data center, an intelligent retrieval engine for archival data is used to generate a knowledge graph based on several real-time retrieval topics. This results in a real-time topic knowledge graph for several real-time retrieval topics, including the following steps: The cloud data center collects several real-time knowledge data for different real-time retrieval topics, and preprocesses the real-time knowledge data to obtain several preprocessed real-time knowledge data for different real-time retrieval topics. Based on several preprocessed real-time knowledge data of the same real-time retrieval topic, a knowledge graph generation model of the archive data intelligent retrieval engine is used to generate a real-time topic knowledge graph. By traversing several preprocessed real-time knowledge data of all real-time retrieval topics, a real-time topic knowledge graph of several real-time retrieval topics is obtained.

5. The intelligent retrieval method for archival data based on a multimodal large model according to claim 4, characterized in that: In a cloud data center, based on several real-time topic knowledge graphs, data clustering is performed on several standard real-time multimodal archive data sets with real-time retrieval tags to obtain several real-time topic clustered archive data clusters, including the following steps: The cloud data center obtains the real-time retrieval tags of each standard real-time multimodal archive data and the similarity of all real-time topic knowledge graphs; Standard real-time multimodal archive data is divided into real-time topic knowledge graphs with the highest similarity to real-time retrieval tags, resulting in several standard real-time multimodal archive data belonging to different real-time topic knowledge graphs; Several standard real-time multimodal archives belonging to the same real-time topic knowledge graph are used as real-time topic clustering archive data clusters for the corresponding real-time topic knowledge graph.

6. The intelligent retrieval method for archival data based on a multimodal large model according to claim 5, characterized in that: Cloud data centers, using blockchain networks, distribute the storage of several real-time topic-based archival data clusters and corresponding real-time retrieval topics, including the following steps: Several real-time topic clustering archive data clusters are stored in the IPFS system of the blockchain network to obtain several real-time data hash values; Based on the real-time data hash value and real-time timestamp of each real-time topic cluster archive data cluster, the corresponding real-time transaction data is generated using the smart contract of the blockchain network; Several data servers connected in a distributed manner using a blockchain network reach a consensus on several real-time transaction data. After the consensus is successful, several real-time consensus records and corresponding real-time storage records are generated. The real-time consensus record, real-time storage record, and several real-time search tags and corresponding real-time search topics of each real-time topic cluster archive data cluster are synchronized to the distributed ledger of the blockchain network.

7. An intelligent archival data retrieval system based on a multimodal large model, used to implement the intelligent archival data retrieval method as described in any one of claims 1-6, characterized in that: The system includes a cloud data center and several user terminals. All user terminals are communicatively connected to the cloud data center. The cloud data center is equipped with an intelligent archive data retrieval platform and an intelligent archive data retrieval engine. The cloud data center includes an initialization unit, a data processing unit, a knowledge graph generation unit, a data clustering unit, a distributed storage unit, an intelligent retrieval unit, and a visualization display unit, which are connected in sequence.

Images