A Digital Archives Classification and Query Method Based on Large Model Index Recognition

Abstract

This invention relates to the field of information technology and discloses a digital archive classification and query method based on large-model index recognition. The method includes: generating a unified feature representation through the collection and processing of multimodal data; optimizing the information expression of multimodal data based on feature fusion methods; improving the consistency of data features and the accuracy of classification results using a dynamic verification mechanism; embedding the optimized features into a low-dimensional semantic space through dimensionality reduction methods to generate classification labels; and constructing a vector index that supports query and feedback optimization. The collection of multimodal data includes the following steps: collecting text data and image data. Feature extraction from image data is performed using a convolutional neural network. Through the collection and processing of multimodal data, a unified feature representation of text, images, audio, and structured numerical data is achieved, effectively solving the problems of single data sources and inconsistent feature expressions in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of information technology, specifically to a method for classifying and querying digital archives based on large model index recognition. Background Technology

[0002] With the advent of the information age, a large number of paper documents have been digitized, forming a vast digital archive. These archives encompass various types and formats of data, including text, images, audio, and structured numerical data. Faced with this vast and diverse archive, how to efficiently and accurately manage and retrieve these documents has become an important research topic in the field of archival management.

[0003] With the increasing emphasis on domestic production, various industries have an urgent need for digital management tools with independent intellectual property rights. In the field of digital records management, domestic technologies are also constantly being explored and developed.

[0004] However, most traditional digital archive retrieval methods currently rely on keyword matching or pre-defined category tags. While these methods were useful in the early days, they also have significant limitations: keyword matching cannot understand semantic relationships, resulting in low accuracy of search results; category tags rely on manual setting, lacking flexibility and failing to adapt to dynamically changing user needs and the growth of archive data.

[0005] Existing traditional query techniques lack a deep understanding of user intent, especially in scenarios involving non-textual modal data, resulting in poor query performance. Furthermore, these methods typically employ static indexes and rule-based matching, making it difficult to dynamically adjust index content and optimize query strategies. This leads to slow query speeds and poor system scalability in large-scale archive scenarios. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a digital archive classification and query method based on large model index recognition, which solves the problems of insufficient accuracy and speed in digital archive query methods.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a digital archive classification and query method based on large model index recognition, comprising the following steps:

[0008] By collecting and processing multimodal data, a unified feature representation is generated;

[0009] Optimize the information representation of multimodal data based on feature fusion methods;

[0010] Utilize dynamic validation mechanisms to improve the consistency of data features and the accuracy of classification results;

[0011] The optimized features are embedded into a low-dimensional semantic space using dimensionality reduction methods to generate classification labels.

[0012] Build vector indexes that support query and feedback optimization.

[0013] Preferably, the acquisition of the multimodal data includes the following steps:

[0014] Text data is collected, and word segmentation and embedding extraction are performed on the text data through a pre-trained language model to generate semantic feature vectors; image data is collected, and feature extraction is performed on the image data through a convolutional neural network to generate high-dimensional visual feature vectors.

[0015] Collect audio data, transcribe the audio data into text data using a speech-to-text model, and then extract semantic feature vectors.

[0016] Collect structured numerical data and adjust the data range using normalization methods to generate standardized numerical features.

[0017] Preferably, the processing of the multimodal data includes the following steps:

[0018] Extracting semantic embeddings from text data using language models;

[0019] Using convolutional neural networks to extract and embed features from image data;

[0020] Semantic features are extracted from transcribed audio data using a speech recognition model.

[0021] Normalize the structured numerical data.

[0022] Preferably, the feature fusion method optimization includes the following steps:

[0023] The high-dimensional features of multimodal data are input into a variational autoencoder model, and the encoder compresses the multimodal features to generate latent feature representations.

[0024] In the latent feature space, the optimization objective is defined as:

[0025]

[0026] Where X represents the input multimodal features, Z represents the latent feature representation, Y represents the classification target, I(X;Z) represents the mutual information between the multimodal features and the latent features, I(Z;Y) represents the mutual information between the latent features and the classification target, and β is a hyperparameter that balances compression and classification accuracy;

[0027] Optimize the latent feature representation to maximize the relevance of latent features to the classification task, while eliminating redundant features and reducing redundancy and noise interference among multimodal features.

[0028] A compressed and optimized fusion feature representation is generated for subsequent classification processing.

[0029] Preferably, the step of utilizing the dynamic verification mechanism includes the following:

[0030] Modal consistency verification:

[0031] By calculating the modal features Z i and Z j To optimize the consistency between modalities by considering the prediction discrepancies in the target classification Y, the loss function is defined as follows:

[0032]

[0033] Among them, Z i and Z j Let f(Z) represent the feature representations of modes i and j, and let f(Z) represent the classification function. ||·|| 2 Indicates Euclidean distance;

[0034] Modal complementarity verification:

[0035] Evaluate each modal feature Z i The complementary loss function is defined to determine the independent contribution of each component to the target classification Y:

[0036]

[0037] Among them, I(Z) i ;Y) represents modal feature Z i Mutual information with the target classification Y, where λ is a weighting factor;

[0038] Dynamic weight adjustment:

[0039] Based on the modal characteristics Z i The updated modal weights are related to the target classification Y, and the update formula is:

[0040]

[0041] in, This represents the weight of mode i in the t-th iteration. The weights for the next iteration;

[0042] Comprehensive optimization objective function:

[0043] By combining consistency verification, complementarity verification, and dynamic weight adjustment, the final optimization objective is:

[0044]

[0045] in, Let represent the comprehensive optimization objective function of the dynamic verification mechanism. The loss function for modality consistency verification is used to minimize the differences in classification target prediction results among different modality features. The loss function represents the modal complementarity verification, used to evaluate the independent contribution of each modal feature.

[0046] Preferably, the dimensionality reduction method includes the following steps:

[0047] Construct neighborhood relationships for high-dimensional feature data points, and determine the neighborhood of each data point by calculating the similarity between data points:

[0048] While preserving neighborhood relationships, high-dimensional features are mapped to a low-dimensional space to generate low-dimensional embedded features;

[0049] Optimize the dimensionality reduction process to ensure that low-dimensional embedded features can preserve the global structure and semantic consistency of high-dimensional data.

[0050] Preferably, the generation of the classification labels includes the following steps:

[0051] Low-dimensional embedding features are input into a large language model, and initial classification labels are generated by combining low-dimensional feature representations with contextual semantic information; the initial classification labels are then refined based on the time dimension, topic dimension, and priority dimension of the archival data.

[0052] The content of the category tags is dynamically adjusted, and the final category tags are generated by combining user query needs and background information of the archives.

[0053] Preferably, the construction supporting query and feedback optimization includes the following steps:

[0054] Collect user feedback on the classification query results and use the feedback as new sample input to the model;

[0055] The dynamic classification label generation module is optimized by using feedback samples, and the classification labels are adjusted to more accurately reflect user needs; the feature fusion module is updated using feedback samples to improve the accuracy of multimodal data feature representation and classification effect.

[0056] The low-dimensional vector index is dynamically adjusted so that new samples and optimized features can be updated to the query system in real time, ensuring the dynamism and accuracy of query results.

[0057] Preferably, the construction of the vector index includes the following steps:

[0058] Low-dimensional embedded features are stored in a vector database and indexed based on vector similarity;

[0059] The index is optimized using an approximate nearest neighbor search algorithm to ensure low latency and high efficiency in the query process;

[0060] To address the dynamic updates of classification labels, the feature data in the vector index is adjusted in real time to support multidimensional queries and matching of classification labels.

[0061] This invention also provides a digital archive classification and query system based on large model index recognition, including:

[0062] The data acquisition module is used to collect multimodal data such as text, images, audio, and structured numerical data;

[0063] The data preprocessing module is used to standardize the collected multimodal data and generate a unified feature representation;

[0064] The feature fusion module is used to optimize the information representation of multimodal data and remove redundant information;

[0065] The validation and optimization module is used to validate the consistency of data features and optimize classification results.

[0066] The dimensionality reduction classification module is used to embed feature data into a low-dimensional semantic space and generate dynamic classification labels;

[0067] The index query module is used to build a vector index library that supports fast querying;

[0068] The user feedback module is used to collect user feedback information and optimize the classification query model.

[0069] This invention provides a digital archive classification and query method based on large model index recognition. It has the following beneficial effects:

[0070] 1. This invention achieves unified feature representation of text, images, audio and structured numerical data through the acquisition and processing of multimodal data, effectively solving the problems of single data source and inconsistent feature expression in the prior art.

[0071] 2. This invention uses a variational autoencoder to compress and fuse multimodal features, eliminating redundant information and retaining the core features relevant to the classification task, thereby improving feature representation capabilities and solving the problem of low classification performance caused by information redundancy in the prior art.

[0072] 3. This invention ensures the collaborative performance of multimodal features in classification tasks by verifying modal consistency and complementarity, and dynamically adjusting modal weights. This solves the problem of insufficient feature synergy in existing technologies and significantly improves the accuracy and robustness of classification results.

[0073] 4. This invention maps high-dimensional features to a low-dimensional semantic space using manifold learning technology, reducing computational complexity while preserving global semantic consistency, thus solving the problem of excessively high computational cost of existing high-dimensional features.

[0074] 5. By introducing an approximate nearest neighbor search algorithm and a user feedback optimization mechanism, this invention supports dynamic updates of the classification model and index content, enabling rapid response for classification queries and solving the problem that existing static indexes cannot adapt to dynamic scenarios.

[0075] 6. This invention dynamically incorporates user feedback into the optimization of the index library and classification model, forming a closed-loop optimization mechanism. This improves the system's adaptability to user needs and solves the problem of insufficient response of existing classification systems in scenarios with dynamically changing demands.

[0076] 7. This invention provides efficient and accurate digital archive classification and query services through end-to-end optimization, solving the shortcomings of existing rule-based classification methods that are difficult to adapt to complex multimodal data, and providing strong technical support for digital archive management and intelligent applications. Attached Figure Description

[0077] Figure 1 This is a flowchart of the method of the present invention;

[0078] Figure 2 This is a system architecture diagram of the present invention. Detailed Implementation

[0079] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] Please see the appendix Figure 1 This invention provides a digital archive classification and query method based on large model index recognition, comprising the following steps:

[0081] S1. Generate a unified feature representation through the collection and processing of multimodal data;

[0082] S2. Optimize the information representation of multimodal data based on feature fusion methods;

[0083] S3. Improve the consistency of data features and the accuracy of classification results by using dynamic verification mechanisms;

[0084] S4. Embed the optimized features into a low-dimensional semantic space using dimensionality reduction methods to generate classification labels;

[0085] S5. Construct a vector index that supports query and feedback optimization.

[0086] For step S1, this embodiment provides a method for acquiring and processing multimodal data to generate a unified feature representation. This step is fundamental to the method of this invention and aims to address the inconsistencies in the structure, scale, and semantic representation of multimodal data. Through effective acquisition and standardized processing of multimodal data, heterogeneous data is uniformly converted into a high-dimensional feature representation, laying the foundation for subsequent feature fusion, dynamic verification, and index querying.

[0087] Alternatively, data acquisition can include four modalities: text, image, audio, and structured numerical data. It should be noted that this data can be obtained through sensor devices, database interfaces, or file reading, and then processed through a unified workflow to generate feature vectors, forming standardized input.

[0088] The acquisition and processing of multimodal data includes the following:

[0089] The acquisition and processing of text data can originate from documents, emails, web page content, or text fields in a database. One implementation utilizes a pre-trained language model to process the text data.

[0090] Specifically:

[0091] First, the text data is segmented to remove stop words and meaningless characters.

[0092] Subsequently, a high-dimensional semantic embedding is generated using a pre-trained language model, and the embedding representation preserves the semantic features of the text.

[0093] For example, assuming the input text is "archives contain 2023 classification information", the word segmentation results are "archives", "contains", "2023", "classification", and "information". The embedding representation generated by the pre-trained language model is a 512-dimensional feature vector, with the specific embedding form as follows:

[0094] E t =Encoder(T)

[0095] Among them, E t This represents text feature embedding, where T represents text data, and Encoder is a pre-trained language model.

[0096] Image data acquisition and processing includes the following:

[0097] Image data originates from keyframes of document scans, photographs, or video frames. Specifically, deep convolutional neural networks are used to extract and understand features from image data. Text feature embeddings are high-dimensional, dense vectors that can capture semantic information of the text and are compatible with other modal features.

[0098] Alternatively, the image data is first preprocessed, including operations such as resizing, grayscale conversion, or noise reduction. Then, a convolutional neural network is used to generate high-dimensional visual feature vectors.

[0099] The formula is expressed as:

[0100] E i =CNN(I)

[0101] Among them, E i denoted as image feature embedding, I represents image data, and CNN stands for Convolutional Neural Network.

[0102] It should be noted that in some embodiments, the dimension of image feature extraction can be adjusted according to the specific application scenario. For example, the output layer of ResNet-50 can be truncated to generate a high-dimensional feature vector of 2048 dimensions.

[0103] Audio data acquisition and processing include the following:

[0104] Audio data can originate from voice recordings or multimedia files. In one implementation, the audio data is first transcribed into text using a speech-to-text model, and then semantic embeddings are generated using a text processing workflow.

[0105] Specifically:

[0106] Audio data is input into a speech-to-text model to generate transcribed text.

[0107] The transcribed text is processed by a pre-trained language model to generate high-dimensional semantic features.

[0108] The formula is expressed as:

[0109] E a =Encoder(STT(A))

[0110] Among them, E a The high-dimensional feature embedding of audio is represented by A, where A represents audio data and STT represents a speech-to-text model.

[0111] It should be noted that in some embodiments, audio data can also be directly used to generate feature vectors through an audio feature extraction model, which can be used when semantic layer is not required.

[0112] The acquisition and processing of structured numerical data includes the following:

[0113] Structured numerical data typically originates from sensors, tabular data, or database fields. In this embodiment, the structured data is first normalized to ensure that its values ​​fall within the range of 0, 1, or other standard ranges.

[0114] Specifically:

[0115] Standardize the data using the Min-Max normalization method:

[0116]

[0117] Among them, E n The normalized feature representation is represented by N, which represents the original structured numerical data. min(N) and max(N) are the minimum and maximum values ​​of the data, respectively.

[0118] The understanding is that normalized data can avoid the impact of numerical scale differences on subsequent processing and is compatible with other modal features.

[0119] A unified representation of multimodal features includes the following:

[0120] After all modal data are processed independently, their feature embeddings are spliced ​​or fused to form a unified high-dimensional feature representation.

[0121] Specifically:

[0122] Let the text feature be E t Image features are E i The audio characteristic is E a Numerical characteristics are E n The unified feature is represented as:

[0123] X = [E] t E i E a E n ]

[0124] It is understood that this high-dimensional feature representation contains key information from multimodal data, which can support subsequent feature fusion and classification tasks.

[0125] Extended content and technology adaptation include the following:

[0126] It should be noted that in some embodiments, other modal data (such as video data, sensor data, etc.) can also be used as input to generate feature representations through a similar preprocessing process.

[0127] Specifically: For video data, image features can be extracted from keyframes, or time series features can be modeled through time series modeling.

[0128] For sensor data, signal features can be extracted directly, or frequency domain features can be extracted through Fourier transform.

[0129] In summary, this step details the methods for acquiring and processing multimodal data, including features from text, images, audio, and structured numerical data. The extraction of a unified high-dimensional feature representation ensures the technical foundation and data compatibility for subsequent steps.

[0130] For step S2, a technical solution based on feature fusion to optimize the information representation of multimodal data is proposed. This step, based on the acquisition and processing of multimodal data, compresses and fuses the unified high-dimensional feature representations of text, images, audio, and structured numerical data, eliminating redundant information and improving the effectiveness and expressive power of data features. Through feature fusion, the most useful feature representations for classification tasks are further extracted, providing an accurate and efficient data foundation for subsequent dynamic verification mechanisms and classification tasks.

[0131] It should be noted that this step uses a variational autoencoder to compress and optimize features, preserving useful information between multimodal data features to the maximum extent, while removing irrelevant information and noise, thus achieving efficient fusion and representation of multimodal data.

[0132] Feature fusion methods optimize the information representation of multimodal data, including the following:

[0133] In one implementation, the high-dimensional features X of the multimodal data obtained through step S1 are input into a variational autoencoder to achieve feature compression and fusion.

[0134] Specifically, a variational autoencoder consists of two parts: an encoder and a decoder.

[0135] The encoder is used to map high-dimensional input features to a latent feature space X, generating a latent feature representation Z.

[0136] The decoder is used to reconstruct the latent features Z back to the original features. Used to verify the quality of feature compression.

[0137] The formula is expressed as follows:

[0138]

[0139] Where X represents the input multimodal features, and Z represents the latent feature representation. This represents the reconstructed feature representation, where Enc and Dec represent the encoder and decoder of the variational autoencoder, respectively.

[0140] It should be noted that the latent feature representation Z extracts the most valuable information from multimodal data through an adaptive compression mechanism, while removing redundant and noisy information.

[0141] In the latent feature space, the optimization objective is to maximize the correlation between the latent feature Z and the classification target Y, while minimizing the redundant information between Z and the input data X, thereby achieving efficient feature fusion.

[0142] In one possible implementation, the optimization objective is defined as follows:

[0143]

[0144] in, Let I(X; Z) represent the objective function for feature fusion, where I(X; Z) represents the mutual information between input feature X and latent feature Z, and I(Z; Y) represents the mutual information between latent feature Z and classification target Y, which is used to retain information in latent features that is useful for the classification task. β is a balance coefficient used to control the trade-off between compression and classification accuracy.

[0145] Specifically, by minimizing I(X;Z) and maximizing I(Z;Y), effective information compression and optimization of classification objectives can be achieved in the latent feature space.

[0146] In some embodiments, to further improve the effect of feature fusion, the latent feature representation Z can be optimized through regularization constraints to ensure that the distribution of features has better representational power and generalization ability.

[0147] For example, the regularization constraint for latent features can be expressed by the following formula:

[0148]

[0149] Among them, D KL Let represent the Kullback-Leibler divergence, q(Z|X) represent the distribution of latent features generated by the encoder, and p(Z) represent the prior distribution of the latent features, which is usually taken as a Gaussian distribution N(0,I).

[0150] Understandably, regularization constraints make the latent feature representation Z conform to a predefined distribution, which helps improve the stability and generalization performance of the model.

[0151] It should be noted that in some embodiments, feature fusion can also be achieved through weighted concatenation, where the features of each modality are weighted and superimposed according to a certain weight ratio to generate the final fused feature representation.

[0152] The formula is expressed as follows:

[0153]

[0154] Among them, Z fuseE represents the feature representation after fusion. i α represents the feature representation of the i-th mode. i represents the weighting coefficient of mode i, and n represents the number of modes.

[0155] Understandably, through weighted fusion, features from different modalities can be weighted according to their importance, further improving the effect of feature fusion.

[0156] In summary, this step compresses and fuses high-dimensional features of multimodal data using a variational autoencoder, aiming to retain information relevant to the classification task while removing irrelevant information and noise. Simultaneously, mutual information optimization and regularization constraints further enhance the effectiveness and stability of the feature representation. In some embodiments, weighted concatenation can also be used to achieve feature fusion, ensuring that the feature representation has stronger expressive power and generalization ability. This step provides high-quality input data for subsequent dynamic verification mechanisms and classification tasks.

[0157] For step S3, this embodiment proposes an optimization method based on a dynamic verification mechanism to improve the consistency of multimodal features and the accuracy of classification results. This mechanism works collaboratively through three parts: modality consistency verification, modality complementarity verification, and dynamic weight adjustment. This achieves information coordination and unification among multimodal features and enhances the performance of the classification task.

[0158] Modal consistency verification:

[0159] Modality consistency verification aims to reduce the prediction bias of different modality features on the classification target and ensure the coordinated performance of each modality feature. This is specifically achieved through the following loss function:

[0160]

[0161] in, Z represents the loss function for modality consistency verification, used to measure the difference in predictions between different modalities on the classification target. i Z j Let Z represent the feature representations of modality i and modality j, respectively, derived from the fusion of multimodal features. Let f(Z) represent the classification function, whose output is the predicted value of the classification target, typically a probability distribution or classification label. 2 This represents the Euclidean distance, used to measure the difference between classification prediction results of different modalities.

[0162] Modal complementarity verification:

[0163] Modal complementarity verification is used to evaluate the independent contribution of each modal feature to the classification objective, and the complementarity of modal features is enhanced through mutual information optimization. The optimization formula is as follows:

[0164]

[0165] in, Let I(Z) represent the loss function for modal complementarity verification, used to evaluate and optimize the independent contributions of each modal feature. λ represents the weighting coefficient, used to control the impact of modal complementarity verification on the overall optimization. i ;Y) mutual information, representing modal features Z i The amount of information between Z and the classification target Y is used to measure Z. i Independent contribution to Y.

[0166] Dynamic weight adjustment

[0167] Dynamic weight adjustment updates modal weights in real time based on the contribution of each modal feature to the classification task, thereby achieving optimal resource allocation. The weight update rule is as follows:

[0168]

[0169] in, Let represent the weight of modality i in the t-th iteration, and represent the current contribution ratio of the modality feature to the classification task. I(Z) represents the weight of mode i in the (t+1)th iteration, used for the next optimization. i ;Y) mutual information, representing mode Z i Correlation with classification target Y, ∑ j I(Z j ;Y) represents the total mutual information between all modal features and the classification target Y, used for normalization calculation of weights.

[0170] Comprehensive optimization objectives

[0171] By combining modal consistency verification and modal complementarity verification, a comprehensive optimization objective function for the dynamic verification mechanism is formed:

[0172]

[0173] in, A comprehensive optimization objective function is used to simultaneously optimize the consistency and complementarity of modal features. The loss function for modality consistency verification reduces prediction bias between modal features. Loss function for modal complementarity verification, enhancing the independent contribution of modal features.

[0174] Modality consistency verification reduces prediction bias among different modalities in the classification objective, modality complementarity verification enhances the independent contribution of each modality feature, and dynamic weight adjustment optimizes resource allocation, ensuring the coordination and unity of multimodal features in the latent space. Finally, by comprehensively optimizing the objective function, the dynamic verification mechanism effectively improves the collaborative performance of multimodal features and the overall performance of the classification task.

[0175] For step S4, this embodiment provides a technical solution based on dimensionality reduction to embed optimized high-dimensional features into a low-dimensional semantic space. This aims to reduce redundancy in high-dimensional data, retain core information relevant to the classification target, and reduce computational complexity. This step, based on manifold learning and linear transformation techniques, optimizes feature representation through high-dimensional-to-low-dimensional mapping, providing efficient and accurate low-dimensional feature input for generating classification labels.

[0176] It should be noted that this dimensionality reduction method, while ensuring the consistency of the global data structure and semantics, further optimizes the compactness between features, laying a solid data foundation for subsequent classification steps.

[0177] The optimized high-dimensional feature reduction includes the following:

[0178] Constructing neighborhood relationships for high-dimensional features

[0179] Alternatively, this step first constructs neighborhood relationships for high-dimensional feature data points to preserve the local structure of the data. Specifically, the neighborhood of each data point is determined by calculating the similarity between high-dimensional feature data points.

[0180] In one possible implementation, the similarity measure can be Euclidean distance or cosine similarity, defined as follows:

[0181] d ij =||Z i -Z j || 2

[0182] Where, d ij Represents data point Z i and Z j The Euclidean distance between them, Z i and Z j This represents two data points in a high-dimensional feature space.

[0183] Dimensionality reduction mapping

[0184] Specifically, based on the construction of neighborhood relationships, the high-dimensional feature Z is mapped to the low-dimensional semantic space Z. low This ensures that the dimensionality-reduced data retains the consistency of the local neighborhood structure. Commonly used dimensionality reduction methods include principal component analysis and manifold learning methods.

[0185] In one implementation, a dimensionality reduction method based on manifold learning is used, with the following optimization objective:

[0186]

[0187] in, Let Z represent the dimensionality reduction optimization objective function. low,i and Z low,j Let w represent two data points in a low-dimensional semantic space. ij Representing high-dimensional features Z i and Z j The weights between them are usually calculated using a Gaussian kernel function:

[0188]

[0189] Among them, l ij Representing high-dimensional features Z i and Z j The distance between them, σ represents the parameter of the Gaussian kernel, used to control the similarity decay rate.

[0190] It should be noted that by minimizing It can ensure that the features after dimensionality reduction maintain a similar neighborhood structure in the low-dimensional space as in the high-dimensional space.

[0191] Optimize the dimensionality reduction process

[0192] In one implementation, to ensure the global structural consistency of low-dimensional embedded features, global semantic constraints can be introduced. The optimized dimensionality reduction objective is defined as follows:

[0193]

[0194] in, Global optimization objective function, Cov(Z) and Cov(Z) low ) represent the covariance matrices of the high-dimensional and low-dimensional features, respectively, and α and β are trade-off parameters used to control the balance between local neighborhood consistency and global structural consistency.

[0195] The understanding is that by incorporating global semantic constraints, the dimensionality-reduced features can further enhance the global structural expressiveness of the data while preserving local neighborhood consistency.

[0196] Generate low-dimensional embedding features

[0197] For example, assuming the input high-dimensional features are 1024-dimensional, and are mapped to a 128-dimensional semantic space using dimensionality reduction methods, the dimensionality-reduced features can be represented as:

[0198] Z low=DimReduction(Z)

[0199] Among them, Z low DimReduction represents the low-dimensional embedding feature, and DimReduction represents the dimensionality reduction mapping function, including linear transformations or nonlinear methods.

[0200] It should be noted that the low-dimensional embedding feature Z low This is a crucial input for generating subsequent classification labels, ensuring the efficiency and accuracy of the classification task.

[0201] This embodiment details the specific process of dimensionality reduction of optimized high-dimensional features, including neighborhood relation construction, dimensionality reduction mapping, and optimization. It ensures that the dimensionality-reduced features retain core information in the low-dimensional space and maintain semantic consistency with the high-dimensional space.

[0202] For step S5, this embodiment addresses the efficiency and accuracy issues in dynamic classification query tasks by proposing a solution based on constructing a vector index mechanism that supports query and feedback optimization. By establishing a dynamically updated vector index library and introducing a user feedback optimization mechanism, efficient querying and adaptive optimization of the classification query system are achieved. The core of this step is utilizing the dimensionality-reduced low-dimensional feature Z. low Vectorized storage and indexing are implemented, while the index content is dynamically adjusted to adapt to data updates and user needs.

[0203] It should be noted that this step supports fast querying through vector indexing technology, and combined with a user feedback optimization mechanism, ensures that the categorized query results can be dynamically adjusted, thereby improving the system's adaptability and reliability.

[0204] Building vector indexes that support query and feedback optimization includes the following:

[0205] Building a vector index library

[0206] Alternatively, this step involves processing the low-dimensional embedding feature Z. low Vectorized storage and indexing are implemented to build a vector index library that supports efficient querying. Specifically, the vector index library is implemented based on the approximate nearest neighbor search algorithm.

[0207] In one possible implementation, the low-dimensional embedding feature Z low Stored as a vector set {Z low,1 Z low,2 ,…,Z low,n} and indexed by vector similarity (such as cosine similarity or Euclidean distance).

[0208] For example, the similarity calculation formula based on Euclidean distance is as follows:

[0209] d ij=||Z low,i -Z low,j || 2

[0210] Where, d ij Represents vector Z low,i and Z low,j The Euclidean distance between them, Z low,i and Z low,j These are low-dimensional feature vectors.

[0211] It should be noted that the vector index library uses ANN technology, which can significantly reduce query time complexity while maintaining high query accuracy.

[0212] User feedback collection and dynamic updates

[0213] Specifically, to ensure that the categorized query results are consistent with user needs, this step optimizes the index content and categorization model by collecting user feedback.

[0214] In one implementation, the optimization mechanism based on user feedback includes the following steps:

[0215] First, collect user feedback on the category search results, such as user-marked category tags or preferences for the search results.

[0216] Then the feedback data is used as new samples {Z} feedback,1 Z feedback,2 The input index is used to dynamically update the low-dimensional feature set.

[0217] The formula is expressed as:

[0218] Z low,new =Z low,old ∪Z feedback

[0219] Among them, Z low,new Z represents the updated low-dimensional feature set. low,old Z represents the feature set in the original index. feedback This indicates newly generated samples based on user feedback.

[0220] The understanding is that the user feedback mechanism enables the classification system to adapt to changes in user needs by dynamically adjusting the content of the index.

[0221] Feedback-driven classification model optimization

[0222] Alternatively, this step further utilizes user feedback to optimize the classification model, thereby improving the performance of the classification label generation module.

[0223] In one possible implementation, user feedback is treated as new samples with high confidence and used to retrain the classification model. The objective function for the optimization process is defined as follows:

[0224]

[0225] in, This represents the optimized objective function. This represents the loss function of the original classification model. This represents the confidence-weighted loss function for user feedback, where γ is a tradeoff coefficient used to control the impact of user feedback on model optimization.

[0226] It should be noted that by introducing a feedback-driven optimization mechanism, the classification model can more accurately reflect user needs and improve the accuracy of classification results.

[0227] This embodiment details the technical solution for constructing a vector index that supports query and feedback optimization, including the construction of the vector index library, dynamic updates based on user feedback, and feedback-driven classification model optimization. Through efficient indexing techniques and adaptive optimization mechanisms, it ensures that classification query results can meet dynamically changing user needs.

[0228] Please see the appendix Figure 2 The present invention also provides a digital archive classification and query system based on large model index recognition, including: a data acquisition module for acquiring multimodal data such as text, images, audio and structured numerical data;

[0229] The data preprocessing module is used to standardize the collected multimodal data and generate a unified feature representation;

[0230] The feature fusion module is used to optimize the information representation of multimodal data and remove redundant information;

[0231] The validation and optimization module is used to validate the consistency of data features and optimize classification results.

[0232] The dimensionality reduction classification module is used to embed feature data into a low-dimensional semantic space and generate dynamic classification labels;

[0233] The index query module is used to build a vector index library that supports fast querying;

[0234] The user feedback module is used to collect user feedback information and optimize the classification query model.

[0235] The data acquisition module is responsible for collecting multimodal data (such as text, images, audio, and structured numerical data) to provide the system with basic raw input data. This module solves the heterogeneity problem of different data modalities, ensuring the diversity and breadth of the input data.

[0236] Specific content includes: text data acquisition, providing semantic information about the archives, such as titles, descriptions, or tags; image data acquisition, providing visual information about the archives, such as cover photos or scanned documents; audio data acquisition, providing voice recordings, such as interview recordings or meeting minutes; and structured numerical data acquisition, providing statistical information or timestamps, such as dates, category numbers, or statistical fields.

[0237] Significance: To provide sufficient data input for multimodal systems, ensuring that subsequent modules can process and fuse multimodal data.

[0238] The data preprocessing module is responsible for standardizing the collected multimodal data, transforming the raw data into a standardized feature representation. This module addresses the inconsistencies in structure, scale, and distribution among different modalities, ensuring data compatibility in subsequent processes.

[0239] Specific content includes: text processing (word segmentation, stop word removal, and semantic embedding generation); image processing (resolution adjustment, noise reduction, and visual feature extraction); audio processing (converting audio data into text and generating feature vectors using a speech-to-text model); and numerical data processing (normalization to standardize numerical features).

[0240] Significance: To generate a unified multimodal feature representation for the system, eliminate differences between modes, and improve the efficiency and accuracy of subsequent processing.

[0241] The feature fusion module's function is to compress, fuse, and optimize the features of multimodal data to generate an efficient, unified feature representation. This module eliminates data redundancy while retaining core information relevant to the task.

[0242] Specific content: Joint learning of multimodal features is carried out using deep learning models such as variational autoencoders; the feature representation ability is maximized by optimizing the mutual information in the latent feature space; different modal features are weighted and dynamically adjusted according to their task contribution.

[0243] Significance: It solves the problems of information redundancy and representation differences among multimodal data, and provides more compact and effective feature inputs for classification tasks.

[0244] The validation and optimization module verifies the quality and consistency of multimodal features and optimizes classification results through dynamic adjustments. This module ensures that different modalities can work collaboratively in classification tasks while improving the robustness of feature representation.

[0245] Specific content includes: Modal consistency verification: reducing the bias in prediction results of different modal features; Modal complementarity verification: evaluating and optimizing the independent contribution of each modal feature; Dynamic weight adjustment: dynamically allocating weights according to the importance of each modal feature.

[0246] Significance: It enhances the classification system's ability to comprehensively utilize multimodal data, optimizes feature synergy, and improves the accuracy and stability of classification.

[0247] The dimensionality reduction and classification module embeds high-dimensional features into a low-dimensional semantic space, reducing data dimensionality while preserving task-relevant information and generating dynamic classification labels. This module improves the system's computational efficiency and classification performance.

[0248] Specific content: High-dimensional features are mapped to a low-dimensional semantic space through dimensionality reduction techniques; the global structure and local consistency of the data are preserved during the dimensionality reduction process; the generation of dynamic classification labels combines multi-dimensional information such as time, topic, and priority to meet the needs of different application scenarios.

[0249] Significance: While reducing data complexity, it improves the efficiency and accuracy of classification, providing low-dimensional feature inputs for subsequent indexing and querying.

[0250] The index query module's function is to build a vector index library that supports fast queries, utilizing low-dimensional features to achieve efficient classification queries. This module ensures real-time performance and high accuracy in queries.

[0251] Specific features include: indexing and storing low-dimensional features using approximate nearest neighbor technology; achieving fast retrieval through vector similarity (such as cosine similarity or Euclidean distance); and supporting dynamic updates of index content to meet data changes and user needs.

[0252] Significance: It provides low-latency category query capabilities, offering users fast and accurate query results.

[0253] The user feedback module collects user feedback on the categorized query results and uses this feedback to dynamically optimize the categorization model and index content.

[0254] This module improves the system's adaptability and user satisfaction.

[0255] Specifically, the following steps are taken: dynamically adding new samples based on user feedback to the index and updating vector features; optimizing the loss function of the classification model based on feedback information to enhance the model's adaptability to user needs; and adjusting the classification label content to ensure that the classification results can dynamically adapt to user preferences.

[0256] Significance: A closed-loop optimization system based on user feedback was constructed, enabling the classification results to be continuously improved and dynamically adapted to changing needs.

[0257] Each module plays a crucial role in the system, forming a complete technical chain from data acquisition to final query, ensuring the system's ability to process multimodal data, optimize classification tasks, and dynamically adapt to user needs. The functional design of each module complements each other, forming an efficient, accurate, and dynamically optimized classification and query system.

[0258] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A digital archive classification query method based on large model index recognition, characterized in that, The method comprises the following steps: Generating a unified feature representation through the collection and processing of multi-modal data; Optimizing the information representation of multi-modal data based on a feature fusion method; Improving the consistency of data features and the accuracy of classification results using a dynamic verification mechanism; Embedding the optimized features into a low-dimensional semantic space through a dimension reduction method to generate classification labels; Constructing a vector index that supports query and feedback optimization; The dynamic verification mechanism comprises the following steps: Modal consistency verification: By computing the different modal features and The prediction difference on the target classification Optimize the consistency between the modal, define the loss function as: ； wherein, represents a modality and characteristic representation, represents a classification function, represents a Euclidean distance; Modal complementarity verification: Evaluate each modality feature Classify the target The independent contribution of each modality feature, define the complementary loss function as: ； in, Representing modal features With target classification mutual information, As a weighting factor; Dynamic weight adjustment: According to the modal characteristics The relevance to the target classification The modal weight is updated, and the update formula is: ； wherein, denotes the modality the weight in the sub-iteration, denotes the modality In the first the weight in the sub-iteration, denotes the total mutual information of all modality features and the classification target ; Comprehensive optimization objective function: The consistency verification, complementarity verification, and dynamic weight adjustment are combined, and the final optimization objective is: ； wherein, represents a comprehensive optimization objective function of the dynamic verification mechanism, represents a loss function of modal consistency verification, used to minimize the difference between different modal features on the classification target prediction result, represents a loss function of modal complementarity verification, used to evaluate the independent contribution of each modal feature; The dimension reduction method comprises the following steps: Constructing the neighborhood relationship of high-dimensional feature data points by calculating the similarity between data points to determine the neighborhood of each data point: Mapping the high-dimensional features to a low-dimensional space while maintaining the neighborhood relationship to generate low-dimensional embedding features; Optimizing the dimension reduction process to ensure that the low-dimensional embedding features can preserve the global structure and semantic consistency of the high-dimensional data; Constructing a vector index library by vectorizing and indexing the low-dimensional embedding features; The construction of the vector index that supports query and feedback optimization comprises the following steps: Collecting user feedback information on the classification query results and inputting the feedback information as new samples into the model; Optimizing the dynamic classification label generation module through feedback samples to adjust the classification labels to more accurately reflect user needs; Updating the feature fusion module using feedback samples to improve the accuracy of multi-modal data feature representation and classification results; Dynamically adjusting the low-dimensional vector index library to ensure that the new samples and optimized features can be updated to the query system in real time, ensuring the dynamic and accuracy of the query results.

2. The digital archive classification query method based on large model index recognition according to claim 1, characterized in that, The collection of multi-modal data comprises the following steps: Collecting text data and extracting semantic feature vectors through pre-trained language models for word segmentation and embedding; Collecting image data and extracting high-dimensional visual feature vectors through convolutional neural networks; Transcribing audio data into text data using a speech-to-text model and extracting semantic feature vectors; Collecting structured numerical data and adjusting the data range to generate standardized numerical features through normalization methods.

3. The digital archive classification query method based on large model index recognition according to claim 1, characterized in that, The processing of multi-modal data comprises the following steps: Extracting semantic embeddings from text data using language models; Extracting feature embeddings from image data using convolutional neural networks; Transcribing audio data using a speech recognition model and extracting semantic features; Normalizing structured numerical data.

4. The digital archive classification query method based on large model index recognition according to claim 1, characterized in that, The feature fusion method optimization comprises the following steps: Inputting high-dimensional features of multi-modal data into a variational autoencoder model, compressing multi-modal features through an encoder, and generating latent feature representations; In the latent feature space, the optimization objective is defined as: ； wherein, represents the inputted multi-modal feature, represents the latent feature representation, represents the classification target, represents the mutual information between the multi-modal feature and the latent feature, represents the mutual information between the latent feature and the classification target, is a hyper-parameter balancing compression and classification accuracy; Optimizing the latent feature representation to maximize the relevance of the latent feature to the classification task while eliminating redundant features, reducing the redundancy and noise interference between multi-modal features; Generating compressed and optimized fusion feature representations for subsequent classification processing.

5. The digital archive classification query method based on large model index recognition according to claim 1, characterized in that, The generation of classification labels comprises the following steps: The low-dimensional embedding features are input into a large language model, and the initial classification labels are generated by combining the low-dimensional feature representation and the context semantic information; According to the time dimension, theme dimension and priority dimension of the archive data, the initial classification labels are refined; The classification label content is dynamically adjusted, and the final classification label is generated by combining the user query demand and the archive background information.

6. The digital archive classification query method based on large model index recognition according to claim 1, characterized in that, The construction of the vector index includes the following steps: The low-dimensional embedding features are stored in the vector database, and the features are indexed based on the vector similarity; The approximate nearest neighbor search algorithm is used to optimize the index, ensuring low latency and high efficiency in the query process; For the update of dynamic classification labels, the feature data in the vector index is adjusted in real time to support multi-dimensional query and matching of classification labels.

7. The digital archive classification query system based on large model index recognition, applied to the digital archive classification query method based on large model index recognition according to any one of claims 1-6, characterized in that, It includes: A data acquisition module for acquiring multi-modal data of text, images, audio and structured numerical values; A data preprocessing module for standardizing the acquired multi-modal data and generating a unified feature representation; A feature fusion module for optimizing the information representation of multi-modal data and eliminating redundant information; A verification and optimization module for verifying the consistency of data features and optimizing the classification results; A dimension reduction classification module for embedding feature data into a low-dimensional semantic space and generating dynamic classification labels; An index query module for constructing a vector index library that supports fast query; A user feedback module for collecting user feedback information and optimizing the classification query model.

Images