Multimedia data cross-modal retrieval method and system based on multi-label incremental hashing

Abstract

The application belongs to the field of cross-modal retrieval, and provides a multimedia data cross-modal retrieval method and system based on multi-label incremental hashing, extracts newly added multi-modal combined data and adaptive code length based on an online multimedia data stream; constructs a global hash center according to the relationship between the adaptive code length and the historical adaptive code length; obtains an initial global hash code based on the newly added multi-modal combined data and the global hash center; iteratively and alternately optimizes the initial global hash code based on a hash code optimization objective function, obtains an optimal newly added hash code, and merges the optimal newly added hash code with historical hash codes to obtain a global hash code; solves a linear regression model based on the global hash code to obtain a projection matrix of each mode; encodes a to-be-retrieved sample according to the projection matrix of the corresponding mode to obtain a retrieval result; the application adapts to a multi-label incremental scene, supports dynamic expansion of hash codes, and does not need to play back old data, so as to solve the efficiency, precision and privacy pain points of the existing online cross-modal hashing scheme.

Description

Technical Field

[0001] This invention belongs to the field of cross-modal retrieval technology, specifically relating to a method and system for cross-modal retrieval of multimedia data based on multi-label incremental hashing. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of internet technology, the demand for cross-modal retrieval of large-scale multimedia data (including images, text, and audio) continues to grow. Cross-modal hashing methods, by mapping high-dimensional features of different modalities to a low-dimensional Hamming space, can significantly reduce storage costs and retrieval latency while ensuring retrieval accuracy, and have become a key technology for dealing with large-scale cross-modal retrieval tasks.

[0004] However, existing cross-modal hashing methods still face several key bottlenecks in practical applications in open environments, making it difficult to adapt to the complex needs of continuous data collection, dynamic category expansion, and coexistence of multiple label semantics in real-world business.

[0005] First, traditional offline cross-modal hashing methods require retraining the model and regenerating the global hash code using all historical data when processing new data. The computational complexity increases exponentially with the data size, leading to prolonged system service interruptions and failing to meet the real-time processing requirements of online streaming data. Second, most existing online cross-modal hashing methods are limited to closed-set settings, making them ineffective in handling the dynamic introduction of new categories. Furthermore, most methods are only applicable to single-label data, struggling to address the semantic gap problem faced by multi-label multimedia data (e.g., images simultaneously labeled "landscape," "mountain," and "lake"), resulting in a significant decrease in retrieval accuracy. Additionally, fixed-length hash codes have limited semantic expressive power as categories expand, easily leading to hash collisions; setting excessively long codes in advance introduces unnecessary storage and computational burdens in the early stages of system deployment. On the other hand, to mitigate the catastrophic forgetting problem in incremental learning, existing strategies typically require replaying some historical data, which not only risks user privacy leaks but also introduces additional storage and computational overhead.

[0006] Therefore, current technology lacks an online cross-modal hashing solution that can simultaneously adapt to multi-label incremental scenarios, support dynamic expansion of hash codes, and eliminate the need to replay historical data. This makes it impossible to systematically address the challenges currently faced in terms of retrieval efficiency, model accuracy, and data privacy. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a method and system for cross-modal retrieval of multimedia data based on multi-label incremental hashing. This invention achieves efficient and accurate online cross-modal retrieval through scalable hash centers, multi-label distribution modeling, and no-replay optimization.

[0008] According to some embodiments, the first solution of the present invention provides a method for cross-modal retrieval of multimedia data based on multi-tag incremental hashing, which adopts the following technical solution:

[0009] Multimedia data cross-modal retrieval methods based on multi-label incremental hashing include:

[0010] Preprocessing is performed on online multimedia data streams to extract newly added multimodal combination data and adaptive code length in the current round;

[0011] The historical hash center is expanded based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and a global hash center is constructed based on the newly added hash center of the current round and the expanded historical hash center.

[0012] A global index matrix is ​​constructed based on the newly added multimodal combination data in the current round, and the initial global hash code is obtained by multiplying the global index matrix and the global hash center.

[0013] The initial global hash code is iteratively optimized by optimizing the objective function of hash code optimization, and the optimal newly added hash code is obtained. The optimal newly added hash code and the historical hash code are then merged to obtain the global hash code.

[0014] With the goal of minimizing the error between the global hash code and modal features, a linear regression model is constructed, and the linear regression model is solved based on the global hash code to obtain the projection matrix of each modality;

[0015] The retrieval results are obtained by encoding the sample based on the projection matrix of the modality corresponding to the sample.

[0016] Furthermore, the preprocessing based on online multimedia data streams to extract newly added multimodal combination data and adaptive code length in the current round includes:

[0017] The online multimedia data stream is acquired and divided into new multimodal data blocks according to rounds. Feature extraction is performed on the new multimodal data blocks to obtain the new feature matrix and new label matrix of each modality in the current round.

[0018] The cumulative number of categories in the current round is determined based on the newly added categories in the data label matrix of the current round and the cumulative number of categories in the previous round. The adaptive code length for the current round is determined by a power of 2 that is not less than the cumulative number of categories in the current round.

[0019] Furthermore, the step of expanding the historical hash center based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and constructing a global hash center based on the newly added hash center of the current round and the expanded historical hash center, includes:

[0020] Generate a Hadamard matrix based on the adaptive code length of the current round, and extract the newly added hash center of the current round from the Hadamard matrix;

[0021] If the adaptive code length of the current round is equal to the historical adaptive code length, then the historical hash center remains unchanged;

[0022] If the adaptive code length of the current round is equal to twice the historical adaptive code length, then the historical hash center will be self-replicated to obtain the expanded historical hash center.

[0023] A global hash center is constructed based on the newly added hash centers in the current round and the expanded historical hash centers.

[0024] Furthermore, the step of constructing a global index matrix based on the newly added multimodal combination data in the current round, and obtaining the initial global hash code based on the product of the global index matrix and the global hash center, includes:

[0025] A new index matrix is ​​obtained by clustering the newly added label matrix in the newly added multimodal combined data of the current round;

[0026] The historical global index matrix of the previous round is expanded to obtain the expanded historical global index matrix. The global index matrix is ​​constructed based on the expanded historical global index matrix and the newly added index matrix.

[0027] The initial global hash code is obtained by multiplying the global index matrix and the global hash center.

[0028] Furthermore, the initial global hash code is iteratively optimized alternately based on the hash code optimization objective function to obtain the optimal newly added hash code. The optimal newly added hash code and the historical hash codes are then merged to obtain the global hash code, including:

[0029] Initialize based on the adaptive code length and initial global hash code of the current round to obtain the hash code to be optimized and the historical hash code of the newly added data;

[0030] By combining global hash center constraints and local semantic similarity, a hash code optimization objective function is constructed.

[0031] With the hash code to be optimized fixed, the first auxiliary variable is introduced to solve the hash code optimization objective function, thus obtaining the current round. The Middle Intermediate variables after alternating optimization;

[0032] Fix the current round The Middle After alternating optimization, intermediate variables are introduced as a second auxiliary variable to solve the hash code optimization objective function, thus obtaining the current round. The Middle The newly added hash code after alternating optimization;

[0033] The process continues until the number of alternating optimization iterations reaches a specified threshold, at which point the optimal new hash code is obtained. The optimal new hash code is then merged with the historical hash codes to obtain the global hash code.

[0034] Furthermore, the process of constructing a linear regression model with the objective of minimizing the error between the global hash code and the modal features, and solving the linear regression model based on the global hash code to obtain the projection matrix of each modality, includes:

[0035] A linear regression model is constructed with the goal of minimizing the error between the global hash code and the modal features.

[0036] Based on the global hash code, the third and fourth auxiliary variables from the previous round, and the newly added feature matrix of the current round, calculate the third and fourth auxiliary variables of the current round.

[0037] Based on the third and fourth auxiliary variables of the current round, the projection matrix of each modality is obtained.

[0038] According to some embodiments, a second aspect of the present invention provides a multimedia data cross-modal retrieval system based on multi-tag incremental hashing, employing the following technical solution:

[0039] A multimedia data cross-modal retrieval system based on multi-label incremental hashing includes:

[0040] The data acquisition and processing module is used to preprocess online multimedia data streams and extract newly added multimodal combination data and adaptive code length in the current round.

[0041] The global hash center generation module is used to expand the historical hash center based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and to construct the global hash center based on the newly added hash center of the current round and the expanded historical hash center.

[0042] The initial global hash code construction module is used to construct a global index matrix based on the newly added multimodal combination data in the current round, and obtain the initial global hash code based on the product of the global index matrix and the global hash center.

[0043] The global hash code construction module is used to iteratively optimize the initial global hash code based on the hash code optimization objective function to obtain the optimal newly added hash code. The optimal newly added hash code and the historical hash codes are then merged to obtain the global hash code.

[0044] The hash learning module is used to construct a linear regression model with the goal of minimizing the error between the global hash code and the modal features. It solves the linear regression model based on the global hash code to obtain the projection matrix of each modality.

[0045] The cross-modal retrieval module is used to encode the sample to be retrieved based on the projection matrix of the modality corresponding to the sample to be retrieved, and obtain the retrieval results.

[0046] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0047] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in the first scheme above.

[0048] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0049] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in the first embodiment above.

[0050] According to some embodiments, a fifth aspect of the present invention provides a computer program product or computer program.

[0051] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in the first embodiment above.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] This invention introduces four lightweight auxiliary variables, T1–T4, to achieve continuous model updates without relying on historical raw data, completely avoiding the privacy leakage risk caused by replaying old samples, while significantly reducing storage and computational overhead, realizing replay-free incremental learning, and balancing privacy protection and training efficiency. By using k-means clustering to map multi-label samples to a discrete distribution, and using the Hadamard matrix to construct hash centers with strong orthogonality, it significantly alleviates semantic confusion and hash code collisions in the case of multi-label coexistence, improves cross-modal retrieval accuracy, and effectively solves the semantic gap and hash collision problems of multi-label data.

[0054] The hash code length of this invention is dynamically adjusted to a power of 2 based on the number of categories. While ensuring semantic distinguishability, it avoids early resource waste caused by excessively long preset codes, achieving an adaptive balance between retrieval accuracy and system efficiency. It supports dynamic expansion of hash code length, taking into account both expressive power and retrieval efficiency. During the incremental learning process of the model, the hash codes of historical samples remain stable and do not require retraining or generation, completely avoiding server downtime caused by global recoding. This meets the high availability requirements of streaming data scenarios, and historical hash codes do not need to be regenerated, achieving zero-cost updates.

[0055] This invention has comprehensive advantages in terms of not needing to replay historical data, supporting multi-label semantic modeling, dynamically expanding hash code length, and updating hash code at zero cost. It can effectively cope with the real-time retrieval challenges of large-scale multimodal streaming data in open environments, and achieve simultaneous improvement in three dimensions: retrieval accuracy, system efficiency, and privacy protection. Attached Figure Description

[0056] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0057] Figure 1 This is a flowchart of a multimedia data cross-modal retrieval method based on multi-label incremental hashing in an embodiment of the present invention. Detailed Implementation

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0060] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0061] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0062] As described in the background section, existing technologies have significant limitations in the field of online cross-modal retrieval: offline cross-modal hashing methods require retraining the model based on the full historical data, resulting in extremely high computational costs and an inability to adapt to dynamic incremental data streams; existing online cross-modal hashing methods mostly rely on the closed-set assumption (fixed data categories), making it difficult to handle real-world scenarios containing new categories; some methods also require replaying old data to mitigate catastrophic forgetting, raising the risk of data privacy leaks; they also cannot effectively address the ambiguous task boundaries where new and old categories coexist in multi-label data, and binary logical labels cannot reflect label distribution and inter-class relationships; furthermore, fixed-length hash codes either lead to hash collisions due to insufficient semantic expression or reduce early retrieval efficiency due to pre-setting long code lengths. Therefore, there is an urgent need for an online cross-modal hashing solution that can adapt to multi-label incremental scenarios, support dynamic hash code expansion, and eliminate the need for replaying old data, in order to address the efficiency, accuracy, and privacy pain points of existing technologies. Therefore, it can be seen that the technical problems to be solved by this invention are: how to achieve incremental hash model updates without replaying old data, while taking into account both privacy protection and update efficiency; how to characterize the semantic relationship between the coexistence of new and old categories in multi-label data, and improve the ability of hash codes to distinguish multi-label semantics; and how to adaptively adjust the hash code length to match the dynamic expansion of categories, avoiding the loss of accuracy or low efficiency caused by fixed length.

[0063] Example 1

[0064] like Figure 1 As shown, this embodiment provides a method for cross-modal retrieval of multimedia data based on multi-tag incremental hashing. This embodiment uses the application of this method to a server as an example for illustration. It can be understood that this method can also be applied to terminals, and can also be applied to systems including terminals, servers, and systems, and can be implemented through the interaction between terminals and servers. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0065] Preprocessing is performed on online multimedia data streams to extract newly added multimodal combination data and adaptive code length in the current round;

[0066] The historical hash center is expanded based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and a global hash center is constructed based on the newly added hash center of the current round and the expanded historical hash center.

[0067] A global index matrix is ​​constructed based on the newly added multimodal combination data in the current round, and the initial global hash code is obtained by multiplying the global index matrix and the global hash center.

[0068] The initial global hash code is iteratively optimized by optimizing the objective function of hash code optimization, and the optimal newly added hash code is obtained. The optimal newly added hash code and the historical hash code are then merged to obtain the global hash code.

[0069] With the goal of minimizing the error between the global hash code and modal features, a linear regression model is constructed, and the linear regression model is solved based on the global hash code to obtain the projection matrix of each modality;

[0070] The retrieval results are obtained by encoding the sample based on the projection matrix of the modality corresponding to the sample.

[0071] This invention supports a cross-modal retrieval method for multi-label incremental hash multimedia data with scalable hash codes. The overall process revolves around efficient online cross-modal retrieval of multi-label incremental data streams, as detailed below:

[0072] First, at each time step, a new multimodal data stream (such as an image-text pair) is received and preprocessed, including using the CLIP model to extract features of each modality and normalizing them to eliminate heterogeneity, while regularizing the multi-label matrix (new data labels are binary matrices, and old data labels are expanded by zero-padding matrices to adapt to the new categories).

[0073] Next, we enter the core hash code learning stage. First, we solve for the adaptive code length based on the cumulative number of categories (satisfying a power of 2 or higher). Then, we recursively generate scalable hash centers through the Hadamard matrix (the new category hash center is selected from the corresponding row of the Hadamard matrix, and the old category hash center is selected to remain unchanged or self-replicate and expand depending on whether the code length is doubled).

[0074] Subsequently, k-means clustering is performed on the new data label matrix to construct a multi-label distribution and global index matrix. Then, local semantic information is introduced by minimizing the loss of hash code inner product and label similarity. With the help of auxiliary variables and the transfer of old knowledge, the intermediate variables and hash codes of the new data are iteratively updated (without replaying the old data). Finally, the hash codes of the new and old data are merged according to the code length.

[0075] Next, we learn about hash functions. Through a linear regression model (including regularization terms) and auxiliary variables, we solve for the projection matrices of each modality and generate a hash function in the form of feature mapping + sign function to support fast encoding of offsampled samples.

[0076] Finally, during cross-modal retrieval, features are extracted from the query samples (images / text) and hash codes are generated using a hash function. The Hamming distance between the hash codes of the query samples and the hash codes of the database samples is calculated and sorted. The top-k semantically similar heteromodal results are returned. The entire process uses adaptive code length to achieve dynamic bit expansion, balancing retrieval accuracy and efficiency while avoiding data privacy risks.

[0077] This embodiment presents a method for cross-modal retrieval of multimedia data based on multi-label incremental hashing, including:

[0078] Step S1: Preprocess the online multimedia data stream to extract the newly added multimodal combination data and adaptive code length in the current round, including:

[0079] Step S1.1: Acquire the online multimedia data stream and divide it into new multimodal data blocks according to the round. Extract features from the new multimodal data blocks to obtain the new feature matrix and new label matrix of each modality in the current round.

[0080] Specifically, for feature extraction and standardization, the first step is to acquire the data stream, in each round... Online multimedia data streams are collected and divided into new multimodal data blocks according to rounds. This embodiment takes the image-text modality as an example (which can be extended to retrieval between other multimodal data such as audio and video). For feature extraction and standardization of the new multimodal data blocks, in the image modality, a pre-trained CLIP model is used to extract 512-dimensional features for general scenarios (balancing feature representation ability and efficiency), as expressed in the formula:

[0081]

[0082] in, It is the first New inflow of raw image modal data, The current round The number of new samples, corresponding The sample dimension For the first Round 1 Modus - Initial newly added feature matrix of the image, It is a feature extraction function for image modalities.

[0083] For text modalities, the CLIP model is used to extract features, and the formula is expressed as:

[0084]

[0085] in, It is the first New inflow of raw text modal data, For the first The initial new feature matrix of the second modality of the round - text. It is a feature extraction function for text modalities.

[0086] Furthermore, heterogeneity elimination is performed by processing the newly added feature matrices for each modality. Normalization ensures that the norms of features from different modalities are consistent, avoiding the impact of differences in feature magnitude on subsequent hash learning. The resulting new feature matrices for each modality are as follows:

[0087] ;

[0088] in, For the first The newly added feature matrix for the m-th modality in round m, where m=1 represents images and m=2 represents text. For the first The initial newly added feature matrix for the m-th modality in round m, where m=1 represents images and m=2 represents text. It is a second-order norm.

[0089] For label normalization and expansion, the corresponding label is determined based on the category of each data point in the newly added multimodal data block to form the current round. New label matrix for each modality The newly added label matrix is ​​a binary matrix; for example, Indicates the first The newly added sample belongs to the first If the class is specified, the value is 0; otherwise, it is 0. New samples refer to image-text pairs along with their multiple labels (e.g., [landscape, mountain, lake]). Special handling is needed for overlapping labels (e.g., a sample simultaneously belongs to both animal and pet categories). , , This ensures that the label matrix fully reflects the category affiliation of the samples.

[0090] For the expansion of the newly added label matrix: due to the first New round of additions Each category needs to include the old label matrix from the previous round. Expanded into an extended label matrix , where 0 is The zero matrix, Indicates the first The total number of image-text pairs in the wheel solves the format adaptation problem for old samples without new category labels, and does not require modification of the original labels of old samples, thus avoiding the risk of data tampering.

[0091] In summary, each round The newly obtained multimodal combination data is denoted as ,in, For the current round The newly added feature matrix of the m-th mode, For the current round New label matrix for each modality, The current round The cumulative number of categories;

[0092] Historical data in the database That is, the first The complete database for the wheel only needs to store features and hash codes, without retaining the original data, thus reducing privacy risks; among them, It was the previous round. The extended feature matrix of the global m-th mode. It was the previous round. The extended label matrix.

[0093] Step S1.2: Determine the cumulative number of categories in the current round based on the newly added categories in the data label matrix and the cumulative number of categories in the previous round. Then, determine the adaptive code length for the current round, constrained to be a power of 2 that is not less than the cumulative number of categories in the current round, as follows:

[0094]

[0095] in, This is the adaptive code length for the current round. The adaptive code length must satisfy two core constraints: ensure it is not less than the cumulative number of categories. And it is a power of 2.

[0096] Step S2: Expand the historical hash center based on the relationship between the current round's adaptive code length and the historical adaptive code length. Construct a global hash center based on the newly added hash center in the current round and the expanded historical hash center, including:

[0097] Step S2.1: Generate a Hadamard matrix based on the adaptive code length of the current round, and extract the newly added hash center of the current round from the Hadamard matrix;

[0098] Based on the adaptive code length of the current round Generate the Hadamard matrix ,as follows:

[0099]

[0100] Among them, the Hadama matrix The elements take values ​​of {-1, 1}, satisfying row orthogonality, and any two rows are orthogonal (with the largest Hamming distance), ensuring the semantic distinguishability of the hash center.

[0101] Hadamard matrix ,as follows:

[0102] ;

[0103] The Hadamard matrix – the construction rules for the Hadamard matrix are based on a recursive method, i.e. Initial Hadamard matrix ( This construction method ensures that the expanded matrix retains orthogonality, eliminating the need to retrain the old hash center.

[0104] From the Hadama matrix The Arriving at the Extract the hash center of the newly added category and denote it as the new hash center. The formula is:

[0105] ;

[0106] Step S2.2: If the adaptive code length of the current round is equal to the historical adaptive code length, then the historical hash center remains unchanged;

[0107] Specifically, if If the code length remains unchanged, the historical category hash center will remain unchanged (no modification required, reducing computational cost).

[0108] Step S2.3: If the adaptive code length of the current round is equal to twice the historical adaptive code length, then the historical hash center is self-replicated to obtain the expanded historical hash center;

[0109] Combining steps S2.2 and S2.3, the overall expansion formula for the expanded historical hash center is:

[0110] ;

[0111] in, It was the previous round. Historical adaptive code length, It was the previous round. Historical hash center;

[0112] Step S2.4: Construct a global hash center based on the newly added hash centers and the expanded historical hash centers in the current round;

[0113] Specifically, the expanded historical hash center and the newly added hash center are merged to obtain the global hash center. for:

[0114] ;

[0115] This step ensures that the hash centers of both the old and new categories satisfy orthogonality, that is, maximizes the Hamming distance between categories, resulting in strong semantic distinguishability.

[0116] The core innovation of this embodiment lies in learning hash codes. It addresses the three major challenges of existing technologies one by one through four stages: generating scalable bit hash centers, constructing multi-label distributions, embedding semantic information, and continuously optimizing without replay.

[0117] To address the semantic limitations of fixed code lengths, a scalable hash center based on the Hadamard matrix is ​​designed, with the code length dynamically adjusted as the category increments.

[0118] To address the ambiguous boundaries in multi-label tasks, k-means clustering is used to map multi-label samples to a multi-label distribution, replacing a single category with the distribution and capturing label associations.

[0119] To address the privacy risks associated with replaying old data, auxiliary variables are introduced to pass on old knowledge, achieving replay-free optimization and mitigating catastrophic forgetting.

[0120] To generate an scalable bit hash center, existing online hashing methods often use fixed code lengths (such as 32-bit or 64-bit). When the number of categories exceeds the upper limit of the code length, hash collisions occur (different categories correspond to the same hash code), leading to a sharp drop in retrieval accuracy. Step S2 of this embodiment is based on the orthogonality of the Hadamard matrix (the Hamming distance between any two rows of hash centers is maximized) to construct a global hash center that dynamically expands with the increase in categories, ensuring that each category (or distribution) has a unique and highly discriminative semantic anchor.

[0121] Step S3: Construct a global index matrix based on the newly added multimodal combination data in the current round. Obtain the initial global hash code by multiplying the global index matrix and the global hash center, including:

[0122] Step S3.1: Cluster the newly added label matrix based on the newly added multimodal combined data in the current round to obtain the newly added index matrix;

[0123] The k-means algorithm is used to cluster the newly added label matrix of the new data, with the number of clusters set as the cumulative number of categories, resulting in a one-hot index matrix minus the newly added index matrix. ,as follows:

[0124]

[0125] So, for a certain element in the newly added index matrix ,when Time indicates the first The newly added sample belongs to the first The th cluster, i.e. the th th cluster. Each new sample corresponds to only one distribution. For example, samples such as cat + animal and dog + animal will be clustered into the same animal-related distribution, and samples such as scenery + building and river + scenery will be clustered into scenery-related distribution. In this way, the distribution is used to capture the association of labels, and the vague task boundary is replaced by the clear distribution boundary.

[0126] Step S3.2: Expand the historical global index matrix of the previous round to obtain the expanded historical global index matrix, and construct the global index matrix based on the expanded historical global index matrix and the newly added index matrix;

[0127] Historical global index matrix of the previous round The process is expanded to obtain the expanded historical global index matrix. Zero matrix dimension This indicates that the old samples do not belong to the new distribution;

[0128] Construct a global index matrix based on the expanded historical global index matrix and the newly added index matrix. ,as follows:

[0129]

[0130] in, For the first The total number of image-text pairs in a round, which is also the global sample count, includes the total number of image-text pairs in the previous round and the number of newly added image-text pairs in the current round;

[0131] Step S3.3: Obtain the initial global hash code based on the product of the global index matrix and the global hash center, as follows:

[0132]

[0133] in, It is the initial global hash code for the current round, ensuring that samples from the same multi-label distribution have the same hash code and samples from different distributions have orthogonal hash codes.

[0134] Existing multi-label hashing methods directly bind categories to hash centers. However, if a sample belongs to multiple categories (such as cat + animal + pet), it will lead to a conflict where a sample corresponds to multiple hash centers. Furthermore, binary logical labels cannot reflect the label distribution (such as the correlation between cat and animal being higher than that between cat and scenery). In this embodiment, step S3 maps multi-label samples to multi-label distributions through k-means clustering. Each distribution corresponds to a hash center, which can solve the conflict problem and capture the semantic relationship between labels.

[0135] Step S4: Iteratively optimize the initial global hash code based on the hash code optimization objective function to obtain the optimal newly added hash code. Then, merge the optimal newly added hash code with the historical hash codes to obtain the global hash code, including:

[0136] Global hash centers provide category-level semantic anchors, but lack sample-level local similarity constraints (e.g., for two samples from the same animal family, if one is a cat and the other is a dog, subtle differences need to be reflected in their hash codes). This step embeds local semantic information into the hash codes by minimizing the loss between the inner product of the hash codes and the label similarity, ensuring that samples with similar semantics have closer hash codes.

[0137] Step S4.1: Initialize based on the adaptive code length and initial global hash code of the current round to obtain the hash code to be optimized and the historical hash code of the newly added data;

[0138] The historical hash code is determined based on the adaptive code length of the current round, as follows:

[0139] like ,but ;

[0140] like ,but ;

[0141] in, It was the previous round. The global hash code;

[0142] Based on the relationship between historical hash codes and the initial global hash code, determine the hash code to be optimized. ,as follows:

[0143]

[0144] in, It is the historical hash code corresponding to the old data. This corresponds to the hash code to be optimized for newly added data; the initial global hash code. The dimension is , ;

[0145] Step S4.2: Combining global hash center constraints and local semantic similarity, construct the hash code optimization objective function as follows:

[0146]

[0147] in, It is a newly added tag matrix Normalization function; As an intermediate variable;

[0148] Perform on the newly added tag matrix Normalization (eliminating the influence of differences in the number of labels) yields the normalized label matrix, calculated using the following formula:

[0149]

[0150] in, It is the second norm; the purpose of normalization is to transform multi-label 0 / 1 vectors into unit length vectors so that the subsequent inner product equals the cosine similarity.

[0151] Then the semantic similarity between samples is the inner product of the normalized label matrices. ;in, It is the normalized label matrix of the newly added samples in the current round. It is the transpose of the normalized label matrix of the newly added samples in the current round; this similarity can reflect the similarity of the label distribution (e.g., the similarity between cat + animal and dog + animal is higher than that between cat + animal and scenery + building).

[0152] Step S4.3: Fix the hash code to be optimized, introduce the first auxiliary variable to solve the hash code optimization objective function, and obtain the current round. The Middle Intermediate variables after alternating optimization;

[0153] Step S4.3.1: Fix the hash code to be optimized. Based on the first auxiliary variable after the last alternating optimization in the previous round, the adaptive code length of the current round, the hash code to be optimized, and the newly added label matrix, calculate the current round's... The Middle The first auxiliary variable after alternating optimization;

[0154] The hash code optimization objective function is transformed into a trace maximization problem, with the first auxiliary variable... Responsible for passing on the semantic knowledge of old data and calculating the current round. The Middle The first auxiliary variable after alternation optimization The calculation formula is as follows:

[0155]

[0156] in, It is a newly added tag matrix 2. Normalization function, The current round The Middle The newly added hash code after alternation optimization. , This is a specified threshold for the number of alternating optimization iterations; it can be understood that the initial input of this step is the hash code to be optimized for the corresponding newly added data obtained from the initial global hash code in step S3. ; The current round The Middle The first auxiliary variable after each alternating optimization can be understood as follows: the initial input of this step is the first auxiliary variable after the last alternating optimization in the previous round, and subsequent calculations will be based on the first auxiliary variable after alternating optimization in the current round; the calculation formula of the first auxiliary variable is the calculation process of iterative alternating optimization in the current round.

[0157] Step S4.3.2: Initialize intermediate variables based on the current round. The Middle The optimal intermediate variable is determined by the first auxiliary variable after alternating optimization, the newly added label matrix, and the adaptive code length.

[0158] Intermediate variables were randomly initialized using a standard normal distribution. ;

[0159] Based on the current round The Middle The formulas for the first auxiliary variable, the newly added label matrix, the semantic matrix determined by the adaptive code length, and the centering matrix after alternating optimization are as follows:

[0160]

[0161]

[0162] in, It is a semantic matrix, the function of which is to integrate adaptive code length, normalized labels, and historical knowledge into a unified semantic matrix for subsequent centralized processing, avoiding numerical chaos caused by direct calculation of multiple variables; Is it related to the current round? Number of new samples Identity matrices of the same dimension; It is an all-1 vector of dimension n(t)×1. It is a matrix of all ones. It is a centered matrix that eliminates the mean bias of newly added sample data (i.e., brings the mean of the sample data to zero), avoiding the mean from interfering with the stability of subsequent Singular Value Decomposition (SVD), and optimizes intermediate variables. The necessary preprocessing steps.

[0163] The symmetric covariance matrix after centering is obtained by calculating based on the semantic matrix and the centering matrix. For the symmetric covariance matrix after centering Singular value decomposition yields the eigenvector matrix. ,as follows:

[0164]

[0165] The centered symmetric covariance matrix is ​​used to find the optimal intermediate variable in one step and does not participate in subsequent calculations. Its function is to transform the trace maximization problem with center constraints into a standard symmetric eigenvalue decomposition. After performing eigenvalue decomposition, the eigenvector matrix corresponding to the non-negative eigenvalues ​​is taken to obtain the optimal intermediate variable.

[0166] The optimal solution for intermediate variables is determined based on the eigenvector matrix, i.e., the solution for the current round. The Middle The optimal intermediate variable after alternation optimization The calculation is as follows:

[0167]

[0168] in, The matrix of eigenvectors corresponding to the non-negative eigenvalues. yes orthogonal complement, It is a key component in constructing intermediate variables, and its function is to ensure that the intermediate variables satisfy orthogonality constraints and dimensionality integrity. , It is a random orthogonal matrix. It is a diagonal matrix of non-negative eigenvalues;

[0169] Step S4.4: Fix the current round The Middle After alternating optimization, intermediate variables are introduced as a second auxiliary variable to solve the hash code optimization objective function, thus obtaining the current round. The Middle The newly added hash code after alternating optimization;

[0170] Step S4.4.1: Fix the current round The Middle After the optimal intermediate variable following the alternation optimization, the current round's adaptive code length, optimal intermediate variable, and newly added tag matrix are used to calculate the optimal intermediate variable after the second auxiliary variable following the last alternation optimization in the previous round. The Middle The second auxiliary variable after alternation optimization;

[0171] Fix the current round The Middle The optimal intermediate variable after alternation optimization By introducing a second auxiliary variable to pass on the intermediate variable knowledge of old data, and extending the rules to be consistent with the first auxiliary variable, the current round is calculated. The Middle The second auxiliary variable after alternation optimization The calculation formula (for the case of code length, the logical calculation is the same as that for the first auxiliary variable) is as follows:

[0172]

[0173] in, The current round The Middle The second auxiliary variable after alternating optimization can be understood as follows: the initial input of this step is the second auxiliary variable after the last alternating optimization in the previous round, and the subsequent calculations are the second auxiliary variable after alternating optimization in the current round; the calculation formula of the above second auxiliary variable is the calculation process of iterative alternating optimization in the current round.

[0174] Step S4.4.2: Based on the current round The Middle The second auxiliary variable after alternation optimization, the newly added index matrix, the newly added label matrix, and the global hash center are used to determine the current round. The Middle After the alternation optimization, a new hash code is added. The calculation formula is:

[0175] ;

[0176] in, Ensure that the hash code fits the global semantic center. Incorporating local semantic similarity, symbolic functions Convert the real-valued result into a discrete hash code of {-1,1}.

[0177] Step S4.5: Repeat steps S4.3-S4.4 until the number of alternating optimization iterations reaches the specified threshold, and the optimal new hash code is obtained. The optimal newly added hash code is merged with the historical hash codes to obtain the global hash code. The merging formula is as follows:

[0178]

[0179] in, It was the previous round. The global hash code, and the historical hash code are different depending on the relationship between the adaptive code length of the current round and the adaptive code length of the previous round;

[0180] Existing incremental learning methods require replaying old data to mitigate catastrophic forgetting, but replaying increases storage costs and can lead to privacy leaks. This embodiment introduces auxiliary variables to pass old knowledge, and only uses new data to iteratively update intermediate variables and new hash codes, without the need to replay old data.

[0181] Step S5: With the goal of minimizing the error between the global hash code and the modal features, construct a linear regression model. Solve the linear regression model based on the global hash code to obtain the projection matrix of each modality, including:

[0182] Step S5.1: With the goal of minimizing the error between the global hash code and the modal features, construct a linear regression model as follows:

[0183] ;

[0184] Minimize the F-norm loss of hash code inner product and label similarity, constrain hash codes to {-1, 1}, and discrete hash code retrieval efficiency is high;

[0185] Step S5.2: Based on the global hash code and the third auxiliary variable from the previous round and the fourth auxiliary variable Calculate the newly added feature matrix for the current round, and then calculate the third auxiliary variable for the current round. and the fourth auxiliary variable ;

[0186] Introducing a third and fourth auxiliary variable, the calculation formula is as follows:

[0187]

[0188]

[0189] and It requires storing the matrix but does not need to retain old features and old hash codes, thus reducing storage costs; and the calculation only depends on the new data, resulting in high update efficiency.

[0190] Step S5.3: Based on the third and fourth auxiliary variables of the current round, obtain the projection matrix of each modality. It does not rely on historical sample features, and the formula is as follows:

[0191] .

[0192] Step S6: Encode the sample to be retrieved according to the projection matrix of the modality corresponding to the sample to be retrieved, and obtain the retrieval results, including:

[0193] Step S6.1: Extract the corresponding modal feature vector based on the sample to be retrieved, and perform hash calculation based on the modal feature vector and the projection matrix of the corresponding modality to obtain the sample modal hash code, as follows:

[0194]

[0195] in, It is the modal feature vector of the sample to be retrieved;

[0196] Step S6.2: Calculate the Hamming distance between the sample modal hash code and the global sample hash code. The formula for calculating Hamming distance is as follows:

[0197] ;

[0198] in, It is the sample modal hash code. It is the global sample hash code, i.e., the current round. The hash codes of all historical data and newly added data;

[0199] Step S6.3: Sort the global samples in ascending order of Hamming distance, and select the top... global samples ( Set values ​​for users (such as 10, 20) as search results.

[0200] Example 2

[0201] This embodiment provides a multimedia data cross-modal retrieval system based on multi-label incremental hashing, including:

[0202] The data acquisition and processing module is used to preprocess online multimedia data streams and extract newly added multimodal combination data and adaptive code length in the current round.

[0203] The global hash center generation module is used to expand the historical hash center based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and to construct the global hash center based on the newly added hash center of the current round and the expanded historical hash center.

[0204] The initial global hash code construction module is used to construct a global index matrix based on the newly added multimodal combination data in the current round, and obtain the initial global hash code based on the product of the global index matrix and the global hash center.

[0205] The global hash code construction module is used to iteratively optimize the initial global hash code based on the hash code optimization objective function to obtain the optimal newly added hash code. The optimal newly added hash code and the historical hash codes are then merged to obtain the global hash code.

[0206] The hash learning module is used to construct a linear regression model with the goal of minimizing the error between the global hash code and the modal features. It solves the linear regression model based on the global hash code to obtain the projection matrix of each modality.

[0207] The cross-modal retrieval module is used to encode the sample to be retrieved based on the projection matrix of the modality corresponding to the sample to be retrieved, and obtain the retrieval results.

[0208] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0209] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0210] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0211] Example 3

[0212] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in Embodiment 1 above.

[0213] Example 4

[0214] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in Embodiment 1 above.

[0215] Example 5

[0216] This embodiment provides a computer program product or computer program, including computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing described in Embodiment 1 above.

[0217] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0218] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0219] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0220] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0221] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0222] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for cross-modal retrieval of multimedia data based on multi-label incremental hashing, characterized in that, include: Preprocessing is performed based on online multimedia data streams to extract newly added multimodal combination data and adaptive code length in the current round, including: The online multimedia data stream is acquired and divided into new multimodal data blocks according to rounds. Feature extraction is performed on the new multimodal data blocks to obtain the new feature matrix and new label matrix of each modality in the current round. The cumulative number of categories in the current round is determined based on the newly added categories in the data tag matrix of the current round and the cumulative number of categories in the previous round. The adaptive code length of the current round is determined with a constraint that it is not less than the cumulative number of categories in the current round and is a power of 2. The multimedia data stream includes image-text pairs. The historical hash center is expanded based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and a global hash center is constructed based on the newly added hash center of the current round and the expanded historical hash center. A global index matrix is ​​constructed based on the newly added multimodal combination data in the current round, and the initial global hash code is obtained by multiplying the global index matrix and the global hash center. The initial global hash code is iteratively optimized by alternating the objective function of hash code optimization to obtain the optimal newly added hash code. The optimal newly added hash code and historical hash codes are then merged to obtain the global hash code, which includes: Initialize based on the adaptive code length and initial global hash code of the current round to obtain the hash code to be optimized and the historical hash code of the newly added data; By combining global hash center constraints and local semantic similarity, a hash code optimization objective function is constructed. With the hash code to be optimized fixed, the first auxiliary variable is introduced to solve the hash code optimization objective function, thus obtaining the current round. The Middle The intermediate variables after alternation optimization include: With the hash code to be optimized fixed, calculate the current round's adaptive code length, the hash code to be optimized, and the newly added label matrix based on the first auxiliary variable after the last alternating optimization in the previous round, the hash code to be optimized, and the newly added label matrix. The Middle The first auxiliary variable after alternation optimization The calculation formula is as follows: in, It is a newly added tag matrix 2. Normalization function, For the current round New label matrix for each modality, The current round The Middle The first auxiliary variable after alternation optimization. The current round The Middle The newly added hash code after alternation optimization. , It is a specified threshold for the number of alternating optimization iterations. It is the adaptive code length for the current round. It was the previous round. Historical adaptive code length; Initialize intermediate variables based on the current round. The Middle The optimal intermediate variable is determined by the first auxiliary variable after alternating optimization, the newly added label matrix, and the adaptive code length. Fix the current round The Middle After alternating optimization, intermediate variables are introduced as a second auxiliary variable to solve the hash code optimization objective function, thus obtaining the current round. The Middle The newly added hash codes after this alternation optimization include: Fix the current round The Middle After the optimal intermediate variable following the alternation optimization, the current round's adaptive code length, optimal intermediate variable, and newly added tag matrix are used to calculate the optimal intermediate variable after the second auxiliary variable following the last alternation optimization in the previous round. The Middle The second auxiliary variable after alternation optimization is calculated using the following formula: in, The current round The Middle The second auxiliary variable after alternation optimization The current round The Middle The optimal intermediate variable after alternating optimization; Based on the current round The Middle The second auxiliary variable after alternation optimization, the newly added index matrix, the newly added label matrix, and the global hash center are used to determine the current round. The Middle After the alternation optimization, a new hash code is added. ; The process continues until the number of alternating optimization iterations reaches a specified threshold, yielding the optimal newly added hash code. This optimal newly added hash code is then merged with historical hash codes to obtain the global hash code. ; With the objective of minimizing the error between the global hash code and modal features, a linear regression model is constructed. The linear regression model is solved based on the global hash code to obtain the projection matrices for each modality, including: A linear regression model is constructed with the goal of minimizing the error between the global hash code and the modal features. Based on the global hash code and the third auxiliary variable from the previous round and the fourth auxiliary variable Calculate the newly added feature matrix for the current round, and then calculate the third auxiliary variable for the current round. and the fourth auxiliary variable The calculation formula is as follows: in, For the first The newly added feature matrix of the m-th mode in round m; Based on the third and fourth auxiliary variables of the current round, the projection matrix of each modality is obtained. The formula is as follows: ； The retrieval results are obtained by encoding the sample based on the projection matrix of the modality corresponding to the sample.

2. The multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in claim 1, characterized in that, The step of expanding the historical hash center based on the relationship between the current round's adaptive code length and the historical adaptive code length, and constructing a global hash center based on the newly added hash center in the current round and the expanded historical hash center, includes: Generate a Hadamard matrix based on the adaptive code length of the current round, and extract the newly added hash center of the current round from the Hadamard matrix; If the adaptive code length of the current round is equal to the historical adaptive code length, then the historical hash center remains unchanged; If the adaptive code length of the current round is equal to twice the historical adaptive code length, then the historical hash center will be self-replicated to obtain the expanded historical hash center. A global hash center is constructed based on the newly added hash centers in the current round and the expanded historical hash centers.

3. The multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in claim 1, characterized in that, The process of constructing a global index matrix based on newly added multimodal combination data in the current round, and obtaining the initial global hash code based on the product of the global index matrix and the global hash center, includes: A new index matrix is ​​obtained by clustering the newly added label matrix in the newly added multimodal combined data of the current round; The historical global index matrix of the previous round is expanded to obtain the expanded historical global index matrix. The global index matrix is ​​constructed based on the expanded historical global index matrix and the newly added index matrix. The initial global hash code is obtained by multiplying the global index matrix and the global hash center.

4. A multimedia data cross-modal retrieval system based on multi-label incremental hashing, implemented using the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in any one of claims 1-3, characterized in that, include: The data acquisition and processing module is used to preprocess online multimedia data streams and extract newly added multimodal combination data and adaptive code length in the current round. The global hash center generation module is used to expand the historical hash center based on the relationship between the adaptive code length of the current round and the historical adaptive code length, and to construct the global hash center based on the newly added hash center of the current round and the expanded historical hash center. The initial global hash code construction module is used to construct a global index matrix based on the newly added multimodal combination data in the current round, and obtain the initial global hash code based on the product of the global index matrix and the global hash center. The global hash code construction module is used to iteratively optimize the initial global hash code based on the hash code optimization objective function to obtain the optimal newly added hash code. The optimal newly added hash code and the historical hash codes are then merged to obtain the global hash code. The hash learning module is used to construct a linear regression model with the goal of minimizing the error between the global hash code and the modal features. It solves the linear regression model based on the global hash code to obtain the projection matrix of each modality. The cross-modal retrieval module is used to encode the sample to be retrieved based on the projection matrix of the modality corresponding to the sample to be retrieved, and obtain the retrieval results.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in any one of claims 1-3.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in any one of claims 1-3.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps in the multimedia data cross-modal retrieval method based on multi-label incremental hashing as described in any one of claims 1-3.

Images