Text-image cross-modal hash retrieval model training method based on discriminative generative sampling

Abstract

The present application relates to the technical field of information retrieval, and in particular to a picture-text cross-modal hash retrieval model training method based on discriminative generated sampling. Through hash feature scaling and hash concept offset, discriminative pseudo-embedding is directly generated in the embedding space, effectively solving the problem of insufficient training samples caused by sparse embedding area. This generation method not only significantly increases the number of training samples, but more importantly, ensures the discriminativeness and semantic rationality of the generated samples, providing more abundant training signals for hash learning. In addition, without changing the existing network structure, the computational overhead is much lower than that of traditional generative data enhancement methods, and it can be seamlessly integrated into various deep hash frameworks and adapted to different loss functions and sampling strategies.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology, and more specifically to a training method for a cross-modal hash retrieval model of text and image based on discriminative generation sampling. Background Technology

[0002] Cross-modal retrieval targets multimedia data such as images, text, audio, and video, projecting heterogeneous samples semantically to appropriate positions in the joint embedding space to achieve a near-nearest neighbor retrieval experience. Deep hashing cross-modal retrieval restricts the embedding space and embeddings to Hamming space and hash codes, significantly reducing storage costs, improving retrieval speed, and offering advantages in more application scenarios. Similar to traditional hashing methods, it projects heterogeneous data into a common Hamming space to achieve multimodal semantic alignment, relying on a suitable loss function for training to ensure that semantically similar embeddings are close to each other in the space.

[0003] However, such methods rely on embedding quality; a lack of discriminative embedding distributions contributes little or nothing to training, and can even lead to model collapse. Embeddings with rich and stable sampling information are crucial for promoting training in the embedding space, thus leading to two paths: designing sampling methods that better fit the loss function and providing more discriminative embeddings.

[0004] Sampling methods, as a crucial step in sample selection, have received significant attention. While progress has been made in sampling methods for single-batch embeddings, the size of a single batch is heavily influenced by physical devices due to GPU memory limitations. Smaller batches mean fewer samples, thus impacting sampling effectiveness. Furthermore, embeddings within a single batch are prone to sparse embedding due to uneven distribution, forcing sampling methods to select suboptimal candidates. This reduces the discriminative power of the embedding space and cross-modal alignment, leading to sparse regions eroding semantic distinguishability, widening modal gaps, and affecting retrieval performance. In recent years, pseudo-embedding generation methods have attempted to overcome these bottlenecks, but either introduce significant training overhead or lack general applicability. Summary of the Invention

[0005] To address at least one problem in the prior art, the present invention provides a training method for a cross-modal hash retrieval model of text and image based on discriminative generation sampling.

[0006] The first aspect of this invention provides a method for training a cross-modal hash retrieval model for images and text based on discriminative generation sampling, which includes the following steps:

[0007] Obtain a labeled image-text dataset, which includes image samples and their labels, and text samples and their labels;

[0008] The image features of the image samples and the text features of the text samples are extracted separately and mapped to image embedding vectors and text embedding vectors, respectively.

[0009] Generate image pseudo-embedded vectors or text pseudo-embedded vectors based on the high activation features in the image embedding vectors or text embedding vectors;

[0010] A cross-modal hash retrieval model for images and text is trained using image pseudo-embedding vectors and text pseudo-embedding vectors.

[0011] Update the parameters of the image-text cross-modal hash retrieval model based on the target loss.

[0012] In this invention, discriminative pseudo-embeddings are directly generated in the embedding space based on the high activation features in the image-text embedding vector, which can solve the problem of insufficient training samples caused by the sparsity of the embedding region.

[0013] In some optional embodiments, the image features of image samples and the text features of text samples are extracted separately using the CLIP model.

[0014] In some optional embodiments, the aforementioned high activation feature is the largest value among the image embedding vectors or text embedding vectors. Features in 1 dimension, among which It is a positive integer.

[0015] In some optional embodiments, the above-described generation of image pseudo-embedded vectors or text pseudo-embedded vectors based on high activation features in image embedding vectors or text embedding vectors includes:

[0016] Establish a frequency recording matrix;

[0017] Identify the top value in the image embedding vector or text embedding vector. Each dimension corresponds to a category in the frequency record matrix. Accumulate and count across each dimension;

[0018] Generate a binary mask for each category based on the frequency record matrix;

[0019] By randomly scaling the features in the image embedding vector or text embedding vector based on the binary mask, a pseudo-embedded vector for the image or text is obtained.

[0020] In some optional embodiments, the above-described method for training a cross-modal hash retrieval model based on discriminative generation sampling further includes the following steps:

[0021] For image embedding vectors or text embedding vectors with multiple hot labels, calculate the label semantic difference vector and embedding difference vector between sample pairs;

[0022] Valid embedding difference vectors are selected based on constraints.

[0023] Normalize the effective embedding difference vector and multiply it by a scaling factor to obtain the image pseudo-embedded vector or the text pseudo-embedded vector.

[0024] In some alternative embodiments, the above constraints include nonnegativity constraints, nonzero constraints, and nonidentity constraints.

[0025] In some optional embodiments, the target loss described above includes a spacing-based loss, wherein the spacing-based loss is calculated by the following formula:

[0026] ;

[0027] in, Indicates sample and The label relationship between them, when The value is 1 when the time is right, and 0 otherwise. Indicates sample and The distance between them; and These represent the hyperparameters required to control the loss spacing.

[0028] A second aspect of the present invention provides a cross-modal hash retrieval method for images and text, comprising the following steps:

[0029] Extract image features from the image to be retrieved or text features from the text to be retrieved, and map them into image embedding vectors or text embedding vectors;

[0030] Input the image embedding vector or text embedding vector into the image-text cross-modal hash retrieval model, and the image-text cross-modal hash retrieval model outputs the retrieval results;

[0031] The image-text cross-modal hash retrieval model is obtained by the above-mentioned image-text cross-modal hash retrieval model training method based on discriminative generation sampling.

[0032] A third aspect of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the above-described training method for a cross-modal hash retrieval model based on discriminative generation sampling according to the computer program.

[0033] A fourth aspect of the present invention provides a computer-readable storage medium for storing a computer program; the computer program, when executed by a processor, implements the above-described training method for a cross-modal hash retrieval model based on discriminative generation sampling.

[0034] The technical solution of this invention has the following beneficial effects: by scaling hash features and shifting hash concepts, discriminative pseudo-embeddings are directly generated in the embedding space, effectively solving the problem of insufficient training samples caused by the sparsity of the embedding region; this generation method not only significantly increases the number of training samples, but more importantly, ensures the discriminativeness and semantic rationality of the generated samples, providing richer training signals for hash learning; in addition, it does not require changing the existing network structure, the computational overhead is far lower than that of traditional generative data augmentation methods, it can be seamlessly integrated into various deep hashing frameworks, and it can adapt to different loss functions and sampling strategies. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the network structure of the image-text cross-modal hash retrieval model in an embodiment of the present invention.

[0036] Figure 2 The figure shows the comparison results of the training time and encoding time of different models on the MS COCO dataset in the embodiments of the present invention. (a) is a comparison of training time (in hours), and (b) is a comparison of encoding time (in milliseconds).

[0037] Figure 3 The images show the mAP@all results of different models with or without the DGS module on the MIRFLICKR-25K dataset in this embodiment of the invention. (a) is the mAP@all result of the I2T task, and (b) is the mAP@all result of the T2I task.

[0038] Figure 4 Different parameters are used in the embodiments of the present invention. The model's mAP@all results on the MS COCO dataset are shown in (a) and (b) respectively.

[0039] Figure 5 The images shown are mAP@all results on the MS COCO dataset for models where the DGS module is replaced with GAN and MIX respectively in this embodiment of the invention. (a) is the mAP@all result for the I2T task, and (b) is the mAP@all result for the T2I task.

[0040] Figure 6 The images show the 16-bit mAP@all results of models trained with training sets of different sizes on the MS COCO dataset in this embodiment of the invention. (a) is the mAP@all result of the I2T task, and (b) is the mAP@all result of the T2I task. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0042] In hash learning, let and Based on the dataset samples, respectively, by hash function and The length learned is The characteristic code of (hash bits), where and These represent the image modality and the text modality, respectively. These samples are formatted with a length equal to the number of categories. Multiple hot codes Labeling. The goal of hash learning is to make semantically similar samples appear feature-similar, i.e., they lie in the joint embedding space. ( The projection positions in the samples are similar, and vice versa. This learning method relies on sample pairs. and The choice of signatures. However, the number of signatures is limited by a single mini-batch. The size of the sparse and inefficient relation learning means that all pairwise relations learned in a single session can only be generated from these samples. This sparse and inefficient relation learning impairs the discriminativeness of the learned cospace.

[0043] To address this, this invention proposes a training method for a cross-modal hash retrieval model based on discriminative generative sampling (DGS). By introducing a discriminative generative sampling (DGS) mechanism, discriminative pseudo-embedding vectors are generated without altering the network structure, expanding the sample distribution in sparse regions and thus improving the discriminative power of hash learning and cross-modal retrieval performance. This is achieved by generating discriminative pseudo-embedding vectors. , and its corresponding generated tags ,Will Quantity of feature codes extended To obtain a more discriminative embedding space Applying the trained image-text cross-modal hash retrieval model to image-text cross-modal hash retrieval can achieve superior retrieval performance.

[0044] Specifically, the image-text cross-modal hash retrieval model (denoted as DDGSH) in this embodiment of the invention includes a feature extraction network and a discriminative generative sampling (DGS) module. In the discriminative generative sampling, data augmentation is performed through the hash feature scaling (HFS) submodule and the hash concept offset (HCS) submodule to generate discriminative pseudo-embedding vectors, thereby densifying the embedding space and improving the discriminative power and cross-modal alignment performance of the hash code.

[0045] The feature extraction network can be a two-stream architecture backbone network based on CLIP (ViT-B / 32). CLIP is a classic model for cross-modal learning and a major pillar of current cross-modal hash retrieval efforts. CLIP uses Vision Transformer and Transformer as image and text encoders, respectively. Each encoder contains 12 encoding blocks, consisting of multi-head self-attention and multilayer perceptrons. Image modality samples are processed by the Vision Transformer encoder, while text modality samples undergo feature extraction using the Transformer encoder. The function representation of each layer is as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] in, and These represent the features of the image modality and the text modality, respectively. and These represent the intermediate vectors respectively. Indicates the coded block number; and These represent multi-head self-attention and multilayer perceptron, respectively. Representation layer normalization. For entering... The input of the layer (i.e., the first layer) Layer output and The output of this layer is obtained by passing the data through a multi-head self-attention layer with residual connections, and then through a multilayer perceptron with residual connections. and After 12 layers of image and text encoding blocks, the original semantics of the image and text are extracted into the final high-dimensional features. and In the middle. Then, the hash layer is responsible for... and Mapped to Feature code within a continuous embedding space of bit hash code length and Since binary optimization is an extremely difficult problem in model training, continuous hash codes are used for the training features, and then binarized again during the model testing phase to obtain the true hash codes. To utilize a continuous relaxation strategy similar to binary optimization during training, a method is introduced into the hash layer. The activation function causes the hash layer to output continuous binary-like code. Similar to the two-stream architecture, image features and text features are hashed at the same scale, as shown in the following equation:

[0051] ；

[0052] ；

[0053] in, and These represent the weight parameters of the image hash layer and the text hash layer, respectively. and These represent the corresponding bias parameters.

[0054] Due to the sparsity of the original embedding space, it is difficult to obtain a discriminative hash space by training only these real samples. Therefore, this embodiment of the invention uses a discriminative generation sampling method, takes the embeddings learned from real samples as sampling anchors, and uses two sub-modules, hash feature scaling and hash concept offset, to obtain pseudo-embeddedness that is discriminative in itself, without the need for additional models or training.

[0055] The hash feature scaling submodule aims to generate semantically enhanced or weakened variants around the original samples by randomly scaling the highly activated, discriminative feature dimensions in a single embedding vector. First, a learnable frequency record matrix is ​​constructed. ,in Indicates the total number of categories. This indicates the hash code length. The frequency record matrix is ​​initialized to a zero matrix. For each training sample... and its labels Identify the first element with the largest value in its embedding vector. Each dimension (i.e., the most active feature) is recorded in the frequency matrix. Corresponding category The count is accumulated across these dimensions. Frequency record matrix. The specific update rules are shown in the following formula. For simplicity, the following parts will directly use the formula. Represents its embedding vector :

[0056]

[0057] This formula is used to update the frequency recording matrix. The middle category is The number of characteristic bits is the first The number of times the bit is activated. Among them, Indicates a dimension index. express Center front A set of dimension indices with a maximum value. The number of bits considered to be the most representative in the embedding vector, in the embodiments of this invention... It is 8.

[0058] Then, based on the updated frequency record matrix Generate a binary mask matrix for each category. This identifies the discriminative feature dimension of the category. The binary mask is created by selecting a frequency record matrix. The most frequent front The result is obtained by setting one dimension to 1 and the rest to 0, as shown in the following formula:

[0059]

[0060] This formula is used to finally determine the number of characteristic bits. Is the bit a category? The highly activated feature bits. Indicates for category Recorded before The set of feature dimension indices for the maximum value. The final matrix. Recorded the previous records for each category The index containing the high-active bit.

[0061] Given a true embedding vector and its labels First, based on its category mask Determine the dimensions that need to be scaled; then, generate a scaling factor vector. As shown in the following formula:

[0062] ;

[0063] in, , represents a random vector that is independently and uniformly sampled in each dimension; θ represents the hyperparameter controlling the scaling range; 1 hθ represents a vector of all 1s; ⊙ represents element-wise multiplication. In this embodiment of the invention, θ = 0.1, meaning the features are randomly scaled within the range of (0.9, 1.1). This operation only randomly scales the discriminative feature dimension, while the non-discriminative dimension remains unchanged, thereby generating new discriminative samples while preserving the main semantics of the samples.

[0064] Get scaling factor Then, the new embedding after random scaling is obtained through bitwise multiplication, as shown in the following equation:

[0065] ;

[0066] Newly acquired Based on By scaling the features of key discriminant bits, a new pseudo-embeddedness is obtained that is semantically similar to the original embedding, and its corresponding label is similar to the original embedding. Same, also for .

[0067] The hash concept offset submodule synthesizes discriminative pseudo-embeddedness through semantic transformations between multiple embeddings. This submodule operates via concept difference propagation to generate pseudo-embeddedness vectors with semantic concept offsets. This is particularly relevant for multi-hot tags. batch embedding The semantic difference in labels and the embedding difference vector between sample pairs are calculated as shown in the following formula:

[0068] ;

[0069] ;

[0070] in, Represent two vectors and The difference, Indicates the corresponding tag and Interpolation.

[0071] To ensure that the generated transformation has a clear semantic meaning, three constraints are imposed on the calculated difference pairs: (1) Non-negativity constraint, ensuring that the semantic transformation is an addition rather than a removal, i.e. (2) Non-zero constraints ensure the existence of substantial semantic changes, i.e. (3) Non-identity constraint, excluding invalid cases of subtraction of itself, i.e. Here This represents each dimension of the vector. For valid difference pairs that satisfy the constraints, the difference in their embedding vectors is... Perform normalization and multiply by a controllable scaling factor. Finally, pseudo-embedding vectors representing specific semantic transformation directions and their corresponding pseudo-labels are generated, as shown in the following equation:

[0072] ;

[0073] ;

[0074] This mechanism can effectively generate new sample points along the semantic relationship direction in the embedding space, thereby enriching the semantic diversity of the samples. In Corresponding to anchor point embedding Subtraction Embedding And with scaling.

[0075] The effect of the discriminant generation sampling module (DGS) in this embodiment of the invention can be considered as:

[0076] ;

[0077] Here This represents all new samples generated by hash feature scaling or hash concept offset. This is achieved by generating pseudo-embedding vectors. and its corresponding pseudo-tags This is used to further train the hash layer, thereby improving the samples used for training the actual model. Through two sub-modules, hash feature scaling and hash concept shifting, discriminative generated pseudo-embeddings can be effectively provided for hash retrieval training, making the learned cross-modal hash model more discriminative.

[0078] During the testing phase, use The function binarizes the feature code to obtain the hash code, as shown in the following formula:

[0079] ;

[0080] ;

[0081] in, This function represents the sign function, outputting 1 when the input is greater than 0, and -1 otherwise. It is used to generate consecutive hash codes. , Convert to true binary hash code , .

[0082] In image-text cross-modal hash retrieval, the query sample (image or text) is first input into the trained image-text cross-modal hash retrieval model. Through the processing of a feature extraction network and a hash layer, the high-dimensional features of the query sample are converted into low-dimensional continuous binary-like codes. Then, a sign function is applied. The encoding is discretized to generate the final binary hash code. Similarity is measured by calculating the Hamming distance between the query sample's hash code and the hash codes of all samples in the target modality database. The Hamming distance is implemented using an XOR operation and bit counting. Finally, the search results are returned, returning the samples closest to the query sample in the Hamming space, thus completing the cross-modal similarity retrieval task.

[0083] To verify the effectiveness of the image-text cross-modal hash retrieval model (DDGSH) in this embodiment of the invention, performance verification experiments were conducted on three public datasets: MIRFLICKR-25K, NUS-WIDE, and MS COCO. The model was compared with current mainstream image-text cross-modal hash retrieval models: DSPH, TwDH, DNPH, DNpH, DHaPH, BiLGSEH, and DECH. In the experiments, the three datasets were re-split, with 10,000 image-text pairs used as the training set for each dataset. During the model validation phase, 5,000 pairs were used as the query set, and the remaining image-text pairs were used as the database set.

[0084] During the training phase, 256 pairs of samples were used in each batch, and the best value from 100 training rounds was selected. To comprehensively evaluate the retrieval performance of DDGSH, mAP@all (mean precision) was used as the training metric. The performance comparison results for the two tasks of image retrieval of text (I2T) and text retrieval of images (T2I) are shown in Tables 1-3.

[0085] Table 1. Performance comparison of different models on the MIRFLICKR-25K dataset using mAP@all

[0086]

[0087] Table 2. Performance comparison of different models on the NUS-WIDE dataset using mAP@all

[0088]

[0089] Table 3. Performance comparison of different models on the MSCOCO dataset using mAP@all

[0090]

[0091] The efficiency of DDGSH is intuitively verified by statistically analyzing the time required for 100 training rounds (in hours, h) and the time required for a single inference encoding (in milliseconds, ms). The results are as follows: Figure 2 As shown in the figure. Experiments demonstrate that, compared to the complex algorithm implementations of existing models, the DDGSH implementation of this invention achieves efficient hash retrieval through a concise and clear design approach.

[0092] Furthermore, to rigorously evaluate the generalization ability of DDGSH beyond specific optimization configurations, three classic losses in pairwise learning (contrastive loss, triplet loss, and margin loss), along with their most widely used sampling methods (uniform sampling, semi-hard sampling, and distance-weighted sampling), were selected for decoupled modeling experiments. These experiments validated the generalization ability of the proposed Discriminative Generative Sampling (DGS) module. As shown in Tables 4-6 (√ indicates that DGS was used), the results demonstrate that the discriminative generative sampling module of this embodiment provides consistent performance improvements regardless of the underlying optimization design. This orthogonal experiment confirms that the module's effectiveness stems from its core ability to enhance discriminative embeddings, rather than from synergistic effects with any specific loss sampling combination.

[0093] Table 4. Decoupling experiment results of DGS on the MIRFLICKR-25K dataset.

[0094]

[0095] Table 5. Decoupling Experiment Results of DGS on the NUS-WIDE Dataset

[0096]

[0097] Table 6. Decoupling Experiment Results of DGS on the MSCOCO Dataset

[0098]

[0099] To further verify the generalization ability of the DGS module in producing effects not limited to pairwise learning, the DGS module was added to DHaPH and DDBH. DHaPH with the DGS module added was labeled DHaPH w / DGS, and DDBH with the DGS module added was labeled DDBH w / DGS. Experiments were conducted on the MIRFLICKR-25K dataset. The results are as follows: Figure 3 As shown, both DHaPH and DDBH emphasize the discriminativeness of the learned hash codes, but the former uses hierarchical proxies to obtain more refined semantics, while the latter achieves this goal through boundary preservation and class proxies, indicating that the DGS module of this embodiment has generalizability.

[0100] Furthermore, the ablation effects of the Hash Feature Scaling (HFS) and Hash Concept Offset (HCS) submodules in the discriminative generation sampling module were verified, and the results are shown in Table 7 (√ in the table indicates that the HFS or HCS submodules were used). The ablation experiments demonstrate that both the Hash Feature Scaling and Hash Concept Offset submodules can independently enhance the discriminability of the hash space, and their combined use yields the best results.

[0101] Table 7. Ablation experiments on the MIRFLICKR-25K dataset using the hash feature scaling submodule and the hash concept offset submodule.

[0102]

[0103] Parameters in the hash feature scaling submodule This is considered to be the most discriminative bit in a hash code, but its value can vary with the hash code length. Parametric experiments were conducted on the MS COCO dataset to investigate this, and the results are as follows: Figure 4 As shown, the global optimum is located at the top of the medium-sized area. The ratio is moderate, with four local optima occurring at the extremes of the parameters. Specifically, while dimension-specific optima require precise tuning, perturbations of approximately 8 bits yield competitive results across all dimensions. Therefore, for simplicity, =8 is more appropriate.

[0104] To verify the effectiveness of the DGS module in the generative method of this invention, comparative experiments were conducted by replacing the DGS module with Generative Adversarial Network (GAN) and Hybrid Interpolation Synthesis (MIX), respectively. The results are as follows: Figure 5 As shown. Meanwhile, to verify the effectiveness of the DGS module in handling extreme scenarios, the training set size was reduced to 2500 pairs, 5000 pairs, and 7500 pairs respectively, and the changes in retrieval performance were observed. The results are as follows. Figure 6 As shown in the figure. Experiments demonstrate that the DGS module can guarantee the discriminativeness and semantic rationality of the generated samples, and maintains relatively stable performance even when dealing with a decreasing number of real samples.

[0105] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any form or substance. It should be noted that those skilled in the art can make various improvements and additions without departing from the method of the present invention, and these improvements and additions should also be considered within the scope of protection of the present invention. Any modifications, alterations, and equivalent changes made by those skilled in the art based on the above-disclosed technical content without departing from the spirit and scope of the present invention are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, and evolutions made to the above embodiments based on the essential technology of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for training a cross-modal hashing retrieval model based on discriminative generative sampling, the method comprising: The method comprises the following steps: obtaining a labeled image-text dataset, wherein the image-text dataset comprises image samples and labels thereof and text samples and labels thereof; extracting image features of the image samples and text features of the text samples respectively, and mapping the image features and the text features into image embedding vectors and text embedding vectors respectively; generating image pseudo embedding vectors or text pseudo embedding vectors according to high activation features in the image embedding vectors or the text embedding vectors; training an image-text cross-modal hash retrieval model by using the image pseudo embedding vectors and the text pseudo embedding vectors; updating parameters of the image-text cross-modal hash retrieval model according to a target loss; The high activation feature is a feature with a maximum value in the first K dimensions of an image embedding vector or a text embedding vector. is a positive integer.​ the generating of the image pseudo embedding vectors or the text pseudo embedding vectors according to the high activation features in the image embedding vectors or the text embedding vectors comprises: establishing a frequency record matrix; The top dimensions with the largest values in the image embedding vectors or the text embedding vectors are identified, and the corresponding categories in the frequency recording matrix are accumulated in the top dimensions. generating a binary mask for each category according to the frequency record matrix; randomly scaling features in the image embedding vectors or the text embedding vectors according to the binary mask to obtain the image pseudo embedding vectors or the text pseudo embedding vectors; wherein, for the image embedding vectors or the text embedding vectors with multiple hot labels, a label semantic difference vector and an embedding difference vector between sample pairs are calculated; screening effective embedding difference vectors according to constraint conditions, wherein the constraint conditions comprise non-negative constraint, non-zero constraint and non-identity constraint; normalizing the effective embedding difference vectors and multiplying the effective embedding difference vectors by a scaling factor to obtain the image pseudo embedding vectors or the text pseudo embedding vectors.

2. The method of claim 1, wherein, The image features of the image samples and the text features of the text samples are extracted by using a CLIP model.

3. The method of claim 1, wherein, The target loss comprises a distance-based loss, wherein the distance-based loss is calculated by the following formula: ； wherein, represents a label relation between samples and , which is 1 when , otherwise 0; represents a distance between samples and ; and respectively represent hyperparameters controlling the loss margin requirement.

4. A method for cross-modal hash retrieval of images and texts, characterized in that, The method comprises the following steps: extracting image features of a to-be-retrieved image or text features of a to-be-retrieved text, and mapping the image features or the text features into image embedding vectors or text embedding vectors; inputting the image embedding vectors or the text embedding vectors into the image-text cross-modal hash retrieval model, and outputting retrieval results by the image-text cross-modal hash retrieval model; The image-text cross-modal hash retrieval model is obtained by using the image-text cross-modal hash retrieval model training method based on discriminative generative sampling according to any one of claims 1-3.

5. An electronic device, comprising: The device comprises a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the image-text cross-modal hash retrieval model training method based on discriminative generative sampling according to any one of claims 1-3 according to the computer program.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store a computer program; and the computer program is used to implement the image-text cross-modal hash retrieval model training method based on discriminative generative sampling according to any one of claims 1-3 when executed by a processor.

Images