Data Retrieval Methods and Systems Based on Labeled Sample Augmentation and Sample Selection

Abstract

This disclosure provides a data retrieval method and system based on labeled sample augmentation and sample selection, belonging to the field of data mining and processing technology. The method includes acquiring image samples to be retrieved, calculating their hash codes using an out-of-sample extension mapping; calculating the Hamming distance between the image samples to be retrieved and the hash codes in the retrieval database, and returning similar data in ascending order of Hamming distance; establishing a supervised loss based on labeled samples and their labels, and an unsupervised loss based on unlabeled samples and their pseudo-labels; establishing a semi-supervised hashing objective function based on the supervised and unsupervised losses; iterating between objective function optimization and sample selection to obtain the hash codes in the retrieval database, and using the hash codes in the retrieval database to learn the out-of-sample extension mapping. This method can improve the training efficiency and stability of the semi-supervised hashing method to a certain extent.

Description

Technical Field

[0001] This disclosure relates to the field of data mining and processing technology, specifically to an image data retrieval method and system based on labeled sample augmentation and sample selection. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In the era of big data, people's daily work and communication are increasingly inseparable from massive amounts of image data. Simultaneously, image retrieval technology is being widely applied in various fields. As image size grows exponentially and image dimensionality increases, traditional image retrieval methods face challenges such as large storage requirements and slow retrieval speeds. Among numerous retrieval methods, hash-based retrieval methods not only demonstrate superiority in feature learning but also offer advantages such as small storage space and fast retrieval speed, thus attracting considerable attention.

[0004] Among existing hash-based retrieval methods, supervised hashing methods are the most effective. These methods rely on a large number of labeled samples, using the semantic information of their labels to guide the learning of hash codes and hash functions. However, manual labeling is time-consuming and laborious, labeled samples are difficult to obtain, and a large amount of readily available data is unlabeled. Therefore, research on semi-supervised hashing methods has significant practical implications.

[0005] To explore the potential semantic information in unlabeled samples, some semi-supervised hashing methods based on labeled sample augmentation have been proposed. These methods typically assign pseudo-labels to unlabeled samples based on some labeled sample augmentation strategy, and select unlabeled samples with high prediction confidence from them to guide the hash learning process together with labeled samples.

[0006] However, existing semi-supervised hashing methods based on labeled sample augmentation have the following problems:

[0007] (1) The sample selection did not take into account the difference in prediction difficulty of unlabeled samples and ignored the problem of pseudo-label noise.

[0008] (2) During the iterative process between model optimization and sample selection, the problem of the model's prediction of unlabeled samples changing over time was ignored, which led to slow training and unstable performance. Summary of the Invention

[0009] To address the aforementioned issues, this disclosure proposes a data retrieval method and system based on labeled sample augmentation and sample selection. It employs a labeled sample augmentation strategy to assign pseudo-labels to unlabeled samples and selects unlabeled samples using a self-learning framework. These unlabeled samples, along with labeled samples, guide the hash learning process, thereby filtering feature noise and pseudo-label noise to a certain extent and improving the accuracy of the semi-supervised hashing method.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions:

[0011] Data retrieval methods based on labeled sample augmentation and sample selection include:

[0012] Obtain the image sample to be retrieved and calculate its hash code using the out-of-sample extended mapping.

[0013] Calculate the Hamming distance between the image sample to be retrieved and the hash code in the retrieval database, and return similar data in ascending order of Hamming distance;

[0014] Among them, pseudo-labels are applied to unlabeled samples based on a labeled sample augmentation strategy;

[0015] A supervised loss is established based on labeled samples and their labels, an unsupervised loss is established based on unlabeled samples and their pseudo-labels, and a semi-supervised hash objective function is established based on the supervised loss and the unsupervised loss.

[0016] The algorithm iterates between objective function optimization and sample selection to obtain the hash code of the retrieval database, and uses the hash code of the retrieval database to learn the sample extra-sample extension mapping.

[0017] According to other embodiments, the present disclosure adopts the following technical solutions:

[0018] Data retrieval systems based on labeled sample augmentation and sample selection include:

[0019] The hash code calculation module is used to obtain the image sample to be retrieved and calculate its hash code using the out-of-sample extended mapping.

[0020] The online retrieval module is used to calculate the Hamming distance between the image sample to be retrieved and the hash code in the retrieval database, and return similar data in ascending order of Hamming distance;

[0021] Among them, pseudo-labels are applied to unlabeled samples based on a labeled sample augmentation strategy;

[0022] A supervised loss is established based on labeled samples and their labels, an unsupervised loss is established based on unlabeled samples and their pseudo-labels, and a semi-supervised hash objective function is established based on the supervised loss and the unsupervised loss.

[0023] The algorithm iterates between objective function optimization and sample selection to obtain the hash code of the retrieval database, and uses the hash code of the retrieval database to learn the sample extra-sample extension mapping.

[0024] According to other embodiments, this disclosure also adopts the following technical solutions:

[0025] A computer-readable storage medium having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the steps of the data retrieval method based on labeled sample augmentation and sample selection.

[0026] According to other embodiments, this disclosure also adopts the following technical solutions:

[0027] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the program, implements the steps of the data retrieval method based on labeled sample augmentation and sample selection.

[0028] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0029] This disclosure provides a sample selection strategy that is aware of the difficulty of prediction, which filters feature noise and pseudo-label noise to a certain extent, thereby improving the accuracy of semi-supervised hashing methods.

[0030] This disclosure provides a sample selection strategy that is consistent with the prediction time, which to some extent improves the training efficiency and stability of semi-supervised hashing methods. Attached Figure Description

[0031] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0032] Figure 1 This is a flowchart of the present invention;

[0033] Figure 2 This is a schematic diagram of the present invention. Detailed implementation method:

[0034] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0036] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0037] The data involved in this disclosure is not limited to image data, but may also include text data, audio data, etc.

[0038] The following section uses image data as an example to explain in detail the principle of this invention and its specific implementation process.

[0039] Example 1

[0040] One embodiment of this disclosure provides a data retrieval method based on labeled sample augmentation and sample selection, such as... Figure 1 and Figure 2 ,include:

[0041] S101: Obtain the image sample to be retrieved and calculate its hash code using the out-of-sample extended mapping;

[0042] For image sample x, the hash code h = sign(Px) is calculated using the out-of-sample extension mapping P; where the sign function is a quantization function, which aims to quantize the real-valued vector obtained by hash projection into discrete binary code for subsequent retrieval.

[0043] S102: Calculate the Hamming distance between the hash code of the sample to be retrieved and the hash codes in the retrieval database, and return similar data in ascending order of Hamming distance.

[0044] The search database was built offline.

[0045] Specifically, as one embodiment, the process of constructing the retrieval database is as follows:

[0046] Step a: Apply pseudo-labels to unlabeled samples based on a labeled sample augmentation strategy.

[0047] Specifically, in step a, the labeled sample augmentation strategy is as follows:

[0048] First, for each labeled sample, the bi-kmeans algorithm is used to determine the cluster center, which serves as the centroid of that sample class. Then, these centroids are used as anchor points, and the local anchor embedding algorithm is used to obtain the reconstruction weight coefficients for the unlabeled samples. Finally, the reconstruction weight coefficients are used to weight the corresponding categories to obtain the pseudo-labels for the unlabeled samples.

[0049] Step b: Establish a supervised loss based on labeled samples and their labels, establish an unsupervised loss based on unlabeled samples and their pseudo-labels, and establish a semi-supervised hash objective function based on the supervised loss and the unsupervised loss.

[0050] Specifically, in step b, the objective function of the semi-supervised hashing consists of supervised loss, unsupervised loss, and difference loss between the supervised and unsupervised hashing models. The supervised loss comprises pairwise similarity embedding loss, label embedding loss, and quantization loss based on hash projection, while the unsupervised loss comprises sample selection loss and self-stepping regularization loss.

[0051] Specifically, as an example, the objective function constructed is:

[0052]

[0053]

[0054] in,

[0055] In the formula, X0, x j L0 and l represent the features of labeled samples and unlabeled samples, respectively. j Let n0 represent the labels of labeled samples and the pseudo-labels of unlabeled samples, respectively. Let n0, n, and r represent the number of labeled samples, the total number of samples, and the code length of the hash code, respectively. Let S represent the pairwise similarity, and w0 represent the pseudo-labels of unlabeled samples. j The weight (0 / 1) represents the sample weight, and its corresponding update parameter is λ. B0 represents the hash code of the labeled sample. These represent the hash projection mappings of the supervised hashing model and the unsupervised hashing model, respectively. α, β, and γ represent the classification mappings of supervised hashing and unsupervised hashing models, respectively; α, β, and γ are hyperparameters used to balance the various losses.

[0056] In the formula, the first, second, and third terms are the supervised loss, which aims to use the labels and pairwise similarities of the labeled samples as supervised information to guide hash learning, so that the learned hash code can reflect its semantic information. Specifically, the first term is the pairwise similarity embedding loss, the second term is the label embedding loss, and the third term is the quantization loss based on hash projection.

[0057] The fourth and fifth terms are unsupervised losses, which employ a self-paced learning framework to select samples by calculating the difficulty of the samples. Specifically, the fourth term is the sample selection loss, which is calculated by defining a measure of sample difficulty, L. i Develop a sample selection strategy. The fifth item is the self-synchronizing regularization loss, which is adopted here.

[0058]

[0059] Self-stepping learning, based on the idea of ​​learning from easy to difficult, introduces a binary variable (0 / 1) into the objective function of traditional machine learning methods to indicate whether each sample is selected. In each iteration of the training process, self-stepping learning selects samples with low loss from the entire dataset as simple samples for training and adjusts the model parameters. As the number of iterations increases, more samples are selected. Self-stepping learning can improve the model's robustness to noise points and outliers.

[0060] The fifth and seventh items are the difference loss between the supervised hash model and the unsupervised hash model, and their goal is to maintain the consistency between the supervised hash model and the unsupervised hash model.

[0061] Step c: Iterate between objective function optimization and sample selection, obtain the hash code of the retrieval database, and use the hash code of the retrieval database to learn the sample extrapolation mapping.

[0062] In the process of iterating between objective function optimization and sample selection, a self-stepping learning framework is adopted. First, the variables and parameters in the objective function are updated by alternating direction multipliers. Then, the sample selection is performed by calculating the difficulty of the samples. This process is repeated until the preset termination condition is reached.

[0063] Specifically, by defining a measure of sample difficulty, the following two sample selection strategies are formulated:

[0064] (1) A sample selection strategy that is aware of prediction difficulty, which selects unlabeled samples with low prediction difficulty and high prediction confidence, and uses them together with labeled samples to guide the hash learning process. The prediction difficulty is used to filter feature noise and pseudo-label noise, thereby improving retrieval accuracy.

[0065] (2) A sample selection strategy with consistent prediction time, which selects unlabeled samples whose prediction changes little over time, and guides the hash learning process together with labeled samples to improve training efficiency and stability.

[0066] The objective function for learning the sample extrapolation mapping P using the hash code B0 from the retrieved database is:

[0067]

[0068] Where X0 is the feature of the labeled sample, and δ is the hyperparameter.

[0069] Table 1 shows the experimental results on the CIFAR-10 dataset with 30% labels retained.

[0070]

[0071] Table 2 shows the experimental results on the CIFAR-10 dataset with 60% labels retained.

[0072]

[0073] Tables 1 and 2 present two simulation experiments of this invention, conducted on the CIFAR-10 dataset with 30% and 60% labels retained, respectively. Both experiments used MAP (Mean Mean Precision) as a metric, testing the performance of four hash code lengths (16, 32, 48, and 64 bits). The results were compared with nine existing supervised hashing methods (SDH, NSH, FSDH, R2SDH, SSDH, FSSH, SSLH, SDHMLR, and SCDH). The results show that this disclosure performs better in most cases.

[0074] Example 2

[0075] One embodiment of this disclosure provides a data retrieval system based on labeled sample augmentation and sample selection, including:

[0076] The hash code calculation module is used to obtain the image sample to be retrieved and calculate its hash code using the out-of-sample extended mapping.

[0077] The online retrieval module is used to calculate the Hamming distance between the image sample to be retrieved and the hash code in the retrieval database, and return similar data in ascending order of Hamming distance;

[0078] Among them, pseudo-labels are applied to unlabeled samples based on a labeled sample augmentation strategy;

[0079] A supervised loss is established based on labeled samples and their labels, an unsupervised loss is established based on unlabeled samples and their pseudo-labels, and a semi-supervised hash objective function is established based on the supervised loss and the unsupervised loss.

[0080] The algorithm iterates between objective function optimization and sample selection to obtain the hash code of the retrieval database, and uses the hash code of the retrieval database to learn the sample extra-sample extension mapping.

[0081] The labeled sample augmentation strategy is as follows: the features of labeled samples are used to reconstruct the features of unlabeled samples to obtain the reconstruction error of each category, and the category with the smaller reconstruction error is selected to assign pseudo-labels to the unlabeled samples.

[0082] The above system specifically executes the following methods and steps:

[0083] Obtain the image sample to be retrieved and calculate its hash code using the out-of-sample extended mapping.

[0084] Calculate the Hamming distance between the image sample to be retrieved and the hash code in the retrieval database, and return similar data in ascending order of Hamming distance;

[0085] Among them, pseudo-labels are applied to unlabeled samples based on a labeled sample augmentation strategy;

[0086] A supervised loss is established based on labeled samples and their labels, an unsupervised loss is established based on unlabeled samples and their pseudo-labels, and a semi-supervised hash objective function is established based on the supervised loss and the unsupervised loss.

[0087] The algorithm iterates between objective function optimization and sample selection to obtain the hash code of the retrieval database, and uses the hash code of the retrieval database to learn the sample extra-sample extension mapping.

[0088] Example 3

[0089] One embodiment of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements data retrieval method steps based on labeled sample augmentation and sample selection.

[0090] Example 4

[0091] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the data retrieval method based on labeled sample augmentation and sample selection.

[0092] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A data retrieval method based on labeled sample expansion and sample selection, characterized in that, The method comprises the following steps: An image sample to be retrieved is obtained, and a hash code of the image sample is calculated by using an out-of-sample extension mapping; A Hamming distance between the image sample to be retrieved and hash codes in a retrieval database is calculated, and similar data is returned in an order from small to large Hamming distance; Wherein, a labeled sample expansion strategy is used to pseudo-label unlabeled samples; A supervised loss is established based on labeled samples and their labels, an unsupervised loss is established based on unlabeled samples and their pseudo-labels, and a semi-supervised hash target function is established according to the supervised loss and the unsupervised loss; The target function is optimized and sample selection is iterated, hash codes of the retrieval database are obtained, and an out-of-sample extension mapping is learned by using the hash codes of the retrieval database; The sample selection defines the following two sample selection strategies by defining a difficulty metric of the sample: (1) A prediction difficulty-aware sample selection strategy is used to select an unlabeled sample with low prediction difficulty and high prediction confidence, and the unlabeled sample is used together with the labeled sample to guide the hash learning process, wherein the prediction difficulty is used to filter feature noise and pseudo-label noise; (2) A prediction time-consistent sample selection strategy is used to select an unlabeled sample with small prediction change over time, and the unlabeled sample is used together with the labeled sample to guide the hash learning process; The labeled sample expansion strategy is as follows: for each class of labeled sample, a bi-kmeans algorithm is used to determine a cluster center as a centroid of the class of labeled sample; Then, the centroids are used as anchor points, and a local anchor embedding algorithm is used to obtain reconstruction weight coefficients of the unlabeled samples; finally, the corresponding class is weighted by using the reconstruction weight coefficients to obtain pseudo-labels of the unlabeled samples; In the iteration process of the target function optimization and the sample selection, a self-paced learning framework is used, first, variables and parameters in the target function are updated by using an alternating direction multiplier method, then sample selection is performed by calculating the difficulty of the sample, and the iteration is sequentially performed until a preset termination condition is reached.

2. The data retrieval method based on label sample expansion and sample selection according to claim 1, wherein, The target function is composed of a supervised loss, an unsupervised loss, and a difference loss between a supervised hash model and an unsupervised hash model.

3. The data retrieval method based on label sample expansion and sample selection according to claim 1, wherein, The supervised loss is composed of a pair-wise similarity embedding loss, a label embedding loss, and a quantization loss of hash projection, and the unsupervised loss is composed of a sample selection loss and a self-paced regularization loss.

4. The data retrieval method based on label sample expansion and sample selection according to claim 3, wherein, The unsupervised loss uses a self-paced learning framework to select samples by calculating the difficulty of the samples.

5. A data retrieval system based on labeled sample augmentation and sample selection, characterized in that, The method comprises the following steps: A hash code calculation module is configured to obtain an image sample to be retrieved, and calculate a hash code of the image sample by using an out-of-sample extension mapping; An online retrieval module is configured to calculate a Hamming distance between the image sample to be retrieved and hash codes in a retrieval database, and return similar data in an order from small to large Hamming distance; Wherein, a labeled sample expansion strategy is used to pseudo-label unlabeled samples; A supervised loss is established based on labeled samples and their labels, an unsupervised loss is established based on unlabeled samples and their pseudo-labels, and a semi-supervised hash target function is established according to the supervised loss and the unsupervised loss; The target function is optimized and sample selection is iterated, hash codes of the retrieval database are obtained, and an out-of-sample extension mapping is learned by using the hash codes of the retrieval database; The sample selection defines the following two sample selection strategies by defining a difficulty metric of the sample: (1) A sample selection strategy for predicting difficulty, which selects unlabeled samples with low predicted difficulty and high predicted confidence to guide the hash learning process together with labeled samples, wherein the predicted difficulty is used to filter feature noise and pseudo-label noise; (2) A sample selection strategy for predicting time consistency, which selects unlabeled samples with small prediction changes over time to guide the hash learning process together with labeled samples; The labeled sample expansion strategy is: for each class of labeled samples, the bi-kmeans algorithm is used to determine the cluster center as the centroid of the labeled samples of the class; Then, the centroids are used as anchor points to obtain the reconstruction weight coefficients of the unlabeled samples by using the local anchor embedding algorithm; finally, the reconstruction weight coefficients are used to weight the corresponding categories to obtain the pseudo-labels of the unlabeled samples; In the iterative process between the optimization of the objective function and the selection of samples, a self-paced learning framework is adopted, first, the variables and parameters in the objective function are updated by the alternating direction multiplier method, then the difficulty of the samples is calculated to select the samples, and the iteration is performed in turn until the preset termination condition is reached.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the data retrieval method steps based on labeled sample expansion and sample selection as claimed in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the data retrieval method steps based on labeled sample expansion and sample selection as claimed in any one of claims 1-4.

Images