A consistent cross-modal hashing retrieval method and system

Abstract

The application relates to the technical field of data retrieval, and discloses a cross-modal hash retrieval method and system with consistency, which comprises the following steps: S1. acquiring heterogeneous data; S2. acquiring hash codes of the heterogeneous data, and performing hash code learning on the heterogeneous data; S3. performing hash function learning according to the obtained optimal hash code; and S4. mapping the heterogeneous data to the same low-rank Hamming space through the hash function after the function learning is completed, using an exclusive OR operation to the similarity of the heterogeneous data and a retrieval set, returning a result with high similarity, and completing cross-modal retrieval of the heterogeneous data to be retrieved. The application solves the problems of insufficient retrieval precision and complicated optimization process in the prior art, and has the characteristics of being capable of overcoming the heterogeneity of cross modalities.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data retrieval, and more particularly, to a cross-modal hashing retrieval method and system for maintaining consistency. BACKGROUND

[0002] With the development of the Internet, multimedia data (such as audio, image, text and 3D model) has exploded. Therefore, in recent years, cross-modal retrieval has attracted widespread interest, which is designed for similarity search from different modalities. For example, people can get relevant images from some text, or search for corresponding text through video. However, due to the fact that multi-modal data spans different feature spaces, their distributions are not in the same dimension, resulting in modality heterogeneity as a great challenge. In order to alleviate this problem, a simple method is to design a common latent subspace as a connection of heterogeneous data, so that the common representation can be calculated by a common distance metric. However, these methods still have some limitations, such as slow computing speed when dealing with a large number of samples. In order to effectively retrieve large-scale multimedia data, hash-based approximate nearest neighbor search technology has attracted much attention due to its fast search speed and low storage cost. Generally, hash technology maps high-dimensional multi-modal features to low-dimensional Hamming space, which is represented by binary code. Then, by implementing a simple XOR operation, the nearest neighbor can be extracted by measuring the Hamming distance between binary codes.

[0003] In the prior art, first, most methods directly project the original data into a low-dimensional common feature space using matrix factorization. Although they consider global and local structural information, the preservation of reconstruction residuals is ignored, resulting in energy loss in the mapping process. Second, most methods decompose heterogeneous data into a common latent subspace and then learn hash codes from it. This approach is unreasonable because different modes have different dimensions and distributions. Although some methods have learned multiple latent semantic spaces, the connection between these spaces has not been fully explored, so the heterogeneity of cross-modal is not enough to overcome. Third, since binary codes are discrete, the optimization process becomes an NP-hard problem. In order to alleviate this problem, some methods first relax the discrete constraint and then use a rounding optimization strategy to learn hash codes. However, some studies have shown that this scheme will bring huge quantization error and reduce retrieval accuracy.

[0004] The prior art has a similarity-preserving cross-modal hashing retrieval method, which includes the following steps: (1) construction of a target function based on a similarity-preserving strategy; (2) solving the target function; (3) generating binary hash codes of samples in the query sample and retrieval sample sets; (4) calculating the Hamming distance of the query sample to each sample in the retrieval sample set; (5) using a cross-modal retriever to complete the retrieval of the query sample.

[0005] However, the existing cross-modal retrieval technology based on hash code still has the problems of insufficient retrieval accuracy and complicated optimization process, and therefore how to provide a cross-modal retrieval method based on hash code with accurate retrieval and convenient optimization is an urgent technical problem to be solved in the technical field. SUMMARY

[0006] The present application provides a cross-modal hash retrieval method with consistency, which can overcome the heterogeneity of cross-modal data.

[0007] To achieve the above-mentioned purposes, the technical solutions adopted are as follows:

[0008] A cross-modal hash retrieval method with consistency comprises the following steps:

[0009] S1. Obtain heterogeneous data;

[0010] S2. Obtain hash codes of the heterogeneous data, and perform hash code learning on the heterogeneous data;

[0011] Project the heterogeneous data into their respective latent semantic spaces; use local preserving projection to maintain the consistency of the original data; align the latent semantic spaces of different modalities of the heterogeneous data, and maintain the similarity of different latent semantic spaces during the alignment process; obtain the optimal hash code unified in each latent semantic space of each modality;

[0012] S3. Perform hash function learning according to the obtained optimal hash code;

[0013] S4. Map the heterogeneous data to the same low-rank Hamming space through the hash function obtained by completing the function learning, use the XOR operation to determine the similarity between the heterogeneous data and the retrieval set, return the result with high similarity, and complete the cross-modal retrieval of the heterogeneous data to be retrieved.

[0014] Preferably, the heterogeneous data comprises image modal data and text modal data.

[0015] Further, in step S2, the hash code learning on the heterogeneous data is performed, specifically as follows:

[0016] S201. Convert the original features of the heterogeneous data into kernel feature representation using a radial basis function kernel;

[0017] S202. Project the heterogeneous data into their respective latent semantic spaces;

[0018] aligning the kernel feature representations of the image and text modalities into two respective latent low-dimensional semantic spaces; in the process of aligning, exploring the common high-level semantics of different modalities, calculating the similarity between modalities according to semantic labels, and keeping the similarity of the two latent low-dimensional semantic spaces according to the similarity between modalities, and making the fields of hash codes directly approximate the latent semantic spaces of different modalities;

[0019] S203. Constructing an optimal hash code target function according to the kernel feature representations of the image and text modalities, the projections of the latent low-dimensional semantic spaces, the similarity between modalities, and the hash code fields:

[0020] S204. Converting the target function into a trace form, performing feature learning, and iteratively updating the target function to obtain an optimal hash code.

[0021] Further, S201. converting the original features of the heterogeneous data into kernel feature representations using a radial basis function kernel, specifically:

[0022]

[0023]

[0024] wherein is the image modality data, is the text modality data, d1 represents the dimension of the original features of the image modality, d2 represents the dimension of the original features of the image modality, and n represents the number of training samples; A(A1…A k1 ) represents randomly selected image samples from the training set, B(B1…B k2 ) represents randomly selected text samples from the training set, and σ represents the kernel width of the convolution kernel used for conversion; is the kernel feature representation of the image features, is the kernel feature representation of the text modality.

[0025] Further, in step S202, the kernel feature representations of the image and text modalities are aligned by projecting them into two respective latent low-dimensional semantic spaces, specifically: by minimizing the following objective function (1), the kernel feature representations of the image and text modalities are aligned by projecting them into two respective latent low-dimensional semantic spaces:

[0026]

[0027] wherein and represent different learning mapping matrices of the image modality, and represent different learning mapping matrices of the text modality; r represents the hash code length and θ is a balance parameter; S m∈{0,1} n×n It is a similarity matrix calculated based on the semantic labels of the modalities, where m = v represents the image modality and m = t represents the text modality, and the subscript T indicates transpose.

[0028] Furthermore, in step S202, the similarity between the two potential low-dimensional semantic spaces is maintained based on the similarity between modalities, and the fields of the hash code are made to directly approximate the potential semantic spaces of different modalities. Specifically, the similarity between the two potential low-dimensional semantic spaces is maintained based on the similarity between modalities by minimizing the following objective function (2):

[0029]

[0030] Where α is a balance parameter, and P∈{0,1} n×n It is the consistency matrix between modes; P ij =1 indicates that the image sample x i and text sample y j Sharing the same tags, P ij =0 indicates that they belong to different classes;

[0031] The following objective function (3) allows the fields of the hash code to directly approximate the latent semantic spaces of different modalities:

[0032]

[0033] Where β is a balance parameter.

[0034] Furthermore, in step S203, the objective function for the optimal hash code is constructed, and the objective function is transformed into the form of a trace. Specifically, the objective function (4) for the optimal hash code is as follows:

[0035]

[0036] use Instead of ψ(X), an auxiliary variable is introduced. and And transform the objective function into trace form:

[0037]

[0038] in Equivalent to the trace (5) to optimizing its augmented Lagrangian function (6):

[0039]

[0040] Further, in step S205, feature learning is performed, and the target function is iteratively updated to obtain optimal hash codes; specifically, variables in the augmented Lagrange function (6) are optimized; when one variable is optimized, other variables are fixed, and the gradients of other variables are set to 0, thereby obtaining optimal solutions of these variables:

[0041] The optimal solution of C is:

[0042]

[0043] The optimal solution of D is:

[0044]

[0045] The optimal solution of E is:

[0046] E = Sylvester (J v ,K v ,L v ).

[0047] where J v = 2αFF T , K v = 2H v + (2β+μ)I

[0048] The optimal solution of Sylvester matrix equation is obtained by using the Bartels-Stewart algorithm;

[0049] The optimal solution of F is:

[0050] F = Sylvester (J t ,K t ,L t ).

[0051] where J t = 2αEE T , J t = 2θH t + (2β+μ)I and

[0052]

[0053] Optimizing U1, first using singular value decomposition (SVD):

[0054]

[0055] The optimal solution of U1 is obtained:

[0056] U1 = M v (N v ) T .

[0057] For U2, singular value decomposition is as follows:

[0058]

[0059] Obtain the optimal solution of U2:

[0060] U2=M t (N t ) T .

[0061] The optimal solution of B is:

[0062] B=sgn(E+F).

[0063] Wherein sgn() is a sign function;

[0064] Update C1, C2 and mu:

[0065]

[0066]

[0067] mu <- min (p mu, mu max );

[0068] According to the optimal solution of the variable B, the optimal hash code is obtained.

[0069] Further, in the step S3, according to the obtained optimal hash code, the hash function learning is carried out, specifically: adopting a linear classifier As a hash function:

[0070]

[0071] Wherein, o represents the index of a certain bit; Representing the lth sample in b o , is a balance parameter.

[0072] A cross-modal hash retrieval method system maintaining consistency, comprising a data acquisition module, a hash code learning module, a hash function learning module and a cross-modal retrieval module.

[0073] The data acquisition module is used for acquiring heterogeneous data to be retrieved.

[0074] ​The hash code learning module is used for obtaining hash codes of the heterogeneous data, learning hash codes of the heterogeneous data, projecting the heterogeneous data into respective latent semantic spaces, using local preserving projection to keep consistency of the original data, aligning the latent semantic spaces of different modalities of the heterogeneous data and keeping similarity of the different latent semantic spaces in the alignment process, and obtaining optimal hash codes unified in the latent semantic spaces of each modality.

[0075] The hash function learning module is used for learning hash functions according to the obtained optimal hash codes.

[0076] The cross-modality retrieval module is used for performing cross-modality retrieval on the heterogeneous data to be retrieved according to the hash code functions completed by the function learning.

[0077] The present application has the following beneficial effects:

[0078] The present application discloses a cross-modality hash retrieval method keeping consistency, which projects the heterogeneous data into respective latent semantic spaces, uses local preserving projection to keep consistency of the original data, aligns the latent semantic spaces of different modalities of the heterogeneous data and keeps similarity of the different latent semantic spaces in the alignment process, and obtains optimal hash codes unified in the latent semantic spaces of each modality. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 is a flowchart of the cross-modality hash retrieval method keeping consistency of the present application.

[0080] Figure 2 is a hash code learning flowchart of the cross-modality hash retrieval method keeping consistency of the present application.

[0081] Figure 3 is a hash function learning flowchart of the cross-modality hash retrieval method keeping consistency of the present application. DETAILED DESCRIPTION

[0082] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0083] Embodiment 1

[0084] As shown in the drawings, Figure 1 A cross-modality hash retrieval method keeping consistency includes the following steps:

[0085] S1. Obtain heterogeneous data.

[0086] S2. Obtain hash codes of the heterogeneous data, and learn hash codes of the heterogeneous data;

[0087] Project the heterogeneous data into their respective latent semantic spaces; use local preserving projection to preserve the consistency of the original data; align the latent semantic spaces of different modalities of the heterogeneous data, and maintain the similarity of different latent semantic spaces in the alignment process; obtain the optimal hash codes unified in the latent semantic spaces of each modality;

[0088] S3. Learn hash functions according to the obtained optimal hash codes;

[0089] S4. Map the heterogeneous data to the same low-rank Hamming space through the hash functions learned by function learning, use XOR operation to compare the similarity of the heterogeneous data and a retrieval set, return the result with high similarity, and complete the cross-modal retrieval of the heterogeneous data to be retrieved.

[0090] Embodiment 2

[0091] As shown in Figure 1 A cross-modal hash retrieval method for maintaining consistency includes the following steps:

[0092] S1. Obtain heterogeneous data;

[0093] S2. Obtain hash codes of the heterogeneous data, and learn hash codes of the heterogeneous data;

[0094] Project the heterogeneous data into their respective latent semantic spaces; use local preserving projection to preserve the consistency of the original data; align the latent semantic spaces of different modalities of the heterogeneous data, and maintain the similarity of different latent semantic spaces in the alignment process; obtain the optimal hash codes unified in the latent semantic spaces of each modality;

[0095] The application uses principal component analysis to prevent loss of main energy of data. The latent semantic spaces of different modalities of the heterogeneous data are aligned to share the same high-level semantics and try to overcome the heterogeneity between modalities. The consistency between modalities is calculated using semantic labels, and the similarity of different semantic spaces is maintained in the alignment process. The final optimal hash code is a unified binary code obtained from the latent space of each modality. The binary code is generated discretely, and no relaxation is required in the optimization process, thereby avoiding huge quantization errors;

[0096] S3. Learn hash functions according to the obtained optimal hash codes;

[0097] S4. The hash function learned by completing the function maps the heterogeneous data to the same low-rank Hamming space, uses the XOR operation to compare the similarity of the heterogeneous data and the retrieval set, and returns the result with high similarity, thereby completing the cross-modal retrieval of the heterogeneous data to be retrieved.

[0098] In one specific embodiment, the heterogeneous data includes image modal data and text modal data.

[0099] As shown in Figure 2 one specific embodiment, in step S2, the hash code learning is performed on the heterogeneous data, specifically:

[0100] S201. The original features of the heterogeneous data are converted into kernel feature representations using a radial basis function kernel.

[0101] S202. The heterogeneous data is projected into its respective latent semantic space through nonlinear mapping.

[0102] The kernel feature representations of the image and text modalities are projected into two respective latent low-dimensional semantic spaces and aligned. During the alignment process, the common high-level semantics of different modalities are explored, the similarity between modalities is calculated according to the semantic labels, and the similarity between the two latent low-dimensional semantic spaces is maintained according to the similarity between the modalities, and the fields of the hash codes are directly approximated to the latent semantic spaces of different modalities.

[0103] S203. An optimal hash code objective function is constructed according to the kernel feature representations of the image and text modalities, the projections of the latent low-dimensional semantic spaces, the similarity between the modalities, and the hash code fields.

[0104] S204. The objective function is converted into a trace form, feature learning is performed, the objective function is iteratively updated, and the optimal hash code is obtained.

[0105] In one specific embodiment, S201. The original features of the heterogeneous data are converted into kernel feature representations using a radial basis function kernel, specifically:

[0106]

[0107]

[0108] wherein is the image modal data, is the text modal data, d1 represents the dimension of the original features of the image modality, d2 represents the dimension of the original features of the image modality, and n represents the number of training samples; A(A1…A k1 represents an image sample randomly selected from the training set, B(B1…B k2) represents a text sample randomly selected from the training set, and σ represents the kernel width of the convolution kernel used for the transformation; Kernel feature representation of image features This represents the kernel features of the text modality.

[0109] In one specific embodiment, in step S202, the kernel feature representations of the image and text modalities are projected onto their respective two latent low-dimensional semantic spaces for alignment. Specifically, this is achieved by minimizing the following objective function (1):

[0110]

[0111] in and Different learning mapping matrices representing image modalities and Different learning mapping matrices represent different text modalities; r represents the hash code length and θ is a balancing parameter; S m ∈{0,1} n×n It is a similarity matrix calculated based on the semantic labels of the modalities, where m = v represents the image modality and m = t represents the text modality, and the subscript T indicates transpose.

[0112] In one specific embodiment, in step S202, the similarity between two potential low-dimensional semantic spaces is maintained based on the similarity between modalities, and the fields of the hash code are made to directly approximate the potential semantic spaces of different modalities. Specifically, the similarity between two potential low-dimensional semantic spaces is maintained based on the similarity between modalities by minimizing the following objective function (2):

[0113]

[0114] Where α is a balance parameter, and P∈{0,1} n×n It is the consistency matrix between modes; P ij =1 indicates that the image sample x i and text sample y j Sharing the same tags, P ij =0 indicates that they belong to different classes;

[0115] The following objective function (3) allows the fields of the hash code to directly approximate the latent semantic spaces of different modalities:

[0116]

[0117] Where β is a balance parameter.

[0118] In one embodiment, the step S203, the objective function of the optimal hash code is constructed, and the objective function is converted into the form of trace, specifically: the objective function (4) of the optimal hash code is as follows:

[0119]

[0120] Using Instead of ψ(X), an auxiliary variable Follow And the objective function is converted into the form of trace:

[0121]

[0122] Wherein The trace (5) is equivalent to the augmented Lagrangian function (6) which is optimized:

[0123]

[0124] In one embodiment, the step S205, the feature learning is performed, the target function is iteratively updated, and the optimal hash code is obtained; specifically: the variables in the augmented Lagrangian function (6) are optimized; when one variable is optimized, other variables are fixed, and the gradient of other variables is 0, so that the optimal solution of these variables is obtained:

[0125] The optimal solution of C is:

[0126]

[0127] The optimal solution of D is:

[0128]

[0129] The optimal solution of E is:

[0130] E=Sylvester(J v ,K v ,L v ).

[0131] Wherein J v =2αFF T , K v =2H v +(2β+μ)I

[0132] The optimal solution of is the Sylvester matrix equation, and the Bartels-Stewart algorithm is used to solve;

[0133] The optimal solution of F is:

[0134] F=Sylvester(Jt ,K t ,L t ).

[0135] J t =2αEE T J t =2θH t +(2β+μ)I and

[0136]

[0137] To optimize U1, first use Singular Value Decomposition (SVD):

[0138]

[0139] Obtain the optimal solution for U1:

[0140] U1 = M v (N v ) T .

[0141] For U2, the singular value decomposition is as follows:

[0142]

[0143] Obtain the optimal solution for U2:

[0144] U2 = M t (N t ) T .

[0145] The optimal solution for B:

[0146] B = sgn(E + F).

[0147] Where sgn() is a symbolic function;

[0148] Update C1, C2, and μ:

[0149]

[0150]

[0151] μ←min(ρμ,μ max );

[0152] The optimal hash code is obtained based on the optimal solution for variable B.

[0153] like Figure 3 As shown, in this embodiment, after performing nonlinear mapping on the image modalities, hash function learning is performed in conjunction with the optimal hash code.

[0154] In step S3, according to the obtained optimal hash code, hash function learning is performed, specifically: a linear classifier is used As a hash function:

[0155]

[0156] Wherein, o represents the index of a certain bit; Represent b o The lth sample in b Is a balance parameter.

[0157] The application discloses a cross-modal hash retrieval method for keeping consistency, which projects heterogeneous data into respective latent semantic spaces thereof; uses local preserving projection to keep consistency of original data; aligns latent semantic spaces of different modes of the heterogeneous data and keeps similarity of different latent semantic spaces during the alignment; obtains optimal hash codes unified in each latent semantic space; thus, the application improves sample similarity optimization, solves the problems of insufficient retrieval precision and complicated optimization process of existing cross-modal retrieval technology based on hash codes, and overcomes the heterogeneity of cross-modal.

[0158] Embodiment 3

[0159] A cross-modal hash retrieval method system for keeping consistency, comprising a data acquisition module, a hash code learning module, a hash function learning module and a cross-modal retrieval module.

[0160] The data acquisition module is used to acquire heterogeneous data to be retrieved.

[0161] The hash code learning module is used to acquire hash codes of the heterogeneous data, and to learn hash codes of the heterogeneous data; the heterogeneous data is projected into respective latent semantic spaces thereof; local preserving projection is used to keep consistency of original data; latent semantic spaces of different modes of the heterogeneous data are aligned, and similarity of different latent semantic spaces is kept during the alignment; optimal hash codes unified in each latent semantic space are obtained.

[0162] The hash function learning module is used to learn hash functions according to the obtained optimal hash codes.

[0163] The cross-modal retrieval module is used to perform cross-modal retrieval on the heterogeneous data to be retrieved according to the hash code functions learned.

[0164] Obviously, the above-mentioned embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the implementation modes of the present application. Any modification, equivalent replacement and improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A method for consistent cross-modal hash retrieval, characterized in that: The method comprises the following steps: S1. obtaining heterogeneous data; S2. obtaining hash codes of the heterogeneous data, and learning hash codes of the heterogeneous data; projecting the heterogeneous data into respective latent semantic spaces thereof; using local preserving projection to maintain consistency of the original data; aligning the latent semantic spaces of different modalities of the heterogeneous data, and maintaining similarity of the different latent semantic spaces during the alignment; and obtaining optimal hash codes in the latent semantic spaces of each modality that are unified; wherein the hash codes of the heterogeneous data are learned; specifically: S201. converting original features of the heterogeneous data into kernel feature representations using a radial basis function kernel; S202. projecting the heterogeneous data into respective latent semantic spaces thereof; aligning the kernel feature representations of the image and text modalities in the respective two latent low-dimensional semantic spaces; during the alignment, exploring common high-level semantics of the different modalities, calculating similarity between the modalities according to the semantic labels, and maintaining similarity of the two latent low-dimensional semantic spaces according to the similarity between the modalities, and making fields of the hash codes directly approximate to the latent semantic spaces of the different modalities; S203. constructing a target function of the optimal hash codes according to the kernel feature representations of the image and text modalities, the projections of the latent low-dimensional semantic spaces, the similarity between the modalities, and the fields of the hash codes: S204. converting the target function into a form of trace, performing feature learning, and iteratively updating the target function to obtain the optimal hash codes S3. learning hash functions according to the obtained optimal hash codes; S4. mapping the heterogeneous data to a same low-rank Hamming space through the hash functions learned through the function learning, using an exclusive OR operation to compare similarity of the heterogeneous data with a retrieval set, and returning a result with high similarity, thereby completing cross-modality retrieval of the heterogeneous data to be retrieved. 2.The consistent preserving cross-modal hashing retrieval method of claim 1, wherein: The heterogeneous data comprises image modality data and text modality data. 3.The consistent preserving cross-modal hashing retrieval method of claim 1, wherein: S201. converting original features of the heterogeneous data into kernel feature representations using a radial basis function kernel, specifically: wherein is image modality data, is text modality data, denotes the dimension of the raw features of the image modality, d 2denotes the dimension of the raw features of the image modality, denotes the number of training samples; A 1… A k1 represents an image sample randomly picked from the training set, B B 1… B k2 represents a text sample randomly picked from the training set, denotes the kernel width of the convolution kernel used for the transformation; is the kernel feature representation of the image features, is the kernel feature representation of the text modality.​ 4. The consistent preserving cross-modal hash retrieval method according to claim 3, characterized in that: In the step S202, the kernel feature representations of the image and text modalities are aligned in the respective two latent low-dimensional semantic spaces, specifically: the kernel feature representations of the image and text modalities are aligned in the respective two latent low-dimensional semantic spaces by minimizing the following target function (1): （1） wherein followed by different learning mapping matrices representing the image modalities, followed by different learning mapping matrices representing the text modalities; representing the hash code length and is a balancing parameter; is a similarity matrix computed from the semantic labels of the modalities, representing the image modalities, and representing the text modalities, the index T denotes the transpose.

5. The consistent preserving cross-modal hash retrieval method according to claim 4, characterized in that: In the step S202, the similarity between the modalities is used to maintain similarity of the two latent low-dimensional semantic spaces, and the fields of the hash codes are directly approximated to the latent semantic spaces of the different modalities, specifically: the similarity between the modalities is used to maintain similarity of the two latent low-dimensional semantic spaces by minimizing the following target function (2): （2） wherein is a balance parameter, while is a consistency matrix between modalities; denotes image samples and text samples share the same label, denotes that they belong to different classes; The fields of the hash codes are directly approximated to the latent semantic spaces of the different modalities by the following target function (3): （3） wherein is an equilibrium parameter.

6. The consistent preserving cross-modal hash retrieval method according to claim 5, characterized in that: In the step S203, the target function of the optimal hash codes is constructed, and the target function is converted into a form of trace, specifically: the target function (4) of the optimal hash codes is as follows: （4） Using Instead , introduce auxiliary variables Follow , and transform the objective function into the form of a trace: （5） where ; trace (5) is equated to the augmented Lagrangian function (6) that optimizes it: 。 7. The consistent preserving cross-modal hash retrieval method according to claim 6, characterized in that: In step S205, feature learning is performed, and the target function is iteratively updated to obtain optimal hash codes; specifically, variables in the augmented Lagrange function (6) are optimized; when one variable is optimized, other variables are fixed, and the gradients of other variables are set to 0, thereby obtaining the optimal solution of these variables: Optimal solution: Optimal solution of: Optimal solution: wherein , with The optimal solution of E is Sylvester matrix equation, which is solved by Bartels-Stewart algorithm. Optimal solution: wherein , and ; optimization First, singular value decomposition (SVD) is used: obtained optimal solution: For , singular value decomposition is as follows: obtained optimal solution: Optimal solution: Where sgn() is a sign function. update , and : ； According to the optimal solution of the variable B, the optimal hash code is obtained.

8. The consistent preserving cross-modal hash retrieval method according to claim 7, characterized in that: In step S3, according to the obtained optimal hash code, the hash function learning is performed, specifically: a linear classifier is used As a hash function: wherein denotes the index of a certain bit position; represents the thsample in is a balance parameter.

9. A consistent preserving cross-modal hash retrieval system, characterized in that: The method is used for realizing the method in any one of claims 1-8, comprising a data acquisition module, a hash code learning module, a hash function learning module, and a cross-modal retrieval module; The data acquisition module is used for acquiring heterogeneous data to be retrieved; The hash code learning module is used for acquiring hash codes of the heterogeneous data, learning hash codes of the heterogeneous data, and projecting the heterogeneous data into respective latent semantic spaces of the heterogeneous data; The local preserving projection is used for preserving consistency of the original data; the latent semantic spaces of different modalities of the heterogeneous data are aligned, and similarity of different latent semantic spaces is maintained in the alignment process; and unified optimal hash codes in the latent semantic spaces of each modality are obtained; The hash function learning module is used for learning hash functions according to the obtained optimal hash codes; The cross-modal retrieval module is used for performing cross-modal retrieval on the heterogeneous data to be retrieved according to the hash code functions after the function learning is completed.

Images