A hybrid hypergraph regularization semi-supervised cross-modal hashing method based on concept decomposition

Abstract

The application provides a hybrid hypergraph regularization semi-supervised cross-modal hashing method based on concept decomposition and belongs to the technical field of computers.The application solves the training problem of a large amount of unlabeled data in cross-modal retrieval, fully excavates the hybrid hypergraph high-order relationship between labeled data and unlabeled data, improves the model retrieval capability, and comprises the following steps: concept decomposition-based public semantic representation learning, cross-modal hybrid hypergraph construction, classification loss measurement of the labeled data, overall objective function construction and optimization, and learning of a quantization rotation matrix and a cross-modal hashing function, and finally, cross-modal cross-retrieval is performed by converting the cross-modal data into binary hash codes through the hash function. The application can effectively utilize the unlabeled data for semi-supervised cross-modal hash retrieval.

Description

Technical Field

[0001] This invention belongs to the field of computer technology and relates to a semi-supervised cross-modal retrieval method for images and text on the Internet, and more particularly to a hybrid hypergraph regularization semi-supervised cross-modal hashing method based on concept decomposition. Background Technology

[0002] With the development of social media and the diversification of data collection methods across industries, the explosive growth of multimodal data (such as user-generated images and mosquitoes posted on Weibo) has brought unprecedented challenges to cross-modal data retrieval. Cross-modal retrieval involves using data from one modality to retrieve data from another, such as retrieving related items by taking a picture on an online shopping platform, or retrieving related images and videos by using a piece of text. Due to the large data storage space required and the computationally intensive nature of traditional retrieval metrics, cross-modal hashing has become an effective method for solving cross-modal retrieval problems. It represents data such as images, text, and videos in a binary encoded feature form, greatly saving space. Simultaneously, bit operations enable rapid comparison and retrieval, reducing computational time costs. Therefore, cross-modal hashing is one of the most popular research problems today. Most existing methods are supervised models, with a few unsupervised models. However, supervised models require a large amount of labeled data, which is expensive and labor-intensive in today's big data era. Conversely, while unsupervised models do not require labeled data, their cross-modal retrieval performance is often lower than that of supervised methods. Therefore, researchers have proposed a semi-supervised approach that can leverage the semantic accuracy of labeled data while also utilizing a large amount of unlabeled data to provide feature-assisted learning.

[0003] While many excellent works have been proposed in semi-supervised cross-modal hashing, some shortcomings remain. Some methods employ label prediction on unlabeled data, simulating prediction on labeled data. However, the model's prediction accuracy is crucial, and the accumulation of label prediction errors can degrade model performance. Others combine traditional unsupervised and supervised methods to process labeled and unlabeled data separately; however, most of these methods simply utilize the structural information of unlabeled data, lacking the ability to mine higher-order semantic relationships and cross-modal higher-order relationships between samples.

[0004] To address this, this invention proposes a semi-supervised cross-modal hashing method based on concept decomposition and hybrid hypergraph regularization. Firstly, unlike methods such as matrix factorization for learning latent semantic representations, this invention employs concept decomposition to learn the intrinsic semantic information of cross-modal data, enabling it to accept negative cross-modal features and improve model robustness. Furthermore, this invention proposes cross-modal hybrid hypergraph regularization to constrain samples to maintain high-order relationships, considering both inter-modal and intra-modal high-order neighbor relationships. For both unlabeled and labeled samples, a unified hypergraph for both modalities is constructed, and then the two modalities are combined to form a hybrid hypergraph, thereby constraining the learning of common semantic representations. Additionally, this invention performs classification prediction on the common semantic representations of labeled data, minimizing classification loss and improving the discriminative power of the common semantic representations. After learning the common semantic representations of the training data, these can be used to learn orthogonal rotation matrices and hash functions to achieve hash encoding learning and cross-modal retrieval of new data. Summary of the Invention

[0005] This invention addresses the cross-modal retrieval problem between images and text on the Internet, proposing a semi-supervised cross-modal hashing scheme. It explores a hybrid hypergraph regularized cross-modal hashing method based on concept decomposition. Concept decomposition is used to learn common semantic representations, and a cross-modal hybrid hypergraph is used to optimize and constrain these common semantic representations to improve local structure preservation. The hypergraph models higher-order relationships between samples, guiding the learning of more discriminative common semantic representations. Simultaneously, the classification error of the common semantic representations of labeled data is minimized. Finally, orthogonal rotation matrices and hash functions for both image and text modalities are learned to generate binary hash codes for database data and new image / text query data. The most similar database image or text is calculated using these hash codes to achieve cross-modal retrieval.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A semi-supervised cross-modal hashing method based on concept decomposition and hybrid hypergraph regularization is disclosed. The concept decomposition is divided into a common semantic representation learning module. The hybrid hypergraph regularization integrates hypergraphs from both image and text modalities for regularization constraint learning of the common semantic representation. The semi-supervised cross-modal hashing method includes both labeled and unlabeled data. The entire invention comprises the following steps:

[0008] Step 1, Common Semantic Representation Learning: Based on concept decomposition, learn the intrinsic semantic representation of cross-modal data;

[0009] Step 2: Construct a cross-modal hybrid hypergraph by mixing data from different modalities.

[0010] Step 3: Measurement of classification loss for labeled data. For labeled data, minimize the classification label prediction loss.

[0011] Step 4: Construction and optimization of the overall objective function, combining the previous steps to form the overall objective function;

[0012] Step 5: Quantization rotation matrix learning. Orthogonal rotation matrices are learned through common semantic representations to reduce quantization loss.

[0013] Step 6: Learn the cross-modal projection matrix and construct the hash function.

[0014] The specific steps are as follows:

[0015] A hybrid hypergraph regularized semi-supervised cross-modal hashing method based on concept decomposition includes the following steps:

[0016] Step 1: Learning common semantic representations

[0017] make Representing the features of image modal data, This represents the features of text modal data, where each modality contains n samples, l of which are labeled. The image feature dimension is d1, i.e. The text feature dimension is d2. Given labeled data c represents the number of data categories. The label vector uses 0 and 1 to represent each other. When Y(j,i) = 1, it means that the j-th sample belongs to the i-th category; otherwise, if it does not belong, Y(j,i) = 0. Unlike supervised methods, which can only use labeled data to train the model, this invention can make full use of unlabeled data to improve the robustness of the model.

[0018] For cross-modal data that includes both labeled and unlabeled data, this invention proposes a concept decomposition-based method to learn the inherent semantic information of the cross-modal data. Compared to nonnegative matrix factorization, concept decomposition allows the input data to contain negative numbers, thus offering better compatibility. Let... Let k be the common semantic representation of two modalities, where k is the dimension of the representation space. To relax the common semantic representation, this invention adopts the two representations as similar as possible, rather than constraining them to a common representation. Therefore, the objective function of concept decomposition is shown in Equation (1):

[0019]

[0020] in, This represents the correlation matrix, which records data concepts. α is the modal weight parameter, ||·|| F Let Frobenius norm be the expression. Combined with the requirement to be as close as possible to the common semantic representation, this forms the objective function shown in equation (2).

[0021]

[0022] Step 2, construct a cross-modal hybrid hypergraph

[0023] Typically, unsupervised training models use the local structure of the data itself to constrain the representation learning of subspaces. Therefore, in cross-modal retrieval, many methods employ graph regularization to constrain the representation of subspaces, ensuring that adjacent samples in the original space remain close in the target common semantic space. However, existing methods often treat each modality individually, preserving the local structure within a modality while neglecting the relationships between modalities. This invention proposes a hybrid hypergraph regularization constraint. Hypergraphs can model the adjacency relationships between multiple samples, establishing high-order local relationships within a modality. Simultaneously, through the construction of a hybrid hypergraph, high-order relationships between two modalities are fused to guide the learning of common semantic representations and improve discriminativity.

[0024] First, an association matrix H1 is constructed based on the image modality data X1. The construction method involves building a hyperedge for each node, with each hyperedge consisting of the node and its K most similar neighbors. The similarity is measured using Euclidean distance. Simultaneously, for the text modality, a corresponding association matrix H2 is constructed. Next, a hybrid hypergraph is constructed by concatenating the hyperedges of the two modalities to form a hypergraph with 2n hyperedges, as shown in formula (3).

[0025] H=[H1,H2], (3)

[0026] Then, the Laplacian matrix L of the hybrid hypergraph can be calculated from the incidence matrix H. h Therefore, the hybrid hypergraph Laplace regularization constraint can be defined as the objective function shown in Equation (4).

[0027]

[0028] Here, β is the weight parameter. The objective function constrains the common semantic representations V1 and V2 to maintain higher-order relationships within and between modes.

[0029] Step 3: Measurement of Classification Loss for Labeled Data

[0030] For labeled data, this invention minimizes the classification label prediction loss, and the objective function is shown in formula (5).

[0031]

[0032] Where λ and γ represent hyperparameters. It is a projection matrix. This indicates the selection matrix. Since unlabeled data has no labels, classification loss is only measured for labeled data.

[0033] Step 4: Construction and optimization of the overall objective function

[0034] Combining formulas (1), (2), (4), and (5) from the previous steps, a general objective function is formed, and the relevant variables are optimized, as shown in formula (6).

[0035]

[0036] The next step is to optimize the objective function. This invention uses an alternating iterative approach until convergence. Finally, the common semantic subspace is weighted and averaged for the image and text modalities to obtain the final latent semantic subspace, as shown in formula (7).

[0037] V=αV1+(1-α)V2 (7)

[0038] Step 5: Quantize rotation matrix learning

[0039] Using hash representation can save storage space and improve retrieval speed. Therefore, the common semantic space is converted into a binary hash code. Usually, the symbols of the common semantic representation are directly taken to obtain the hash code, which can easily introduce large quantization loss and large error. Therefore, this invention obtains the hash code by learning an orthogonal rotation matrix, forming the objective function as shown in formula (8).

[0040]

[0041] Where B represents the hash code, R represents the rotation matrix, and I k Let represent the identity matrix. Equation (8) is a typical Orthogonal Procrustes problem, which can be obtained through singular value decomposition. Similarly, iterative optimization is used to obtain the optimal solution.

[0042] Step 6: Learn the cross-modal projection matrix and construct the hash function.

[0043] Finally, the orthogonal rotation matrix of the quantization is learned through the common semantic representation. In order to quickly convert the queried image or text data into binary code, it is also necessary to learn the hash functions of the image modality and the text modality, and directly project the original features into the Hamming space. Therefore, for image modality data and text modality data, the objective function shown in Equation (9) is constructed.

[0044]

[0045] Where P1 and P2 represent projection matrices. After iterative optimization, P1 and P2 can be obtained as shown in formulas (10) and (11).

[0046]

[0047]

[0048] Therefore, finally for the image modal data x 1 The hash code is obtained as shown in formula (12):

[0049] h1 = sgn(RP1x 1 (12)

[0050] For text modal data x 2 The hash code is obtained as shown in formula (13):

[0051] h2=sgn(RP2x 2 (13)

[0052] After learning how to use hash functions, this invention can obtain the hash code of a new query image or query text, and then calculate the text hash code that is closest to the hash code of the query image from the database to achieve image retrieval of text, or calculate the image hash code that is closest to the hash code of the query text from the database to achieve text retrieval of images.

[0053] Analysis of the innovations of this invention: (1) This invention is designed for cross-modal data retrieval and can receive both labeled and unlabeled cross-modal data, not limited to image, text, and voice data. (2) This invention employs concept decomposition to learn the inherent, essential, and common semantic information of cross-modal data. (3) This invention constructs a hybrid hypergraph, which can model high-order relational information of cross-modal data.

[0054] The beneficial effects of this invention are as follows: Based on concept decomposition, this invention accurately learns the common semantic representation of cross-modal data. At the same time, by modeling a hybrid hypergraph, it further improves the discrimination ability by constraining the learning of common semantic representation through regularization. For labeled data, it constrains the learning by minimizing the classification loss of the labels, and finally achieves the training and cross-retrieval of a semi-supervised cross-modal data retrieval model using a large amount of unlabeled data. Attached Figure Description

[0055] Figure 1 This is a diagram of the algorithm steps. Detailed Implementation

[0056] The following description further illustrates the implementation of the present invention by combining cross-retrieval of images and text on the Internet.

[0057] A hybrid hypergraph regularized semi-supervised cross-modal hashing method based on concept decomposition includes the following steps:

[0058] Step 1: Learning Common Semantic Representations of Images and Text

[0059] make Representing the features of image modal data, This represents the features of text modal data, where each modality contains n samples, l of which are labeled. The image feature dimension is d1, i.e. The text feature dimension is d2. Given labeled data c represents the number of data categories. The label vector uses 0 and 1 to represent each other. When Y(j,i) = 1, it means that the j-th sample belongs to the i-th category; otherwise, if it does not belong, Y(j,i) = 0. Unlike supervised methods, which can only use labeled data to train the model, this invention can make full use of unlabeled data to improve the robustness of the model.

[0060] For cross-modal image and text data that includes both labeled and unlabeled data, this invention proposes a concept decomposition-based method to learn the inherent semantic information of the cross-modal image and text data. Compared to nonnegative matrix factorization, concept decomposition allows the input data to contain negative numbers, thus offering better compatibility. Let... Let k be the common semantic representation of two modalities, where k is the dimension of the representation space. To relax the common semantic representation, this invention adopts the two representations as similar as possible, rather than constraining them to a common representation. Therefore, the objective function of concept decomposition is shown in Equation (1):

[0061]

[0062] in, This represents the correlation matrix, which records data concepts. α is the modal weight parameter, ||·|| F Let Frobenius norm be the expression. Combined with the requirement to be as close as possible to the common semantic representation, this forms the objective function shown in equation (2).

[0063]

[0064] Step 2: Construct a cross-modal hybrid hypergraph for image and text.

[0065] Typically, unsupervised training models use the local structure of the data itself to constrain the representation learning of subspaces. Therefore, in cross-modal retrieval, many methods employ graph regularization to constrain the representation of subspaces, ensuring that adjacent samples in the original space remain close in the target common semantic space. However, existing methods often treat each modality individually, preserving the local structure within a modality while neglecting the relationships between modalities. This invention proposes a hybrid hypergraph regularization constraint. Hypergraphs can model the adjacency relationships between multiple samples, establishing high-order local relationships within a modality. Simultaneously, through the construction of a hybrid hypergraph, high-order relationships between two modalities are fused to guide the learning of common semantic representations and improve discriminativity.

[0066] First, an association matrix H1 is constructed based on the image modality data X1. The construction method involves building a hyperedge for each node, with each hyperedge consisting of the node and its K most similar neighbors. The similarity is measured using Euclidean distance. Simultaneously, for the text modality, a corresponding association matrix H2 is constructed. Next, a hybrid hypergraph is constructed by concatenating the hyperedges of the two modalities to form a hypergraph with 2n hyperedges, as shown in formula (3).

[0067] H=[H1,H2], (3)

[0068] Then, the Laplacian matrix L of the hybrid hypergraph can be calculated from the incidence matrix H. h Therefore, the hybrid hypergraph Laplace regularization constraint can be defined as the objective function shown in Equation (4).

[0069]

[0070] Here, β is the weight parameter. The objective function constrains the common semantic representations V1 and V2 to maintain higher-order relationships within and between modes.

[0071] Step 3: Measurement of Classification Loss for Labeled Data

[0072] For labeled data, this invention minimizes the classification label prediction loss, and the objective function is shown in formula (5).

[0073]

[0074] Where λ and γ represent hyperparameters. It is a projection matrix. Represents the selection matrix

[0075] Step 4: Construction and optimization of the overall objective function

[0076] Combining formulas (1), (2), (4), and (5) from the previous steps, a general objective function is formed, and the relevant variables are optimized, as shown in formula (6).

[0077]

[0078] The next step is to optimize the objective function. This invention uses an alternating iterative approach until convergence. Finally, the common semantic subspace is weighted and averaged for the image and text modalities to obtain the final latent semantic subspace, as shown in formula (7).

[0079] V=αV1+(1-α)V2 (7)

[0080] Step 5: Quantize rotation matrix learning

[0081] Using hash representation can save storage space and improve retrieval speed. Therefore, the common semantic space is converted into a binary hash code. Usually, the symbols of the common semantic representation are directly taken to obtain the hash code, which can easily introduce large quantization loss and large error. Therefore, this invention obtains the hash code by learning an orthogonal rotation matrix, forming the objective function as shown in formula (8).

[0082]

[0083] Where B represents the hash code, R represents the rotation matrix, and I k Let represent the identity matrix. Equation (8) is a typical Orthogonal Procrustes problem, which can be obtained through singular value decomposition. Similarly, iterative optimization is used to obtain the optimal solution.

[0084] Step 6: Learn the cross-modal projection matrix and construct the hash function.

[0085] Finally, the quantized orthogonal rotation matrix was learned through the common semantic representation. In order to quickly convert the test data into binary code, it is also necessary to learn the hash function of each modality and directly project the original features into the Hamming space. Therefore, for image modality data and text modality data, the objective function shown in Equation (9) is constructed.

[0086]

[0087] Where P1 and P2 represent projection matrices. After iterative optimization, P1 and P2 can be obtained as shown in formulas (10) and (11).

[0088]

[0089]

[0090] Therefore, finally for the image modal data x 1 The hash code is obtained as shown in formula (12):

[0091] h1 = sgn(RP1x 1 (12)

[0092] For text modal data x 2 The hash code is obtained as shown in formula (13):

[0093] h2=sgn(RP2x 2 (13)

[0094] After learning the hash function, this invention can obtain hash codes for different modalities, and then learn the encoding of the retrieval library, and quickly retrieve similar data of another modality through the Hamming distance of binary encoding.

[0095] Application examples:

[0096] The following describes the effectiveness of this invention by applying it to real image and text datasets crawled from the internet. The three datasets are Wikipedia, Pascal Sentence, and INRIA-Websearch. The Wikipedia dataset contains 2866 image-text pairs, with each image corresponding to a descriptive text, belonging to 10 categories. 2173 pairs were used as the training set to train the model of this invention, with only 75% of the data containing category labels. The remainder was used as the query set to implement image-to-text and text-to-image queries. First, 4096-dimensional CNN image features were extracted from all images, and 100-dimensional LDA text features were extracted from the paragraph text. Then, the overall objective function and hash function were constructed according to the aforementioned steps of the invention. Finally, the queried image and text were used to query another modality of data. The cross-referencing metric (mean precision, MAP) for image-to-text and text-to-image retrieval is shown in Table 1. The hash encoding lengths were set to 16 bits, 32 bits, 64 bits, and 128 bits to observe the application effects of different lengths.

[0097] As can be seen from the application results in Table 1, the method proposed in this invention achieves the best results in both image-to-text and text-to-image queries. It outperforms the current best methods at different hash encoding lengths.

[0098] Table 1. Semi-supervised cross-modal search results on the Wikipedia dataset (MAP@50)

[0099]

[0100] Furthermore, this invention also applied cross-modal retrieval to two other image and text datasets collected from the internet: Pascal Sentence and INRIA-Websearch. The Pascal Sentence dataset contains 1000 image-text pairs from 20 categories. Thirty pairs were randomly selected from each category as the training set, and the remainder as the test set. The image features were 4096-dimensional CNN features, and the text features were 100-dimensional LDA features. The INRIA-Websearch dataset contains 71478 image-text pairs from 353 categories. After removing irrelevant text pairs, a subset containing 14698 text pairs was obtained, of which 10332 pairs were used as the training set and 4366 pairs as the test set. The data features are the same as above.

[0101] Table 2. Semi-supervised cross-modal retrieval results on the Pascal Sentence dataset (MAP@50)

[0102]

[0103] Table 3. Semi-supervised cross-modal retrieval results on the INRIA-Websearch dataset (MAP@50)

[0104]

[0105] As can be seen from the results in Tables 2 and 3, the method proposed in this invention also achieves the best cross-modal retrieval application performance, obtaining optimal cross-modal retrieval results in most cases. Furthermore, on the INRIA-Websearch dataset, the method of this invention significantly outperforms the current state-of-the-art methods in different length hash encodings and different cross-modal retrieval tasks. Therefore, based on the overall application example effect analysis, the method proposed in this invention is effective for fully utilizing unlabeled image and text data for semi-supervised cross-modal image-text retrieval, and achieves superior cross-modal retrieval performance compared to previous methods.

[0106] In summary, the semi-supervised image-text cross-modal hashing method based on hybrid hypergraph regularization of the present invention can fully leverage the importance of unlabeled data in preserving semantic structure based on hybrid hypergraph and concept decomposition, improve the learning performance and retrieval capability of semi-supervised cross-modal retrieval models, and solve the problems of difficult and costly labeling of labeled data.

[0107] The examples described above are merely illustrative of embodiments of the present invention, but should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements are all within the scope of protection of the present invention.

Claims

1. A hybrid hypergraph regularized semi-supervised cross-modal hashing method based on concept decomposition, characterized in that, The concept is decomposed into a common semantic representation learning module; the hybrid hypergraph regularization fuses hypergraphs of image and text modalities for regularization constraint learning of common semantic representations; the semi-supervised cross-modal hashing method includes both labeled and unlabeled data; specifically, it includes the following steps: Step 1: Common semantic representation learning, based on concept decomposition, learns the inherent semantic representation of cross-modal data; make Representing the features of image modal data, This represents text modal data features, where each modality contains n samples, l of which are labeled; the image feature dimension is... ,Right now The text feature dimension is , Given labeled data , Indicates the number of data categories; the label vector uses 0 and 1 to represent them, when When , it means that the j-th sample belongs to the i-th class; otherwise, if it does not belong, then ; This paper proposes a concept decomposition-based method to learn the inherent semantic information of cross-modal data, which includes both labeled and unlabeled data. Concept decomposition allows the input data to contain negative numbers. Let k be the common semantic representation of the two modalities, where k is the dimension of the representation space. To relax the common semantic representation, the two representations are made as similar as possible, rather than constrained to a common representation. Therefore, the objective function of concept decomposition is shown in Equation (1): （1）； in, Representing the concept of an association matrix, recording data. These are the modal weight parameters. Let Frobenius norm be the expression; and then, by taking the closest possible representation of the common semantics, we form the objective function as shown in formula (2). （2）； Step 2: Construct a cross-modal hybrid hypergraph by mixing data from different modalities. We propose a hybrid hypergraph regularization constraint, which leverages hypergraphs to model the adjacency relationships between multiple samples and preserves high-order local relations within a modality. Simultaneously, through the construction of a hybrid hypergraph, we fuse high-order relations between two modalities to guide the learning of common semantic representations and improve discriminability. In other words, the cross-modal hybrid hypergraph can fuse high-order semantic relations within and between modalities. Step 3: Measurement of classification loss for labeled data. For labeled data, minimize the classification label prediction loss. For labeled data, the objective function is to minimize the classification label prediction loss, as shown in formula (5). （5）； in, , This represents hyperparameters. It is a projection matrix. This represents the selection matrix, and since unlabeled data has no labels, classification loss is only measured for labeled data. Step 4: Construction and optimization of the overall objective function, combining the previous steps to form the overall objective function; Combining formulas (1), (2), (4), and (5) from the previous steps, a general objective function is formed, and the relevant variables are optimized, as shown in formula (6). （6）； The objective function is optimized until convergence; finally, the common semantic subspace is weighted and averaged for the image modality and the text modality to obtain the final latent semantic subspace, as shown in formula (7); （7）； Step 5: Quantization rotation matrix learning. Orthogonal rotation matrices are learned through common semantic representations to reduce quantization loss. The public semantic space is transformed into a hash code in binary representation; the hash code is obtained by learning an orthogonal rotation matrix, forming the objective function as shown in formula (8); （8）； in, Represents hash encoding, Represents the rotation matrix. Let represent the identity matrix; Equation (8) is a typical Orthogonal Procrustes problem, which can be obtained through singular value decomposition. Similarly, iterative optimization is used to obtain the optimal solution. Step 6: Learn the cross-modal projection matrix and construct the hash function; Finally, the orthogonal rotation matrix of the quantization is learned through the public semantic representation. In order to enable the query image or text data to be quickly converted into binary code, it is also necessary to learn the hash function of image modality and text modality and directly project the original features into Hamming space. Therefore, for image modality data and text modality data, the objective function shown in formula (9) is constructed. （9）； in, , The projection matrix is ​​represented; it can be obtained through alternating iterative optimization. , The results are shown in formulas (10) and (11); （10）； （11）； Finally, regarding image modal data... The hash code is obtained as shown in formula (12): （12）； For text modal data The hash code is obtained as shown in formula (13): （13）； After learning about hash functions, we can obtain the hash code of a new query image or query text, and then calculate the text hash code that is closest to the hash code of the query image from the database to achieve image retrieval of text, or calculate the image hash code that is closest to the hash code of the query text from the database to achieve text retrieval of images.

2. The hybrid hypergraph regularized semi-supervised cross-modal hashing method based on concept decomposition according to claim 1, characterized in that, The specific steps of step 2 are as follows: First, based on image modal data Constructing an association matrix The construction method involves building a hyperedge for each node, with each hyperedge consisting of the node and its K most similar neighbors. The similarity is measured using Euclidean distance. Simultaneously, for text modalities, a corresponding association matrix is ​​constructed. Next, we begin constructing a hybrid hypergraph by splicing the hyperedges of the two modes together to form a hypergraph with 2n hyperedges, as shown in formula (3): （3）； Then, based on the hybrid hypergraph incidence matrix The Laplacian matrix of the hybrid hypergraph can be calculated. ; Therefore, the hybrid hypergraph Laplace regularization constraint can be defined as the objective function shown in Equation (4); （4）； in, These are weight parameters; the objective function can constrain the common semantic representation. and Preserve higher-order relationships within and between modes.

3. The hybrid hypergraph regularized semi-supervised cross-modal hashing method based on concept decomposition according to claim 1, characterized in that, The overall objective function in step 4 is optimized using an alternating iterative optimization method.

Images