Hashing image retrieval method based on width learning

Abstract

This invention discloses a hash-based image retrieval method based on width learning, comprising: 1) extracting features from an image set and recording the extracted two-dimensional vector data and corresponding label data in a dataset D; 2) assigning ideal hash codes to each category in dataset D based on Hadamard; 3) constructing and training a basic width hash network based on a width learning system to learn the relationship between the two-dimensional vector data and the ideal hash codes; 4) performing image retrieval using the trained basic width hash network and calculating the retrieval accuracy; 5) proposing two types of node-based incremental width hash networks and one type of data-based incremental width hash network for static and dynamic data environments, respectively. This invention alleviates the time consumption problem commonly found in deep hashing methods and the performance degradation caused by short hash codes, while improving image retrieval accuracy and enabling more flexible and accurate downstream applications.

Description

Technical Field

[0001] This invention relates to the technical field of hash retrieval and width learning, and in particular to a hash image retrieval method based on width learning. Background Technology

[0002] With the exponential growth of images and other multimedia data uploaded to the internet every second, content-based image retrieval has become a challenging task and has attracted widespread attention. Hash retrieval methods use a set of hash functions to map high-dimensional image content features to low-dimensional, compact binary hash codes. The similarity between two images is measured by comparing the Hamming distance between their binary hash codes. Hash methods use less memory overhead and achieve high retrieval efficiency, thus they are widely used in content-based image retrieval.

[0003] Current hashing methods can be divided into two categories: traditional hashing methods and deep hashing methods. Traditional hashing methods first use manual feature extraction to extract features from the image, then learn and train the hash function based on the semantic information reflected by the extracted features. However, the semantic information expressed by the image features extracted by manual feature extraction methods is limited, becoming a bottleneck for image retrieval using traditional hashing methods. Deep learning networks alleviate this bottleneck by extracting rich and representative features from complex, high-dimensional images; deep hashing methods extract features using deep learning networks. However, due to their complex deep learning network structure and numerous hyperparameters, training deep hashing networks is time-consuming and memory-intensive. Furthermore, because the number of neurons in the deep hashing network structure is significantly reduced from the penultimate layer to the final hash layer, retrieval performance degrades when short hash codes are used. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a hash image retrieval method based on width learning. This method can use a special network structure of width learning to learn image features, train more compact hash codes, and thus obtain deeper semantic information of the image. This alleviates the time consumption problem commonly found in deep hashing methods and the retrieval performance degradation caused by short hash codes. At the same time, it improves the image retrieval accuracy and can further enable more flexible and accurate downstream applications.

[0005] To achieve the above objectives, the technical solution provided by this invention is: a hash image retrieval method based on width learning, comprising the following steps:

[0006] 1) Use feature extraction methods to extract features from the image set, record the extracted two-dimensional vector data and corresponding label data in dataset D, and record the basic information of dataset D, including the number of categories l and the number of data samples n;

[0007] 2) Using the balanced hash table H generated by the Hadamard matrix, assign an ideal hash code to each category of dataset D based on the number of categories and label data of the dataset obtained in step 1);

[0008] 3) To learn the mapping relationship between the data features of dataset D and the ideal hash code, a novel wide hash network is constructed and trained based on the wide learning system. The wide hash network consists of a shallow mapping layer, an enhancement layer and a hash layer. The shallow mapping layer and the enhancement layer form the full feature layer.

[0009] 4) Use the trained width hashing network for image retrieval and calculate the retrieval accuracy to evaluate the current width hashing network's ability to retrieve images of the same category;

[0010] 5) Based on the width hash network trained in step 3), two types of node-based incremental width hash networks and one type of data-based incremental width hash network are proposed for static and dynamic data environments, respectively. In the static data environment, the two types of node-based incremental width hash networks are an enhancement layer node incremental width hash network and a shallow mapping layer node incremental width hash network, respectively. The network structure is adjusted based on the enhancement layer and the shallow mapping layer, and the width hash network is retrained. In the dynamic data environment, the data-based incremental width hash network adds a set of enhancement layer network nodes and shallow mapping layer network nodes to perform feature learning on the data newly entering the width hash network, without needing to repeat training on the old data.

[0011] Furthermore, in step 1), the feature extraction method refers to a method of converting three-dimensional image data into two-dimensional vector data. After extraction, the resulting vector set X is a two-dimensional matrix, represented as:

[0012]

[0013] In the formula, Let X represent a real number. i It refers to the two-dimensional vector obtained by feature extraction from the i-th image. Each two-dimensional vector has a dimension of 1×d, and there are n such two-dimensional vectors in total, which means that the original image set has n images.

[0014] The set of tags corresponding to each image, i.e., the tag set Y, is represented as:

[0015]

[0016] In the formula, Represents natural numbers, y i It refers to the category label corresponding to the i-th image, with a total of n labels, i∈[1,n];

[0017] The vector set X and the label set Y are merged and defined as dataset D, i.e., D = {X, Y}; based on the values ​​in the label set Y, the number of categories l and y in dataset D are obtained. i ∈[1,l].

[0018] Furthermore, in step 2), the Hadamard matrix is ​​a special type of square matrix whose elements are all -1 or +1; for an r×r Hadamard matrix H r Build it as follows:

[0019]

[0020] In the formula, matrix H r The order r satisfies r = {r|r = 2} τ ,r≥l}, τ is a natural number; at the same time, r represents the number of bits in the hash code, l is the number of categories in D, H1=[1]; H r Its product with its transpose satisfies:

[0021]

[0022] In the formula, H represents r The transpose matrix, where T represents the transpose sign, I r I represents the identity matrix of order r; r All off-diagonal elements are equal to 0, which indirectly confirms that H r The row and column orthogonality of the matrix; based on the orthogonality of the Hadamard matrix and the special properties of its values, Ha is used. r Generate a valid hash code;

[0023] For dataset D, choose H. r The first l rows form the hash table H l , because y i ∈[1,l], for label y i vector X i Assign H l The yth i Row as vector X i Ideal hash code U i Similarly, for label y j vector X j Assign H l The yth j Row as vector X j Ideal hash code U j Where j∈[1,n], then the Hamming distance dist between the two vectors is... ij The calculation is as follows:

[0024]

[0025] Therefore, after assigning the ideal hash code, when the width hash network trained based on the ideal hash code generates the hash code for the corresponding vector of the test image, as long as the number of erroneous bits is less than half the number of bits of the hash code, the Hamming distance between the hash code of the test image and the hash code of the correct category is minimized. In this case, the test image can still be correctly classified. This shows that the ideal hash code can fully contain the semantic information of the original image.

[0026] Furthermore, step 3) includes the following steps:

[0027] 3.1) Construct the full feature layer of the width hash network. The full feature layer of the width hash network consists of a shallow mapping layer and an enhancement layer. First, the vector set X is mapped a times, and each mapping produces α nodes. The specific mapping process is as follows:

[0028] S i' =f(XW si' +β si' ), i' = 1, 2, ..., a

[0029] In the formula, S i' W represents a set of nodes obtained by the i'th mapping of the vector set X. si' and β si' Let S represent the weights and biases of the randomly generated i'-th mapping, respectively, and f represent the activation function; merge the a groups of nodes obtained from a mappings into a shallow mapping layer, denoted as S. a =[S1,S2,...,S a ];

[0030] To obtain richer semantic information, the nodes in the shallow mapping layer are subjected to a secondary mapping to obtain enhanced nodes. The specific mapping process is as follows:

[0031] E j' =g(S a W ej' +β ej' ),j'=1,2,…,c

[0032] In the formula, E j' Indicates shallow mapping layer S a The set of nodes obtained by the j'th mapping is mapped c times in total, W ej' and β ej' Let J and G represent the weights and biases of the randomly generated j'-th mapping, respectively, and g represent the activation function. The c groups of nodes obtained from the c-th mapping are merged into an enhancement layer, denoted as E. c =[E1,E2,...,E c ];

[0033] Shallow mapping layer Sa With reinforcement layer E c Merged into a full feature layer, denoted as

[0034] 3.2) Construct the hash layer B of the width hash network. The hash layer consists of the corresponding hash code of each vector. The binary hash code matrix obtained after hashing through a hash network is specifically represented as follows:

[0035]

[0036] In the formula, W h Let B represent the weight matrix of the width hash network, i.e., the hash mapping matrix, which consists of r hash functions, i.e., r column vectors. Therefore, the resulting binary hash code matrix consists of n rows of r bits of hash code, represented as follows:

[0037]

[0038] In the formula, B i X represents i The hash code is obtained by first mapping to the full feature layer and then performing hash hashing.

[0039] In order to make W h To better learn the semantic information of the original image, different categories of data are hashed into different hash spaces. B should be as close as possible to the ideal hash code U. Therefore, when constructing a wide hash network, the ideal hash code U is used as the hash layer of the wide hash network.

[0040] 3.3) Calculate the width hash network weight matrix W h Reduce W is trained using the quantization loss generated when hashing from a high-dimensional space to a binary Hamming space. h That is, to minimize the quantization loss caused by sign(·), the loss function is expressed as follows:

[0041]

[0042] In the formula, λ represents a regularization parameter. This indicates the solution to W. h To minimize the value of the expression, based on ridge regression theory, the weight matrix W of the width hash network... h The calculation formula is as follows:

[0043]

[0044] In the formula, I is the identity matrix, and the width hash network constructed in this way is called the basic width hash network.

[0045] Furthermore, in step 4), image retrieval is performed using the trained width hashing network; the same feature extraction method as in step 1) is applied to the test image set, and the extracted test vector set is represented as X. test Input test vector set X test The trained wide hash network is used for testing to obtain the hash code B of the test vector set. test Calculate hash code B test The Hamming distances between the vector set X and its ideal hash code U are sorted. For each test vector, the vector in X with the smallest Hamming distance to its ideal hash code U is selected, and the corresponding image in the image set is found as the retrieval result. The correctness of the returned retrieval results is measured using the mean precision (MAP) as the evaluation criterion. The MAP is calculated as follows:

[0046]

[0047] In the formula, Q represents the number of test vectors in the test vector set, i.e., the number of test images in the test image set; n represents the number of vectors in the vector set X, i.e., the number of images in the image set; R q Represents the image set and the qth th The number of images of the same category as the test image, R qi Indicates the qth th When a test image is used to retrieve data from an image set, the number of images of the same category among the first i retrieved images is rel. qi This represents the relevance between the i-th retrieved image and the q-th test image. If the i-th retrieved image and the q-th test image are images of the same category, then rel... qi =1, otherwise rel qi =0.

[0048] Furthermore, in step 5), for static data environment and dynamic data environment, two types of node-type incremental width hash network and one type of data-type incremental width hash network are proposed for the width hash network constructed in step 3).

[0049] In static data environments, node-based incremental width hashing networks improve the image retrieval capabilities of the original width hashing network by adding network nodes to the full feature layer. Since the full feature layer of a width hashing network consists of a shallow mapping layer and an enhancement layer, the two node-based incremental width hashing networks address different components of the full feature layer: enhancement layer node incremental width hashing networks and shallow mapping layer node incremental width hashing networks. In dynamic data environments, data-based incremental width hashing networks specifically add a set of enhancement and shallow mapping layer network nodes to learn features from new data, eliminating the need for repeated training on old data. The three network structures are described in detail below:

[0050] ①Incremental Width Hash Network for Enhanced Layer Nodes

[0051] The newly added c0 group enhancement layer nodes are represented as follows:

[0052]

[0053] In the formula, and Let the weights and biases of the newly added k-th group be represented as follows: c2 represents the number of node groups in the enhancement layer after adding the enhancement layer node; the full feature layer is represented as... The pseudo-inverse matrix is ​​calculated as follows:

[0054]

[0055] In the formula, intermediate variables The calculation of the intermediate variable ω is expressed as follows:

[0056]

[0057] In the formula, intermediate variables

[0058] The network weight calculation for a width-hash network after the addition of nodes in the enhancement layer is as follows:

[0059]

[0060] In the formula, This represents the width hash network weights of the group of shallow mapping layer nodes (group a) and the group of enhancement layer nodes (group c). This indicates that after the addition of this enhancement layer node, it has the width hash network weight of group a shallow mapping layer nodes and group c2 enhancement layer nodes;

[0061] After increasing the number of nodes in the enhancement layer, step 4) is performed again. The hash code of the test vector set is recalculated using the newly constructed width hash network, and MAP is calculated to measure the image retrieval capability of the newly constructed width hash network. If the retrieval accuracy does not meet the satisfactory standard, c = c2 is set, and step ①) is repeated on the basis of the current width hash network structure. That is, the final width hash network is confirmed after the retrieval accuracy stabilizes.

[0062] (② Shallow mapping layer node incremental width hash network)

[0063] If adding enhancement layer nodes cannot stably and significantly improve MAP, then incremental width hashing is achieved by adding shallow mapping layer nodes; the newly added group a0 of shallow mapping layer nodes is represented as follows:

[0064]

[0065] In the formula, and This represents the weights and biases corresponding to the newly added t-th group of mapping layer nodes, thus forming a new shallow mapping layer. a1 represents the number of node groups in the shallow mapping layer after the addition of a new shallow mapping node;

[0066] For the newly added shallow mapping node group If enhancement is performed c times, the newly formed enhancement layer is represented as follows: Among them, E' c express The new enhancement layer nodes obtained by enhancing c times, c new This represents the number of groups of all enhanced nodes in the new enhanced layer; then the update of the full feature layer after adding the shallow mapping layer node group is expressed as: The pseudo-inverse matrix is ​​represented as:

[0067]

[0068] In the formula, intermediate variables The calculation of the intermediate variable ω1 is expressed as follows:

[0069]

[0070] In the formula, intermediate variables

[0071] The network weight calculation for a width-hash network after adding nodes is as follows:

[0072]

[0073] In the formula, This indicates that after the addition of this node, there are shallow mapping layer nodes in group a1 and c. new The width of the group enhancement layer node hash network weight;

[0074] After incrementing the shallow mapping layer nodes and corresponding enhancement layer nodes, step 4) is performed again. The hash codes of the test vector set are recalculated using the newly constructed width hash network, and the MAP is calculated to measure the image retrieval capability of the newly constructed width hash network. If the retrieval accuracy does not meet the satisfactory standard, let a = a1, c = c new Based on the current width hash network structure, repeat step ②, and confirm the final width hash network after the retrieval accuracy stabilizes.

[0075] ③ Data-driven incremental width hash network

[0076] In a dynamic data environment, the data environment of a width hash network is constantly changing, and the distribution of data features in the width hash network will also change accordingly; the newly added training data to the width hash network is denoted as X. θ and X θ Based on the width hash network structure and step 3), random mapping is performed to the shallow mapping layer and enhancement layer, then X... θ The newly added full feature layer nodes obtained by mapping Represented as:

[0077]

[0078] In the formula, and These represent the newly added shallow mapping layer node and the newly added enhancement layer node, respectively. The new full feature layer is represented as follows: Among them, a θ Indicates by X θ The number of node groups in the shallow mapping layer obtained by mapping is the number of newly added shallow mapping layer node groups, and a3 represents the total number of node groups in the new shallow mapping layer, i.e., a3 = a + a θ c θ Indicates by The number of enhancement layer node groups obtained by mapping is the number of newly added enhancement layer node groups, and c3 represents the total number of nodes in the new enhancement layer, i.e., c3 = c + c. θ Then the full feature layer The pseudo-inverse matrix is ​​calculated as follows:

[0079]

[0080] In the formula, intermediate variables The calculation of the intermediate variable ω2 is expressed as follows:

[0081]

[0082] In the formula, intermediate variables

[0083] After new data is input into the width hash network, the weight calculation of the width hash network is represented as follows:

[0084]

[0085] In the formula, This indicates the width hash network weights of the a3 group of shallow mapping layer nodes and the c3 group of enhancement layer nodes after this new input data;

[0086] After performing incremental hashing, proceed to step 4) again, recalculate the hash code of the test vector set using the newly trained width hashing network, and measure the accuracy of test vector set retrieval. If there is new data input, let a = a3, c = c3, and repeat step 3 based on the current width hashing network structure. After there is no new data input, confirm it as the final width hashing network.

[0087] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0088] 1. The method of this invention is the first to apply a wide network to the field of hash image retrieval. Compared with other deep hashing methods, it greatly reduces training time and improves image retrieval efficiency.

[0089] 2. Compared with other traditional hash methods, the method of the present invention improves the accuracy of image retrieval.

[0090] 3. This invention proposes three incremental width hashing networks based on basic width hashing. On the existing width hashing network, image retrieval accuracy can be improved by adding more width hashing network nodes; when new data is input into the network, the width hashing network structure can be adjusted to learn the features of the new data and adapt to changes in the data environment.

[0091] 4. The method of the present invention has a wide range of applications in image retrieval tasks. It is simple to operate, highly adaptable, and has broad application prospects.

[0092] In summary, this invention utilizes a width-learning system to train hash codes for image retrieval. This system employs a unique network structure to learn image features, training more compact hash codes to obtain deeper semantic information from the image. This invention alleviates the time-consuming issues common in deep hashing methods and the performance degradation caused by short hash codes, while simultaneously improving image retrieval accuracy. Furthermore, it enables more flexible and precise downstream applications. Attached Figure Description

[0093] Figure 1 This is a schematic diagram of the logic flow of the present invention.

[0094] Figure 2 This is a diagram of the basic width hash network structure.

[0095] Figure 3 This is a diagram of a width hash network structure based on incremental hashing of enhancement layer nodes.

[0096] Figure 4 This is a diagram of a width hash network structure based on incremental hashing using a shallow mapping layer.

[0097] Figure 5This is a diagram of the incremental width hash network structure in a dynamic data environment. Detailed Implementation

[0098] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the description method of the present invention is not limited thereto.

[0099] like Figure 1 As shown, this embodiment discloses a hash image retrieval method based on width learning, which includes the following steps:

[0100] 1) The GIST feature extraction method is used to extract features from the Cifar10 3D image set, converting each 3D image data into a 2D vector data. All 2D vectors are combined into a vector set, denoted as X, and represented as:

[0101]

[0102] in, Let X represent a real number. i It refers to the two-dimensional vector obtained by feature extraction from the i-th image. Each two-dimensional vector has a dimension of 1×d, and there are n such two-dimensional vectors in total, that is, Cifar10 has n images;

[0103] Correspondingly, the set of tags corresponding to each image, i.e., the tag set Y, is represented as:

[0104]

[0105] in Represents natural numbers, y i It refers to the category label corresponding to the i-th image, with a total of n labels, i∈[1,n];

[0106] For the 3D image set Cifar10, the vector set X obtained by using the GIST feature extraction method is merged with the label set Y to obtain the dataset D, i.e., D = {X, Y}, where n = 59000, d = 512, l = 10, and y i ∈[1,10].

[0107] 2) Construct an r×r Hadamard matrix H as follows: r H r All elements are either -1 or +1:

[0108]

[0109] In the formula, matrix H r The order r satisfies r = {r|r = 2} τ ,r≥l}, τ is a natural number; at the same time, r represents the number of bits in the hash code, l is the number of categories in D, H1=[1]; Hr Its product with its transpose satisfies:

[0110]

[0111] In the formula, H represents r The transpose matrix, where T represents the transpose sign, I r I represents the identity matrix of order r; r All off-diagonal elements are equal to 0, which indirectly confirms that H r The row and column orthogonality of the matrix; based on the orthogonality of the Hadamard matrix and the special properties of its values, Ha is used. r Generate a valid hash code; at this point, r also represents the number of bits in the hash code, r = 32.

[0112] For dataset D, choose H. r The first 10 rows form the hash table H l , because y i ∈[1,10], for label y i vector X i Assign H l The yth i Row as vector X i Ideal hash code U i Similarly, for label y j vector X j Assign H l The yth j Row as vector X j Ideal hash code U j Where j∈[1,n], then the Hamming distance dist between the two vectors is... ij The calculation is as follows:

[0113]

[0114] Therefore, after assigning the ideal hash code, the width hash network trained based on the ideal hash code can generate a hash code for the corresponding vector of the test image. As long as the number of erroneous bits is less than 16, the Hamming distance between the hash code of the test image and the hash code of the correct class is minimized. In this case, the test image can still be correctly classified. This shows that the ideal hash code can fully contain the semantic information of the original image.

[0115] 3) such as Figure 2 As shown, a wide hash network is constructed to learn the relationship between a vector set X and a label set Y, and to learn the semantic features of the vectors in X. The wide hash network consists of a shallow mapping layer, an enhancement layer, and a hash layer, with the shallow mapping layer and the enhancement layer forming the full feature layer. The construction of the wide hash network includes the following steps:

[0116] 3.1) Construct the full feature layer of the width hash network. The full feature layer of the width hash network consists of a shallow mapping layer and an enhancement layer. First, X is mapped a times, and each mapping produces α nodes. The specific mapping process is as follows:

[0117] S i' =f(XW si' +β si' ), i' = 1, 2, ..., a

[0118] In the formula, S i' W represents a set of nodes obtained by the i'th mapping of the vector set X. si' and β si' Let S represent the weights and biases of the randomly generated i'-th mapping, respectively, and f represent the activation function; merge the a groups of nodes obtained from a mappings into a shallow mapping layer, denoted as S. a =[S1,S2,…,S a Here, a is set to 100 and α to 20;

[0119] To obtain richer semantic information, the nodes in the shallow mapping layer are subjected to a secondary mapping to obtain the nodes in the enhancement layer. The specific mapping process is as follows:

[0120] E j' =g(S a W ej' +β ej' ),j'=1,2,…,c

[0121] In the formula, E j' Indicates shallow mapping layer S a The set of nodes obtained by the j'th mapping is mapped c times in total, W ej' and β ej' Let E represent the weights and biases of the randomly generated j'-th mapping, respectively, and g represent the activation function; the c groups of nodes obtained from the c mappings are merged into an enhancement layer, denoted as E. c =[E1,E2,…,E c Here, c is set to 1000;

[0122] Shallow mapping layer S a With reinforcement layer E c Merged into a full feature layer, denoted as

[0123] 3.2) Construct the hash layer B of the width-based hash network. The hash layer consists of the hash code of each vector X, which is F. a c The binary hash code matrix obtained after hashing through a hash network is specifically represented as follows:

[0124]

[0125] In the formula, W h The width hash network weight matrix, also known as the hash mapping matrix, consists of r hash functions, i.e., r column vectors. Therefore, the resulting binary hash code matrix consists of n rows of r bits of hash code, represented by B as follows:

[0126]

[0127] In the formula, B i X represents i The hash code is obtained by first mapping to the full feature layer and then performing hash hashing.

[0128] In order to make W h To better learn the semantic information of the original image, different categories of data are hashed into different hash spaces. B should be as close as possible to the ideal hash code U. Therefore, when constructing a width hash network, the ideal hash code U is used as the hash layer of the width hash network.

[0129] 3.3) Calculate the width hash network weight matrix W h Reduce W is trained using the quantization loss generated when hashing from a high-dimensional space to a binary Hamming space. h That is, to minimize the quantization loss caused by sign(·), the loss function is expressed as follows:

[0130]

[0131] In the formula, λ represents a regularization parameter. This indicates the solution to W. h To minimize the value of the expression, based on ridge regression theory, the weight matrix W of the width hash network... h The calculation formula is as follows:

[0132]

[0133] In the formula, I is the identity matrix, and the width hash network constructed in this way is called the basic width hash network.

[0134] 4) Image retrieval is performed using a trained width-hash network. The GIST feature extraction method is also used on the test image set, and the extracted test vector set is represented as X. test Input test vector set X test The trained wide hash network is used for testing to obtain the hash code B of the test vector set. test Calculate hash code B testThe Hamming distance between each test vector and the ideal hash code U of the vector set X is calculated and sorted. For each test vector, the vector in X with the smallest Hamming distance to its ideal hash code U is selected, and the corresponding image in the image set is found as the retrieval result. The correctness of the returned retrieval results is measured using the mean precision (MAP) as the evaluation criterion. The MAP is calculated as follows:

[0135]

[0136] In the formula, Q represents the number of test vectors in the test vector set, i.e., the number of test images in the test image set; n represents the number of vectors in the vector set X, i.e., the number of images in the image set; R q Represents the image set and the qth th The number of images of the same category as the test image, R qi Indicates the qth th When a test image is used to retrieve data from an image set, the number of images of the same category among the first i retrieved images is rel. qi This represents the relevance between the i-th retrieved image and the q-th test image. If the i-th retrieved image and the q-th test image are images of the same category, then rel... qi =1, otherwise rel qi =0.

[0137] 5) For static and dynamic data environments, two types of node-type incremental width hash networks and one type of data-type incremental width hash network are proposed for the basic width hash network constructed in step 3).

[0138] In static data environments, node-based incremental width hashing networks improve the image retrieval capabilities of the original basic width hashing network by adding network nodes to the full feature layer. Since the full feature layer of a width hashing network consists of a shallow mapping layer and an enhancement layer, the two node-based incremental width hashing networks address different components of the full feature layer: an enhancement layer node incremental width hashing network and a shallow mapping layer node incremental width hashing network. In dynamic data environments, data-based incremental width hashing networks specifically add a set of enhancement and shallow mapping layer network nodes to learn features from new data in the network, eliminating the need for repeated training on old data. The three network structures are described in detail below:

[0139] ①For example Figure 3 As shown, the incremental width hash network of the enhancement layer nodes is described as follows:

[0140] The newly added c0 group enhancement layer nodes are represented as follows:

[0141]

[0142] In the formula, and Let the weights and biases of the newly added k-th group be represented as follows: c2 represents the number of node groups in the enhancement layer after adding the enhancement layer node; here, c0 is set to 100. Correspondingly, the full feature layer is represented as... The pseudo-inverse matrix is ​​calculated as follows:

[0143]

[0144] In the formula, intermediate variables The calculation of the intermediate variable ω is expressed as follows:

[0145]

[0146] In the formula, intermediate variables

[0147] The network weight calculation for the width hash network after the addition of nodes in the enhancement layer is as follows:

[0148]

[0149] In the formula, The width hash network weight W represents the sum of the number of shallow mapping layer nodes (group a) and enhancement layer nodes (group c). h , This indicates that after the addition of this enhancement layer node, it has the width hash network weight of group a shallow mapping layer nodes and group c2 enhancement layer nodes.

[0150] After incrementing the nodes of the enhancement layer, step 4) is performed again. The hash code of the test vector set is recalculated using the newly constructed width hash network, and the MAP is calculated to measure the image retrieval capability of the newly constructed width hash network. If the retrieval accuracy does not meet the satisfactory standard, let c = c2, and repeat step ① on the basis of the current width hash network structure. That is, the final width hash network is confirmed after the retrieval accuracy is stable.

[0151] ②For example Figure 4 As shown, the shallow mapping layer node incremental width hash network is described as follows:

[0152] If adding enhancement layer nodes cannot stably and significantly improve MAP, incremental width hashing can be achieved by adding shallow mapping layer nodes; the newly added group a0 of shallow mapping layer nodes is represented as follows:

[0153]

[0154] In the formula, and This represents the weights and biases corresponding to the newly added t-th group of mapping layer nodes, thus forming a new shallow mapping layer. a1 represents the number of node groups in the shallow mapping layer after adding a new shallow mapping node; here, a0 is set to 100.

[0155] Correspondingly, for the newly added shallow mapping node group If enhancement is performed c times, the newly formed enhancement layer is represented as follows: Among them, E' c express The new enhancement layer nodes obtained by enhancing c times, c new This represents the number of groups of all enhanced nodes in the new enhanced layer; then the update of the full feature layer after adding the shallow mapping layer node group is expressed as: The pseudo-inverse matrix is ​​represented as:

[0156]

[0157] Among them, intermediate variables The calculation of the intermediate variable ω1 is expressed as follows:

[0158]

[0159] Among them, intermediate variables

[0160] The network weight calculation for the width hash network after adding nodes is as follows:

[0161]

[0162] in, This indicates that after the addition of this node, there are shallow mapping layer nodes in group a1 and c. new The width of the group enhancement layer node hash network weight;

[0163] After incrementing the shallow mapping layer nodes and corresponding enhancement layer nodes, step 4) is performed again. The hash codes of the test vector set are recalculated using the newly constructed width hash network, and the MAP is calculated to measure the image retrieval capability of the newly constructed width hash network. If the retrieval accuracy does not meet the satisfactory standard, let a = a1, c = c new Based on the current width hash network structure, repeat step ②, and confirm the final width hash network after the retrieval accuracy stabilizes.

[0164] ③ For example Figure 5 As shown, the data-based incremental width hash network is described as follows:

[0165] In a dynamic data environment, the data environment of a width hash network is constantly changing, and the distribution of data features in the width hash network will also change accordingly; the newly added training data to the width hash network is denoted as X. θ and Xθ Based on the width hash network structure and step 3), random mapping is performed to the shallow mapping layer and enhancement layer, then X... θ The newly added full feature layer node obtained by mapping is represented as follows:

[0166]

[0167] In the formula, and These represent the newly added shallow mapping layer node and the newly added enhancement layer node, respectively. The new full feature layer is represented as follows: Among them, a θ Indicates by X θ The number of node groups in the shallow mapping layer obtained by mapping is the number of newly added shallow mapping layer node groups, and a3 represents the total number of node groups in the new shallow mapping layer, i.e., a3 = a + a θ c θ Indicates by The number of enhancement layer node groups obtained by mapping is the number of newly added enhancement layer node groups, and c3 represents the total number of nodes in the new enhancement layer, i.e., c3 = c + c. θ Then the full feature layer The pseudo-inverse matrix is ​​calculated as follows:

[0168]

[0169] In the formula, intermediate variables The calculation of the intermediate variable ω2 is expressed as follows:

[0170]

[0171] In the formula, intermediate variables

[0172] After new data is input into the width hash network, the weight calculation of the width hash network is represented as follows:

[0173]

[0174] In the formula, This indicates the width hash network weights of the a3 group of shallow mapping layer nodes and the c3 group of enhancement layer nodes after this new input data.

[0175] After performing incremental hashing, proceed to step 4) again, recalculate the hash code of the test vector set using the newly trained width hashing network, and measure the accuracy of test vector set retrieval. If there is new data input, let a = a3, c = c3, and repeat step 3 based on the current width hashing network structure. After there is no new data input, confirm it as the final width hashing network.

[0176] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A hash image retrieval method based on width learning, characterized in that, Includes the following steps: 1) Use feature extraction methods to extract features from the image set, record the extracted two-dimensional vector data and corresponding label data in dataset D, and record the basic information of dataset D, including the number of categories l and the number of data samples n; 2) Using the balanced hash table H generated by the Hadamard matrix, assign an ideal hash code to each category of dataset D based on the number of categories and label data of the dataset obtained in step 1); 3) To learn the mapping relationship between the data features of dataset D and the ideal hash code, a novel wide hash network is constructed and trained based on the wide learning system. The wide hash network consists of a shallow mapping layer, an enhancement layer and a hash layer. The shallow mapping layer and the enhancement layer form the full feature layer. 4) Use the trained width hashing network for image retrieval and calculate the retrieval accuracy to evaluate the current width hashing network's ability to retrieve images of the same category; 5) Based on the width hash network trained in step 3), two types of node-based incremental width hash networks and one type of data-based incremental width hash network are proposed for static and dynamic data environments, respectively. In the static data environment, the two types of node-based incremental width hash networks are an enhancement layer node incremental width hash network and a shallow mapping layer node incremental width hash network, respectively. The network structure is adjusted based on the enhancement layer and the shallow mapping layer, and the width hash network is retrained. In the dynamic data environment, the data-based incremental width hash network adds a set of enhancement layer network nodes and shallow mapping layer network nodes to perform feature learning on the data newly entering the width hash network, without needing to repeat training on the old data.

2. The hash image retrieval method based on width learning according to claim 1, characterized in that, In step 1), the feature extraction method refers to the method of converting three-dimensional image data into two-dimensional vector data. After extraction, the resulting vector set X is a two-dimensional matrix, represented as: In the formula, Let X represent a real number. i It refers to the two-dimensional vector obtained by feature extraction from the i-th image. Each two-dimensional vector has a dimension of 1×d, and there are n such two-dimensional vectors in total, which means that the original image set has n images. The set of tags corresponding to each image, i.e., the tag set Y, is represented as: In the formula, Represents natural numbers, y i It refers to the category label corresponding to the i-th image, with a total of n labels, i∈[1,n]; The vector set X and the label set Y are merged and defined as dataset D, i.e., D = {X, Y}; based on the values ​​in the label set Y, the number of categories l and y in dataset D are obtained. i ∈[1,l].

3. The hash image retrieval method based on width learning according to claim 2, characterized in that, In step 2), the Hadamard matrix is ​​a special type of square matrix whose elements are all -1 or +1; for an r×r Hadamard matrix H r Build it as follows: In the formula, matrix H r The order r satisfies r = {r|r = 2} τ ,r≥l}, τ is a natural number; at the same time, r represents the number of bits in the hash code, l is the number of categories in D, H1=[1]; H r Its product with its transpose satisfies: In the formula, H represents r The transpose matrix, where T represents the transpose sign, I r I represents the identity matrix of order r; r All off-diagonal elements are equal to 0, which indirectly confirms that H r The row and column orthogonality of the matrix; based on the orthogonality of the Hadamard matrix and the special properties of its values, Ha is used. r Generate a valid hash code; For dataset D, choose H. r The first l rows form the hash table H l , because y i ∈[1,l], for label y i vector X i Assign H l The yth i Row as vector X i Ideal hash code U i For label y j vector X j Assign H l The yth j Row as vector X j Ideal hash code U j Where j∈[1,n], then the Hamming distance dist between the two vectors is... ij The calculation is as follows:

4. The hash image retrieval method based on width learning according to claim 3, characterized in that, Step 3) includes the following steps: 3.1) Construct the full feature layer of the width hash network. The full feature layer of the width hash network consists of a shallow mapping layer and an enhancement layer. First, the vector set X is mapped a times, and each mapping produces α nodes. The specific mapping process is as follows: S i′ =f(XW si′ +β si′ ),i'=1,2,…,a In the formula, S i′ W represents a set of nodes obtained by the i′-th mapping of vector set X. si′ and β si′ Let S represent the weights and biases of the randomly generated i′-th mapping, respectively, and f represent the activation function; merge the a groups of nodes obtained from a mappings into a shallow mapping layer, denoted as S. a =[S1,S2,…,S a ]; The nodes in the shallow mapping layer are then subjected to a secondary mapping to obtain enhanced nodes. The specific mapping process is as follows: E j′ =g(S a W ej′ +b ej′ ),j′=1,2,…,c In the formula, E j′ Indicates shallow mapping layer S a The set of nodes obtained by the j′th mapping is mapped c times in total, W ej′ and β ej′ Let J and G represent the weights and biases of the randomly generated j′-th mapping, respectively, and g represent the activation function. The c groups of nodes obtained from the c-th mapping are merged into an enhancement layer, denoted as E. c =[E1,E2,…,E c ]; Shallow mapping layer S a With reinforcement layer E c Merged into a full feature layer, denoted as 3.2) Construct the hash layer B of the width hash network. The hash layer consists of the corresponding hash code of each vector. The binary hash code matrix obtained after hashing through a hash network is specifically represented as follows: In the formula, W h Let B represent the weight matrix of the width hash network, i.e., the hash mapping matrix, which consists of r hash functions, i.e., r column vectors. Therefore, the resulting binary hash code matrix consists of n rows of r bits of hash code, represented as follows: In the formula, B i X represents i The hash code is obtained by first mapping to the full feature layer and then performing hash hashing. When constructing a width hash network, the ideal hash code U is used as the hash layer of the width hash network; 3.3) Calculate the width hash network weight matrix W h Reduce W is trained using the quantization loss generated when hashing from a high-dimensional space to a binary Hamming space. h That is, to minimize the quantization loss caused by sign(·), the loss function is expressed as follows: In the formula, λ represents a regularization parameter. This indicates the solution to W. h To minimize the value of the expression, based on ridge regression theory, the weight matrix W of the width hash network... h The calculation formula is as follows: In the formula, I is the identity matrix, and the width hash network constructed in this way is called the basic width hash network.

5. The hash image retrieval method based on width learning according to claim 4, characterized in that, In step 4), image retrieval is performed using the trained width hashing network; the same feature extraction method as in step 1) is used on the test image set, and the extracted test vector set is represented as X. test Input test vector set X test The trained wide hash network is used for testing to obtain the hash code B of the test vector set. test Calculate hash code B test The Hamming distances between the vector set X and its ideal hash code U are sorted. For each test vector, the vector in X with the smallest Hamming distance to its ideal hash code U is selected, and the corresponding image in the image set is found as the retrieval result. The correctness of the returned retrieval results is measured using the mean precision (MAP) as the evaluation criterion. The MAP is calculated as follows: In the formula, Q represents the number of test vectors in the test vector set, i.e., the number of test images in the test image set; n represents the number of vectors in the vector set X, i.e., the number of images in the image set; R q Represents the image set and the qth th The number of images of the same category as the test image, R qi Indicates the qth th When a test image is used to retrieve data from an image set, the number of images of the same category among the first i retrieved images is rel. qi This represents the relevance between the i-th retrieved image and the q-th test image. If the i-th retrieved image and the q-th test image are images of the same category, then rel... qi =1, otherwise rel qi =0.

6. The hash image retrieval method based on width learning according to claim 5, characterized in that, In step 5), for static data environment and dynamic data environment, two types of node-type incremental width hash network and one type of data-type incremental width hash network are proposed for the width hash network constructed in step 3). In static data environments, node-based incremental width hashing networks improve the image retrieval capabilities of the original width hashing network by adding network nodes to the full feature layer. Since the full feature layer of a width hashing network consists of a shallow mapping layer and an enhancement layer, the two node-based incremental width hashing networks address different components of the full feature layer: enhancement layer node incremental width hashing networks and shallow mapping layer node incremental width hashing networks. In dynamic data environments, data-based incremental width hashing networks specifically add a set of enhancement and shallow mapping layer network nodes to learn features from new data, eliminating the need for repeated training on old data. The three network structures are described in detail below: ①Incremental Width Hash Network for Enhanced Layer Nodes The newly added c0 group enhancement layer nodes are represented as follows: In the formula, and Let the weights and biases of the newly added k-th group be represented as follows: c2 represents the number of node groups in the enhancement layer after the addition of the enhancement layer node; The full feature layer is represented as The pseudo-inverse matrix is ​​calculated as follows: In the formula, intermediate variables The calculation of the intermediate variable ω is expressed as follows: In the formula, intermediate variables The network weight calculation for a width-hash network after the addition of nodes in the enhancement layer is as follows: In the formula, This represents the width hash network weights of the group of shallow mapping layer nodes (group a) and the group of enhancement layer nodes (group c). This indicates that after the addition of this enhancement layer node, it has the width hash network weight of group a shallow mapping layer nodes and group c2 enhancement layer nodes; After increasing the number of nodes in the enhancement layer, step 4) is performed again. The hash code of the test vector set is recalculated using the newly constructed width hash network, and MAP is calculated to measure the image retrieval capability of the newly constructed width hash network. If the retrieval accuracy does not meet the satisfactory standard, c = c2 is set, and step ① is repeated on the basis of the current width hash network structure. That is, the final width hash network is confirmed after the retrieval accuracy stabilizes. ② Shallow mapping layer node incremental width hash network If adding enhancement layer nodes cannot stably and significantly improve MAP, then incremental width hashing is achieved by adding shallow mapping layer nodes; the newly added group a0 of shallow mapping layer nodes is represented as follows: In the formula, and This represents the weights and biases corresponding to the newly added t-th group of mapping layer nodes, thus forming a new shallow mapping layer. a1 represents the number of node groups in the shallow mapping layer after the addition of a new shallow mapping node; For the newly added shallow mapping node group If enhancement is performed c times, the newly formed enhancement layer is represented as follows: Among them, E ′c express The new enhancement layer nodes obtained by enhancing c times, c new This represents the number of groups of all enhanced nodes in the new enhanced layer; then the update of the full feature layer after adding the shallow mapping layer node groups is expressed as: The pseudo-inverse matrix is ​​represented as: In the formula, intermediate variables The calculation of the intermediate variable ω1 is expressed as follows: In the formula, intermediate variables The network weight calculation for a width-hash network after adding nodes is as follows: In the formula, This indicates that after the addition of this node, there are shallow mapping layer nodes in group a1 and c. new The width of the group enhancement layer node hash network weight; After incrementing the shallow mapping layer nodes and corresponding enhancement layer nodes, step 4) is performed again. The hash codes of the test vector set are recalculated using the newly constructed width hash network, and the MAP is calculated to measure the image retrieval capability of the newly constructed width hash network. If the retrieval accuracy does not meet the satisfactory standard, let a = a1, c = c new Based on the current width hash network structure, repeat step ②, and confirm the final width hash network after the retrieval accuracy stabilizes. ③ Data-driven incremental width hash network In a dynamic data environment, the data environment of a width hash network is constantly changing, and the distribution of data features in the width hash network will also change accordingly; the newly added training data to the width hash network is denoted as X. θ and X θ Based on the width hash network structure and step 3), random mapping is performed to the shallow mapping layer and enhancement layer, then X... θ The newly added full feature layer nodes obtained by mapping Represented as: In the formula, and These represent the newly added shallow mapping layer node and the newly added enhancement layer node, respectively. The new full feature layer is represented as follows: Among them, a θ Indicates by X θ The number of node groups in the shallow mapping layer obtained by mapping is the number of newly added shallow mapping layer node groups, and a3 represents the total number of node groups in the new shallow mapping layer, i.e., a3 = a + a θ c θ Indicates by The number of enhancement layer node groups obtained by mapping is the number of newly added enhancement layer node groups, and c3 represents the total number of nodes in the new enhancement layer, i.e., c3 = c + c. θ Then the full feature layer The pseudo-inverse matrix is ​​calculated as follows: In the formula, intermediate variables The calculation of the intermediate variable ω2 is expressed as follows: In the formula, intermediate variables After new data is input into the width hash network, the weight calculation of the width hash network is represented as follows: In the formula, This indicates the width hash network weights of the a3 group of shallow mapping layer nodes and the c3 group of enhancement layer nodes after this new input data; After performing incremental hashing, proceed to step 4) again, recalculate the hash code of the test vector set using the newly trained width hashing network, and measure the accuracy of test vector set retrieval. If there is new data input, let a = a3, c = c3, and repeat step 3 based on the current width hashing network structure. After there is no new data input, confirm it as the final width hashing network.

Images