A hierarchical retrieval method for class-imbalanced images based on deep hashing

By using a class imbalanced image hierarchical retrieval method with deep hash in image retrieval, the lightweight network and the structure of the fully connected layer are used, combined with Focal Loss loss function and transfer learning, the low retrieval efficiency and model parameters in image retrieval are solved. The problem of large quantities has achieved efficient and accurate image retrieval and convenient model transplantation.

CN115080781BActive Publication Date: 2024-11-12TONGJI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210571972.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-24
Publication Date
2024-11-12
Estimated Expiration
2042-05-24

AI Technical Summary

Technical Problem

Existing image retrieval technology is inefficient in retrieval and takes up a large space in large-scale image data, especially the 'dimensional disaster' problem caused by the high dimension of image feature vectors, and the large amount of deep learning model parameters leads to inconvenience in model transplantation and storage.

Method used

Using a class imbalanced image hierarchical retrieval method based on deep hash, the model parameters are reduced by lightening the network and removing the fully connected layer, using 1*1 convolution kernel and global average pooling operations or blocked fully connected layer, combined with Focal Loss The loss function handles data imbalance problem and uses transfer learning and fine-tuning to accelerate training.

Benefits of technology

While ensuring the retrieval accuracy, the amount of model parameters is reduced, the convenience of model transplantation and storage is improved, the problems of slow retrieval speed and large space occupancy are solved, and the efficiency and accuracy of image retrieval is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115080781B_ABST
    Figure CN115080781B_ABST
Patent Text Reader

Abstract

The present invention relates to a class-imbalanced image hierarchical retrieval method based on deep hashing, comprising: S1, obtaining a training data set and inputting it into a feature extraction module of a network model, and obtaining a high-dimensional feature vector of the image after being processed by multiple convolutional layers; S2, the high-dimensional feature vector is input into a hidden layer of the network model, and the hidden layer learns the feature representation of the image according to the high-dimensional feature vector, and outputs an approximate hash code corresponding to the image; S3, the approximate hash code is processed by an activation function and quantization to form a discrete binary string, which is input into the classification layer of the hidden layer as the final hash code, and the classification layer calculates the corresponding image in the gallery through a secondary retrieval method to complete the training of the classification model; S4, the image to be retrieved is input into the trained classification model, and the image corresponding to the image to be retrieved in the gallery is output. Compared with the prior art, the present invention has the advantages of improving image retrieval accuracy, reducing the amount of model parameters, and making model transplantation and storage more convenient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a class-imbalanced image hierarchical retrieval method based on deep hashing. Background Art

[0002] In the 21st century, the world has entered the "Internet era". The development of the Internet has provided great convenience to human life. The emergence of various social software and e-commerce platforms has enabled people to share what they see with their friends and family and to buy good things from all over the world without leaving home. The rapid development of these software has also led to a rapid increase in the amount of data related to images and videos on the Internet. According to relevant data, the number of pictures uploaded by users around the world on social platforms every day is close to 1 billion, and more than 28.6 billion pictures are stored in online shopping platforms. In this case, how to enable each user to quickly find the desired image from large-scale image data has become a difficult problem that needs to be solved urgently. The solution to this problem is to use image retrieval technology.

[0003] The main process of image retrieval technology first needs to find a quantifiable expression of the image, then use a distance measurement method to compare the image to be retrieved with the images in the image library one by one, obtain the similarity measurement between them, and then sort and output the corresponding images according to the measurement results, that is, obtain the image retrieval results. In the early days of the development of image retrieval technology, the text-based image retrieval method (TBIR) was mainly used. The main idea is to first express the content of the image with text, that is, to manually annotate the image. The subsequent image retrieval process is actually the process of matching the annotated content. After finding the matching image annotation according to the search content, the corresponding image data is found according to the annotation. This method is simple to operate and has high retrieval efficiency. However, when the scale of image data increases sharply, it is extremely labor-intensive and material-intensive to annotate images, and the content of image annotations is controlled by human subjective consciousness. Different people may have different views on the same image. Therefore, the content-based image retrieval method (CBIR) gradually replaced the text-based image retrieval method. This method first obtains the feature representation of the image, and then performs image retrieval operations by comparing the similarities between the features. This method of using content retrieval can be figuratively described as the process of "searching images by image". This method is now widely used in search engines, e-commerce platforms and other scenarios.

[0004] Image retrieval technology is not only applicable to the above scenarios, it also plays an important role in many other aspects: 1) Public security and criminal investigation fields: it can be used to compare suspect photos, retrieve fingerprints, etc.; 2) Smart medical fields: early disease self-detection systems, auxiliary doctor diagnosis, similar case queries, etc.; 3) Traffic management: vehicle retrieval and identification, vehicle tracking, etc.

[0005] However, due to the increase in image quality and quantity, the dimension of image feature vectors is getting higher and higher, and the number is also increasing, which leads to a rapid decrease in the speed of content-based image retrieval and a rapid increase in the space occupied, which is called the "curse of dimensionality". In order to solve the above problems, a hash-based retrieval method was proposed, which can map the high-dimensional feature vector of the image into a low-dimensional binary string through a hash function. Using this hash code for image retrieval can speed up the speed and save space.

[0006] In recent years, the rapid growth of computer GPU computing power has promoted the development of deep learning technology. Especially after the emergence of AlexNet in 2012, deep learning technology has made breakthrough progress in the field of computer vision, including image classification, object detection, image segmentation and other tasks. Researchers have also used it in image retrieval tasks, combining deep learning and hash coding technology to propose an image retrieval method based on deep hashing, which has produced very good results, improving both retrieval accuracy and speed.

[0007] In summary, image retrieval technology has broad application scenarios and research value today. The deep hash-based image retrieval method combined with hash coding and deep learning technology can make full use of big data resources to achieve high accuracy, fast speed, and small space usage, accelerate the process of artificial intelligence application from theory to implementation, and provide convenience for people's lives. Summary of the invention

[0008] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a class-imbalanced image hierarchical retrieval method based on deep hashing. While ensuring the retrieval accuracy, the method reduces the amount of model parameters as much as possible by using a lightweight network and removing the fully connected layer, making model transplantation and storage more convenient.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] A class-imbalanced image hierarchical retrieval method based on deep hashing specifically comprises the following steps:

[0011] S1, obtain the training data set and input it into the feature extraction module of the network model, and obtain the high-dimensional feature vector of the image after processing through multiple convolutional layers;

[0012] S2, the high-dimensional feature vector is input into the hidden layer of the network model. The hidden layer learns the feature representation of the image based on the high-dimensional feature vector and outputs the approximate hash code corresponding to the image.

[0013] S3, the approximate hash code is processed by activation function and quantization to form a discrete binary string, which is input into the classification layer of the hidden layer as the final hash code. The classification layer calculates the corresponding image in the gallery through the secondary retrieval method to complete the training of the classification model;

[0014] S4. Input the image to be retrieved into the trained classification model, and output the image corresponding to the image to be retrieved in the gallery.

[0015] The role of the hidden layer is to use the feature vector obtained by the neural network to generate the hash code corresponding to the image, and at the same time guide the generation of the prediction result of the final image classification. Generally, the fully connected layer is used as the hidden layer, but studies have shown that the parameters of the general network on the fully connected layer can account for more than 80% of the total network parameters, and because the value of each node in the hidden layer is obtained by linearly transforming the values ​​of all nodes in the upper feature vector, overfitting is prone to occur.

[0016] The structure of the hidden layer specifically adopts a 1*1 convolution kernel and a global average pooling operation, or adopts a block-based fully connected layer structure to process redundant information.

[0017] Furthermore, the 1*1 convolution kernel realizes the fusion of information at the same position in different feature maps, and realizes the dimension increase and reduction of the channel. The 1*1 convolution kernel is used to reduce the dimension of the high-dimensional feature vector, and at the same time, the receptive field is increased to extract the main features.

[0018] Furthermore, the global average pooling operation compresses the feature map, takes the average of the data in the generated feature map as the output, and simplifies the process of generating classification results by expanding the feature map and performing linear transformation to directly calculating the average value, thereby reducing network parameters and being equivalent to regularizing the network to prevent overfitting.

[0019] The block fully connected layer outputs a q-dimensional vector as a hash code or classification result for the expanded high-dimensional feature vector through the fully connected layer operation. Assuming that the input layer has p nodes and the output layer has q nodes, if p = q × n + s, where 0 < s < n, then the first s fully connected layer segments have n + 1 nodes, and the remaining segments have n nodes.

[0020] Furthermore, the processing process of the block fully connected layer on the high-dimensional feature vector is specifically to divide the high-dimensional feature vector into multiple feature segments, and each feature segment generates an output node through a fully connected layer from n to 1, and the formula is as follows:

[0021] fci (x (i) )=W i x (i)

[0022] Among them, x (i) is the characteristic fragment, W i is the corresponding parameter matrix, fc i is the eigenvalue of the output node.

[0023] The formula for calculating the final hash code in step S3 is as follows:

[0024]

[0025]

[0026] in, is the binary string corresponding to the final hash code, σ is the Tanh activation function, f(x) is the approximate hash code, W h and b h As the process parameter, the value of the approximate hash code is mapped to the range of [-1, +1] through the Tanh activation function, and then [-1, +1] n The continuous value of becomes a discrete binary string, such as {0,1} n , get the final hash code corresponding to the image, the formula of Tanh activation function is as follows:

[0027]

[0028] Among them, x is the input value of the activation function.

[0029] A loss function is provided in the classification layer. The loss function increases the loss weight corresponding to the few-sample category in training and reduces the loss weight corresponding to the easily distinguishable sample to deal with the data imbalance problem. The distance between the predicted output and the true label is calculated through the loss function and is used for weight updating in the model back propagation.

[0030] Furthermore, the formula of the loss function is as follows:

[0031]

[0032] Among them, Loss(p,q) is the cross entropy loss function, N is the number of categories, p is the one-hot code vector corresponding to the label, q is the probability of the output of the neural network after passing through the softmax layer, the subscript i represents the i-th sample, the subscript k represents the true label, α is the weight factor, and β is the adjustment factor.

[0033] Furthermore, the value of α is affected by the number of categories. The smaller the number of categories, the larger α is. iThe larger the corresponding value, the more it increases the loss weight of the minority class.

[0034] Furthermore, the calculation formula of the adjustment factor is as follows:

[0035] β=(1-q i ) γ

[0036] Among them, γ is a hyperparameter and γ≥0. As γ increases, the influence of the adjustment factor on the weight becomes greater. When classifying difficult samples, q k The value of will be very small, then the value of the adjustment factor β will be very large, and the loss corresponding to the difficult-to-distinguish samples will increase. When facing simple sample classification, q k The value is very large, and the corresponding loss is very small.

[0037] During training, the network model uses the pre-training results of the original network on the ImageNet dataset as the initial parameters of the original layers in the new network, randomly initializes the parameter values ​​of the hidden layers, and uses the back-propagation algorithm to train on the new dataset based on the loss value calculated by the loss function. The stochastic gradient descent method is used to minimize the loss value, thereby achieving the goal of improving model performance and accelerating training speed.

[0038] The process of the secondary retrieval method in step S3 is specifically as follows: the first step is to search the image library according to the final hash code, and the second step is to search the image library according to the feature vector, and the results of the two steps are combined to obtain the final retrieved target image.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The present invention uses the label of the image as supervisory information, and directly learns the feature representation of the image and the corresponding hash code at the same time by adding a hidden layer in the network model, effectively processing redundant information and avoiding overfitting.

[0041] 2. The present invention increases the loss weight corresponding to the few-sample category in training in the loss function, and reduces the loss weight corresponding to the easy-to-distinguish samples to deal with the data imbalance problem. At the same time, the idea of ​​transfer learning is used in the model training process, and the results of pre-training of the original network on ImageNet are used as the initial parameters. Then, the fine-tuning method is used to update the parameters of the existing network, thereby speeding up the training speed and enhancing the training effect.

[0042] 3. In the image retrieval process, the present invention uses a secondary retrieval method to fully utilize the information of the generated hash code and the image feature vector in the convolutional neural network to improve the retrieval accuracy. While ensuring the retrieval accuracy, the model parameters are reduced as much as possible by using a lightweight network and removing the fully connected layer, making model transplantation and storage more convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the process of the present invention;

[0044] Figure 2 is a comparison diagram of the fully connected layer and the global average pooling operation of the present invention, where Figure 2 (a) is a schematic diagram of the fully connected layer. Figure 2 (b) is a schematic diagram of the global average pooling operation;

[0045] Figure 3 It is a schematic diagram of the structure of the block fully connected layer of the present invention;

[0046] Figure 4 Schematic diagram of the secondary search method in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0048] Example

[0049] like Figure 1 As shown in the figure, a class-imbalanced image hierarchical retrieval method based on deep hashing generates hash codes for large-scale image retrieval through a supervised lightweight deep hashing network, which specifically includes the following steps:

[0050] S1, obtain the training data set and input it into the feature extraction module of the network model, and obtain the high-dimensional feature vector of the image after processing through multiple convolutional layers;

[0051] S2, the high-dimensional feature vector is input into the hidden layer of the network model. The hidden layer learns the feature representation of the image based on the high-dimensional feature vector and outputs the approximate hash code corresponding to the image.

[0052] S3, the approximate hash code is processed by activation function and quantization to form a discrete binary string, which is input into the classification layer of the hidden layer as the final hash code. The classification layer calculates the corresponding image in the gallery through the secondary retrieval method to complete the training of the classification model;

[0053] S4. Input the image to be retrieved into the trained classification model, and output the image corresponding to the image to be retrieved in the gallery.

[0054] The role of the hidden layer is to use the feature vector obtained by the neural network to generate the hash code corresponding to the image, and at the same time guide the generation of the final image classification prediction results. Generally, the fully connected layer is used as the hidden layer, but studies have shown that the parameters of the general network on the fully connected layer can account for more than 80% of the total network parameters, and because the value of each node in the hidden layer is obtained by linearly transforming the values ​​of all nodes in the upper feature vector, overfitting is prone to occur.

[0055] The structure of the hidden layer specifically uses a 1*1 convolution kernel and a global average pooling operation, or uses a block-based fully connected layer structure to process redundant information.

[0056] The 1*1 convolution kernel realizes the information fusion of the same position in different feature maps, and realizes the dimension increase and reduction of the channel. The 1*1 convolution kernel is used to reduce the dimension of the high-dimensional feature vector, increase the receptive field, and extract the main features.

[0057] The global average pooling operation compresses the feature map and takes the average of the data in the generated feature map as the output. The difference between global average pooling and the fully connected layer is as follows: Figure 2 As shown in the figure, the global average pooling operation simplifies the process of generating classification results by expanding the feature map and performing linear transformation to directly averaging, which reduces the network parameters and is equivalent to regularizing the network to prevent overfitting.

[0058] The block fully connected layer outputs a q-dimensional vector as a hash code or classification result for the expanded high-dimensional feature vector through the fully connected layer operation. Assuming that the input layer has p nodes and the output layer has q nodes, if p = q × n + s, where 0 < s < n, then the first s fully connected layer segments have n + 1 nodes, and the remaining segments have n nodes.

[0059] like Figure 3 As shown in the figure, the processing process of the block fully connected layer on the high-dimensional feature vector is to divide the high-dimensional feature vector into multiple feature fragments, and each feature fragment generates an output node through a fully connected layer from n to 1. The formula is as follows:

[0060] fc i (x (i) )=W i x (i)

[0061] Among them, x (i) is the characteristic fragment, W i is the corresponding parameter matrix, fc i is the eigenvalue of the output node.

[0062] The formula for calculating the final hash code in step S3 is as follows:

[0063]

[0064]

[0065] in, is the binary string corresponding to the final hash code, σ is the Tanh activation function, f(x) is the approximate hash code, W h and b h As the process parameter, the value of the approximate hash code is mapped to the range of [-1, +1] through the Tanh activation function, and then [-1, +1] n The continuous value of becomes a discrete binary string, such as {0,1} n , get the final hash code corresponding to the image, the formula of Tanh activation function is as follows:

[0066]

[0067] Among them, x is the input value of the activation function.

[0068] In practical applications, especially for medical images, the number of images of different categories varies greatly, and the number of severe case images is far less than that of normal or mild case images. In this case, the traditional cross entropy loss function will not be able to distinguish the parts with fewer categories, and the model will simply judge all input images as the categories with more numbers in the training set to achieve the highest accuracy. In order to deal with this category imbalance phenomenon, the traditional method starts with the image data and enhances the small category images by rotating, translating, changing the contrast, adding noise, etc., to achieve the purpose of balancing the number of images in each category, but this method may lead to overfitting.

[0069] In this embodiment, a loss function is provided in the classification layer. According to the idea of ​​Focal Loss, it is extended to the multi-classification problem as the loss function used in this embodiment. The loss function is obtained on the basis of cross entropy. The loss weight corresponding to the small sample category in training is increased in the loss function, and the loss weight corresponding to the easy-to-distinguish sample is reduced to deal with the data imbalance problem. The distance between the predicted output and the true label is calculated by the loss function, which is used to update the weight in the back propagation of the model. The loss function solves the problem of category imbalance in the real data set from two perspectives: the imbalanced distribution of the number of categories and the weight of distinguishing difficult and easy samples. Practice has proved that the use of image enhancement technology and loss function at the same time can effectively increase the retrieval effect of category imbalanced data sets.

[0070] The formula of the loss function is as follows:

[0071]

[0072] Among them, Loss(p,q) is the cross entropy loss function, N is the number of categories, p is the one-hot code vector corresponding to the label, q is the probability of the output of the neural network after passing through the softmax layer, the subscript i represents the i-th sample, the subscript k represents the true label, α is the weight factor, and β is the adjustment factor.

[0073] The value of α is affected by the number of categories. The smaller the number of categories, the larger the value of α. i The larger the corresponding value, the more it increases the loss weight of the minority class.

[0074] The calculation formula of the adjustment factor is as follows:

[0075] β=(1-q i ) γ

[0076] Among them, γ is a hyperparameter and γ≥0. As γ increases, the influence of the adjustment factor on the weight becomes greater. When classifying difficult samples, q k The value of will be very small, then the value of the adjustment factor β will be very large, and the loss corresponding to the difficult-to-distinguish samples will increase. When facing simple sample classification, q k The value is very large, and the corresponding loss is very small.

[0077] With the rapid development of convolutional neural networks, many works have proved that CNN has good feature extraction capabilities, and as the number of network layers increases, the extracted features are more discriminative for specific data sets. However, in the initial convolutional layer, the extracted features are basic features that almost all images have. In addition, the training of convolutional neural networks requires large-scale image data sets and computing equipment support, which individuals and small organizations do not have.

[0078] The model training of this embodiment adopts the idea of ​​transfer learning, and obtains a new network model by fine-tuning the basic network SqueezeNet structure, that is, adding hidden layers. When training the network model, the results of pre-training of the original network on the ImageNet dataset are used as the initial parameters of the original layer in the new network, and the parameter values ​​of the hidden layer are randomly initialized. According to the loss value calculated by the loss function, the back propagation algorithm is used for training on the new dataset, and the stochastic gradient descent method is used to minimize the loss value, so as to achieve the goal of improving model performance and speeding up training.

[0079] The process of the secondary retrieval method in step S3 is specifically as follows: the first step is to search the image library according to the final hash code, and the second step is to search the image library according to the feature vector, and the results of the two steps are combined to obtain the final retrieved target image.

[0080] When implementing it, Figure 4 As shown, it includes the first stage and the second stage, wherein the first stage:

[0081] Define image dataset I = {i1,i2,...,i n}, i n For the nth image, the corresponding set of d-dimensional hash codes is H = {h1,h2,...,h n}, h n For image i n The corresponding final hash code, and h n ∈{0,1} d For the image q to be retrieved, its corresponding hash code is h q .

[0082] First calculate h q The Hamming distance between each hash code in H is calculated, and then sorted by size. Set a threshold a, and select m images corresponding to binary strings less than a in the sorted Hamming distance table as the retrieval results of the first stage, expressed as The specific formula is as follows:

[0083] O={h i |Dis hm (h q ,h i )-a≤0}

[0084] Among them, Dis hm (·) is the Hamming distance.

[0085] Phase 2:

[0086] In this stage, the feature vector of CNN is used as the retrieval basis. Define the image to be retrieved as q and the image group generated after the first stage as O, and its corresponding feature vector is f q and Calculate f q With F O The Euclidean distance e between each vector in i , the specific formula is as follows:

[0087]

[0088] e i The smaller the value, the more similar the two images are. i The distances in set E are arranged in ascending order. For the final top k search results, only the images corresponding to the top k distances in E need to be selected for output.

[0089] In the two stages of retrieval in this embodiment, the Hamming distance between the image to be retrieved and the hash code of the image in the data set is first calculated, and images with a distance within the threshold are selected to form the first stage retrieval result image group O. In the second step of retrieval, the Euclidean distance between the feature vector of the image to be retrieved and the feature vector in O is calculated, and the top k retrieval results are finally obtained after sorting. This method first performs a coarse-grained retrieval and then a fine-grained retrieval. Compared with the retrieval method using only hash code, the information of the feature vector is used to make the retrieval results more accurate; compared with the retrieval method using only high-dimensional feature vectors, the "dimensionality curse" problem is eliminated, and the reduction of the retrieval range in the first step greatly reduces the amount of calculation.

[0090] In addition, it should be noted that the specific embodiments described in this specification may be named differently, and the above contents described in this specification are merely examples of the structure of the present invention. Any equivalent changes or simple changes made based on the structure, features and principles of the present invention are included in the protection scope of the present invention. Those skilled in the art of the present invention may make various modifications or supplements to the specific examples described or adopt similar methods, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A hierarchical retrieval method for class-imbalanced images based on deep hashing, characterized by: The specific steps include: S1. Obtain the training data set and input it into the feature extraction module of the network model. After processing through multiple convolutional layers, the high-dimensional feature vector of the image is obtained. S2. The high-dimensional feature vector is input to the hidden layer of the network model. The hidden layer learns the feature representation of the image based on the high-dimensional feature vector and outputs an approximate hash code corresponding to the image. The structure of the hidden layer specifically adopts a 1*1 convolution kernel and a global average pooling operation, or a block-wise fully connected layer structure. S3: After the approximate hash code is processed by activation function and quantization, it forms a discrete binary string, which is input into the classification layer of the hidden layer as the final hash code. The classification layer calculates the corresponding image in the image library through the secondary retrieval method to complete the training of the classification model. The formula for calculating the final hash code is as follows: in, is the binary string corresponding to the final hash code, σ is the Tanh activation function, f(x) is the approximate hash code, W h and b h is the process parameter; A loss function is provided in the classification layer, wherein the loss weight corresponding to the minority sample class in training is increased, and the loss weight corresponding to the easily distinguishable sample is reduced; S4. Input the image to be retrieved into the trained classification model, and output the image corresponding to the image to be retrieved in the gallery.

2. The method for hierarchical retrieval of class-imbalanced images based on deep hashing according to claim 1, characterized in that: The 1*1 convolution kernel performs a dimensionality reduction operation on the high-dimensional feature vector, the global average pooling operation compresses the feature map, and the average of the data in the generated feature map is taken as the output.

3. The method for hierarchical retrieval of class-imbalanced images based on deep hashing according to claim 1, characterized in that: The block-wise fully connected layer processes the high-dimensional feature vector by dividing it into multiple feature segments, and each feature segment generates an output node through a fully connected layer. The formula is as follows: fc i (x (i) )=W i x (i) Among them, x (i) is the feature fragment, W i is the corresponding parameter matrix, fc i is the eigenvalue of the output node.

4. The method for hierarchical retrieval of class-imbalanced images based on deep hashing according to claim 1, characterized in that: The formula of the loss function is as follows: Where Los(p,q) is the cross entropy loss function, N is the number of categories, p is the one-hot code vector corresponding to the label, q is the probability of the output of the neural network after passing through the softmax layer, subscript i represents the i-th sample, subscript k represents the true label, α is the weight factor, and β is the adjustment factor.

5. The method for hierarchical retrieval of class-imbalanced images based on deep hashing according to claim 4, characterized in that: The calculation formula of the adjustment factor is as follows: β=(1-q i ) γ Here, γ is a hyperparameter and γ ≥ 0.

6. The method for hierarchical retrieval of class-imbalanced images based on deep hashing according to claim 1, characterized in that: During training, the network model uses the results of pre-training of the original network on the ImageNet dataset as the initial parameters of the original layers in the new network, randomly initializes the parameter values ​​of the hidden layers, and uses the backpropagation algorithm to train on the new dataset based on the loss value calculated by the loss function, and uses the stochastic gradient descent method to minimize the loss value.

7. The method for hierarchical retrieval of class-imbalanced images based on deep hashing according to claim 1, characterized in that: The process of the secondary retrieval method in step S3 is specifically as follows: the first step is to search the image library according to the final hash code, and the second step is to search the image library according to the feature vector, and the results of the two steps are combined to obtain the final retrieved target image.