An image retrieval method based on hyperdimensional vector computing

Abstract

The present invention discloses an image retrieval method based on hyperdimensional vector calculation, comprising: preprocessing a collected image data set; dividing the preprocessed data set into a training set and a test set; based on the training set, training a ResNet18 network using a small sample learning method to obtain m+1 feature extraction networks; using the trained ResNet18 network to extract features from each image in the test set, generating a hyperdimensional vector of the test set image, and obtaining a hyperdimensional vector library; using the trained ResNet18 network to extract features from an input query image, generating a hyperdimensional vector corresponding to the query image; matching the hyperdimensional vector corresponding to the query image with all vectors in the hyperdimensional vector library, and outputting the vector with the highest matching degree. The present invention realizes image retrieval with low resource consumption and high robustness, can effectively complete image matching tasks in the field of the Internet of Things, and meets the demand for high robustness image retrieval in scenarios with limited hardware resources.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperdimensional computing, and in particular to an image retrieval method based on hyperdimensional vector computing. Background Art

[0002] Image retrieval is a core technology in the Internet of Things (IoT) field, playing a key role in intelligent security, digital decision-making, and other tasks, with significant economic benefits and strategic significance. Existing research generally uses deep learning-based image retrieval schemes, achieving high-precision image retrieval through deep neural networks. However, these schemes are based on low-dimensional vectors and high-precision data. High-precision data not only consumes a large amount of hardware resources but also places high demands on hardware storage and computational accuracy. Consequently, these existing schemes are less robust in harsh application scenarios such as high temperature and humidity, making them difficult to meet the hardware deployment requirements of IoT edge devices. Therefore, a new image retrieval framework is urgently needed to achieve lightweight embedding and highly robust operation of this function.

[0003] Hyperdimensional vector computing is a computational framework that exploits the properties of random high-dimensional vector spaces. It employs holographic encoding technology to map vectors from low-dimensional spaces to high-dimensional spaces. The target task is then accomplished by processing the high-dimensional data. Unlike floating-point operations in neural networks, hyperdimensional vector computing can binarize hyperdimensional vectors, converting arithmetic operations into logical operations, thereby reducing hardware overhead. Therefore, compared to traditional deep learning-based approaches, hyperdimensional vector computing offers the advantages of low resource consumption and ease of hardware implementation. Furthermore, compared to classical computing, hyperdimensional vector computing uses ultra-fast caches as its data type, with large dimensions, typically in the tens of thousands. These ultra-wide words introduce redundancy against noise, making the computation inherently robust. Therefore, hyperdimensional vector computing is a promising approach for image retrieval. However, effective methods for applying hyperdimensional vector computing to image retrieval have yet to be developed. Summary of the Invention

[0004] The purpose of the present invention is to provide an image retrieval method based on hyperdimensional vector calculation. In response to the problems existing in the existing image retrieval schemes in the prior art, hyperdimensional coding is used to realize hyperdimensional mapping of feature vectors. First, the encoding of feature vectors is completed by constructing a hyperdimensional position coding matrix, and then the similarity matching between input images and library images is completed by inner product sorting between hyperdimensional vectors.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] An image retrieval method based on hyperdimensional vector calculation, the image retrieval method comprising the following steps:

[0007] Preprocess the collected image dataset, which contains images of different categories and binary annotations of all features of each image. Select the top m features and the bottom feature in terms of inter-class variance. Divide the preprocessed dataset into a training set and a test set.

[0008] Based on the training set, the ResNet18 network is trained using a small sample learning method to obtain m+1 feature extraction networks;

[0009] Use the trained ResNet18 network to extract features from each image in the test set, generate a hyperdimensional vector for the test set images, and obtain a hyperdimensional vector library;

[0010] Use the trained ResNet18 network to extract features from the input query image and generate a hyperdimensional vector corresponding to the query image;

[0011] The hyperdimensional vector corresponding to the query image is matched with all vectors in the hyperdimensional vector library, and the vector with the highest matching degree is output.

[0012] Furthermore, the process of selecting the top m features and the last feature in terms of between-class variance includes the following steps:

[0013] Let the dataset contain images of k categories and binary annotations of t features of each image;

[0014] For each category, find the average value of each feature and obtain a k×t mean matrix;

[0015] For each feature, the inter-class variance is calculated to obtain a t×1 variance matrix. Each element in the variance matrix represents the inter-class variance of the feature, that is, the inter-class difference.

[0016] Sort the elements in the variance matrix and select the top m features and the last feature in between-class variance ranking.

[0017] Furthermore, the ResNet18 network is trained for the m+1 features selected in step 1. The network input is defined as a three-channel image matrix of a×b×3, where a is the number of pixels in each row of the image and b is the number of pixels in each column of the image. The network output is a 1×n feature vector used to represent one of the features. Through training, m+1 feature extraction networks are obtained.

[0018] The process of training the ResNet18 network using small sample learning includes the following steps:

[0019] Take r images of k types of objects in the training set as comparison benchmarks;

[0020] During the training process, a sample is selected from the training set as the anchor point x a , select another positive sample x according to the anchor point + and a negative sample x - , input the three selected samples into the neural network f, and calculate the distance d between the positive sample and the anchor point + And the distance d between the negative sample and the anchor point - ,in is the square of the two norm, and the training loss function is set as follows:

[0021] Loss(x a , x + , x - )=max{0,d + +α-d -}

[0022] Where α is a hyperparameter greater than 0;

[0023] After inputting each training image, it is compared with the selected benchmark and back-propagated to make the inner product of the feature vectors output by images of the same category as large as possible, and the inner product of the feature vectors output by images of different categories as small as possible.

[0024] Furthermore, the trained ResNet18 network is used to extract features from the input image. The process of generating the hyperdimensional vector corresponding to the image includes the following steps:

[0025] Use the trained ResNet18 network to extract features from the input image and binarize the extracted feature vectors; generate a hyperdimensional position encoding matrix;

[0026] For the features with the top m variances in the dataset, the corresponding n-dimensional feature vectors are sequentially concatenated into a 1×mn expansion vector, generating mn independent and orthogonal h1-dimensional address super-dimensional vectors to represent the position of each pixel. Each pixel value in the expansion vector is multiplied by the corresponding address super-dimensional vector, and then added to form a 1×h1 super-dimensional vector. The super-dimensional vector is normalized to obtain the super-dimensional vector H1.

[0027] For the feature with the lowest variance in the data set, generate n independent and orthogonal h2-dimensional address super-dimensional vectors to represent the position of each pixel, and multiply the vector corresponding to the feature by the n×h2 position code to obtain the super-dimensional vector H2;

[0028] The super-dimensional vector H1 and the super-dimensional vector H2 are directly connected to obtain the super-dimensional vector H representing the input image.

[0029] Furthermore, the process of binarizing the feature vector includes the following steps:

[0030] Each element in each eigenvector is binarized, and values ​​greater than or equal to 0 are assigned to 1, and values ​​less than 0 are assigned to 1.

[0031] Furthermore, the process of matching the super-dimensional vector corresponding to the query image with all vectors in the super-dimensional vector library includes the following steps:

[0032] Perform inner product operations on the hyperdimensional vector of the query image and all vectors in the established hyperdimensional vector library, and sort the inner product results;

[0033] The image corresponding to the library vector with the largest inner product result is selected as the output result, and the image with the highest matching degree is output.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The image retrieval method based on hyperdimensional vector computing of the present invention realizes low resource consumption and high robustness image retrieval by integrating the feature extraction method based on deep learning and hyperdimensional coding technology, as well as the small sample learning strategy. Therefore, it can effectively complete the image matching task in the field of Internet of Things and meet the high robustness image retrieval requirements in scenarios with limited hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of an image retrieval method based on hyperdimensional vector calculation according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of super-dimensional vector encoding provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of establishing and testing a superdimensional vector library according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

[0040] An image retrieval method based on hyperdimensional vector calculation, the image retrieval method comprising the following steps:

[0041] Preprocess the collected image dataset, which contains images of different categories and binary annotations of all features of each image. Select the top m features and the bottom feature in terms of inter-class variance. Divide the preprocessed dataset into a training set and a test set.

[0042] Based on the training set, the ResNet18 network is trained using a small sample learning method to obtain m+1 feature extraction networks;

[0043] Use the trained ResNet18 network to extract features from each image in the test set, generate a hyperdimensional vector for the test set images, and obtain a hyperdimensional vector library;

[0044] Use the trained ResNet18 network to extract features from the input query image and generate a hyperdimensional vector corresponding to the query image;

[0045] The hyperdimensional vector corresponding to the query image is matched with all vectors in the hyperdimensional vector library, and the vector with the highest matching degree is output.

[0046] As described in the background technology of this application, existing image retrieval technology has problems such as large consumption of hardware storage and computing resources and poor computational robustness, making it difficult to meet the flexible deployment and application requirements of edge devices in the Internet of Things field.

[0047] Therefore, in order to solve the above problems, the embodiments of the present invention partially provide an image retrieval method based on super-dimensional vector calculation. Figure 1 The entire implementation process of the method is shown and will be systematically explained below. The numbers in the embodiments are not the only ones selected but are only for illustration.

[0048] (1) Dataset collection and preprocessing

[0049] Step S11: Obtain the dataset required for feature network training and verification. Taking the open source dataset CelebA as an example, the training set contains 202,599 images belonging to 10,177 people, and binary annotations of 40 features for each image (1 if the image contains the feature, -1 otherwise). For each category, the average value of each feature is calculated, and finally a 10,177×40 mean matrix is ​​obtained.

[0050] In step S12, for each feature, the inter-class variance is calculated to obtain a 40×1 variance matrix, where each element in the matrix represents the inter-class variance of the feature, that is, the inter-class difference.

[0051] In step S13, the variance matrix is ​​sorted and the top 10 features and the last-ranked feature are selected for subsequent processing.

[0052] (2) Feature extraction network training

[0053] In step S21, the ResNet18 network is trained on the training set. Taking the open source dataset CelebA as an example, the 11 features selected in step (1) are trained. Since the training set contains pictures belonging to different categories and 40 binary labels of features for each picture, the input of the network is set to a 64×64×3 three-channel image matrix, and the output is a 1×128 feature vector corresponding to one of the features. Through training, 11 feature extraction networks can be obtained.

[0054] Step S22: In order to improve the discrimination between classes, the training process adopts the small sample learning method. During the training process, a sample is selected from the training set as the anchor point x a , then select a positive sample x according to the anchor point + and a negative sample x - , then input the three samples into the neural network f and calculate the distance d between the positive sample and the anchor point + And the distance d between the negative sample and the anchor point - ,in in is the square of the two norm, so the training loss function is set as follows:

[0055] Loss(x a , x + , x - )=max{0,d + +α-d -}

[0056] Where α is a hyperparameter greater than 0. After inputting each training image, it is compared with the selected benchmark and back-propagated to make the inner product of the feature vectors output by images of the same category as large as possible, and the inner product of the feature vectors output by images of different categories as small as possible.

[0057] (3) Generation of super-dimensional vectors of input images

[0058] Step S31, the process of generating the super-dimensional vector is as follows: Figure 2 As shown in the figure, first, during the image retrieval process, the trained residual neural network is used to extract features of the input image. In order to further reduce resource consumption, the feature vector is binarized, and features greater than or equal to 0 are assigned a value of 1, and features less than 0 are assigned a value of -1.

[0059] Step S32: generating a hyper-dimensional position coding matrix in order to realize hyper-dimensional mapping of the feature vector.

[0060] In step S33, taking the open source dataset CelebA as an example, for the top ten features in (1), the feature vectors are sequentially concatenated into a 1×1280 expansion vector. First, 1280 independent and orthogonal 10,000-dimensional address hyperdimensional vectors are generated to represent the position of each pixel. Each pixel value in the expansion vector is multiplied by the corresponding address hyperdimensional vector, and then added to form a 1×10,000 hyperdimensional vector. This vector is normalized to obtain the hyperdimensional vector H1 representing the image.

[0061] In step S34, for the feature ranked last in (1), the vector corresponding to the feature is multiplied by the 128×10000 position code obtained in (3b) to obtain a 1×10000 super-dimensional vector H2.

[0062] Step S35 : directly connect the two super-dimensional vectors H1 and H2 to obtain a 20,000-dimensional super-dimensional vector H representing the image.

[0063] (4) Generation of super-dimensional vector retrieval library

[0064] The process of generating a hyperdimensional vector retrieval library is as follows: Figure 3 As shown, all the pictures in the test set are passed through the encoder shown in step (3) to generate super-dimensional vectors, forming a super-dimensional vector library of the test set pictures as a matching source for subsequent retrieval.

[0065] (5) Generation of hyperdimensional vectors for query images

[0066] The testing process is also Figure 3 Given in , for the input query image, the corresponding super-dimensional vector is generated by the encoder shown in step (3).

[0067] (6) Inner product sorting of hyperdimensional vectors

[0068] Perform an inner product operation on the query image's hyperdimensional vector and all vectors in the hyperdimensional vector library established in step (4), and sort the inner product results. Select the image corresponding to the library vector with the largest inner product result as the output result, that is, output the image with the highest matching degree, thus completing the entire matching process.

[0069] The image retrieval method based on hyperdimensional vector computing provided by this invention achieves low-resource, highly robust image retrieval by integrating deep learning and hyperdimensional coding techniques with a small sample learning strategy. It can effectively complete image matching tasks in the Internet of Things (IoT) field and meet the demand for highly robust image retrieval in scenarios with limited hardware resources.

[0070] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0071] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions for executing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0074] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0075] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An image retrieval method based on hyperdimensional vector calculation, characterized in that: The image retrieval method comprises the following steps: Preprocess the collected image dataset, which contains images of different categories and binary annotations of all features of each image. Select the top m features and the bottom feature in terms of inter-class variance. Divide the preprocessed dataset into a training set and a test set. Based on the training set, the ResNet18 network is trained using a small sample learning method to obtain m+1 feature extraction networks; Use the trained ResNet18 network to extract features from each image in the test set, generate a hyperdimensional vector for the test set images, and obtain a hyperdimensional vector library; Use the trained ResNet18 network to extract features from the input query image and generate a hyperdimensional vector corresponding to the query image; Match the hyperdimensional vector corresponding to the query image with all vectors in the hyperdimensional vector library and output the vector with the highest matching degree; The ResNet18 network is trained for the m+1 features selected in step 1. The network input is a three-channel image matrix of a×b×3, where a is the number of pixels in each row of the image and b is the number of pixels in each column of the image. The network output is a 1×n feature vector used to represent one of the features. Through training, m+1 feature extraction networks are obtained. The process of training the ResNet18 network using small sample learning includes the following steps: Take r images of k-type objects in the training set as comparison benchmarks; During the training process, a sample is selected from the training set as the anchor point x a , select another positive sample x according to the anchor point + and a negative sample x - , input the three selected samples into the neural network f, and calculate the distance d between the positive sample and the anchor point + And the distance d between the negative sample and the anchor point - ,in is the square of the two norm, and the training loss function is set as follows: Loss(x a ,x + ,x - )=max{0,d + +α-d - } Where α is a hyperparameter greater than 0; After inputting each training image, it is compared with the selected benchmark and back-propagated to make the inner product of the feature vectors output by images of the same category as large as possible, and the inner product of the feature vectors output by images of different categories as small as possible.

2. The image retrieval method based on superdimensional vector calculation according to claim 1, characterized in that: The process of selecting the top m features and the last feature in terms of between-class variance includes the following steps: Let the dataset contain images of k categories and binary annotations of t features of each image; For each category, find the average value of each feature and obtain a k×t mean matrix; For each feature, the inter-class variance is calculated to obtain a t×1 variance matrix. Each element in the variance matrix represents the inter-class variance of the feature, that is, the inter-class difference. Sort the elements in the variance matrix and select the top m features and the last feature in between-class variance ranking.

3. The image retrieval method based on superdimensional vector calculation according to claim 1, characterized in that: The trained ResNet18 network is used to extract features from the input image. The process of generating the corresponding hyperdimensional vector of the image includes the following steps: Use the trained ResNet18 network to extract features from the input image and binarize the extracted feature vectors; generate a hyperdimensional position encoding matrix; For the features with the top m variances in the dataset, the corresponding n-dimensional feature vectors are sequentially concatenated into a 1×mn expansion vector, generating mn independent and orthogonal h1-dimensional address super-dimensional vectors to represent the position of each pixel. Each pixel value in the expansion vector is multiplied by the corresponding address super-dimensional vector, and then added to form a 1×h1 super-dimensional vector. The super-dimensional vector is normalized to obtain the super-dimensional vector H1. For the feature with the lowest variance in the data set, generate n independent and orthogonal h2-dimensional address super-dimensional vectors to represent the position of each pixel, and multiply the vector corresponding to the feature by the n×h2 position code to obtain the super-dimensional vector H2; The super-dimensional vector H1 and the super-dimensional vector H2 are directly connected to obtain the super-dimensional vector H representing the input image.

4. The image retrieval method based on superdimensional vector calculation according to claim 3, characterized in that: The process of binarizing the feature vector includes the following steps: Each element in each eigenvector is binarized, with values ​​greater than or equal to 0 assigned to 1 and values ​​less than 0 assigned to -1.

5. The image retrieval method based on superdimensional vector calculation according to claim 1, characterized in that: The process of matching the hyperdimensional vector corresponding to the query image with all vectors in the hyperdimensional vector library includes the following steps: Perform inner product operations on the hyperdimensional vector of the query image and all vectors in the established hyperdimensional vector library, and sort the inner product results; The image corresponding to the library vector with the largest inner product result is selected as the output result, and the image with the highest matching degree is output.

Images