A method for calculating image hash values ​​using convolutional neural networks and orthogonal transformation

Abstract

The present invention relates to a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformation, and belongs to the field of image processing. The present invention selects an appropriate image classification convolutional neural network, trains the convolutional neural network using training images, and makes it reach a stable state; inputs the image for which the hash value needs to be calculated into the trained convolutional neural network, and retains the input of the convolutional neural network class probability layer as the primary feature of the image; uses orthogonal transformation to transform the primary feature of the image into the secondary feature of the image; according to the required hash value length, intercepts part of the secondary feature of the image, and binarizes each component of the secondary feature of the intercepted image into 0 or 1 according to the average value of the part of the secondary feature of the intercepted image. The present invention can effectively utilize the feature information of the input image extracted by the convolutional neural network, so that the hash values ​​of similar images are similar, which is conducive to the rapid retrieval of image big data.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and relates to a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformation. More specifically, it relates to a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformation neural networks, which facilitates rapid judgment of similarity between images using image big data. Background Art

[0002] In recent years, with the advancement of photography technology and the popularization of devices such as mobile phones with photography functions, people have accumulated a large number of images, forming image big data. Given an image, searching for the same or similar images in image big data is a problem that needs to be solved in image big data.

[0003] In conventional feature-based image retrieval methods, features are represented by floating-point numbers, which requires a large amount of storage and computing resources. Therefore, in image big data, simple image hash values ​​are usually used to represent and retrieve images. Hash values ​​are a kind of fingerprint of features contained in multimedia files such as images, audio and video. They are generally represented by bytes in the form of integers. Logical operations can be used to quickly determine the difference between two hash values, which can quickly improve the retrieval speed.

[0004] At present, there are many methods for calculating the hash value of an image, such as the image-aware hash value method proposed by Google. In these methods, the content of the image, that is, the features of the image, are rarely considered. Therefore, different images may have similar hash values; the hash values ​​of the same image may differ greatly after being reduced, enlarged, translated, or rotated to a small extent.

[0005] The use of deep neural networks can achieve similar hashes for visually similar images. Most image hash calculation methods based on deep learning use the method of directly outputting hash values. Since it is difficult to determine the hash value of an image in advance, the learning process of the neural network is relatively complicated, and the length of the image hash value is difficult to control. In the "A Method for Calculating Image Hash Values ​​Using Convolution" with acceptance number 202110854478.0, the result obtained by the convolution operation of the middle layer of the convolutional neural network is used as the feature of the image. When this method has more convolution templates, the image hash value obtained is longer, which is not conducive to rapid retrieval of images based on image hash values.

[0006] Therefore, a hash value calculation method that takes image content into consideration is needed so that visually similar images have similar hash values ​​and the length of the image hash value can be controlled. The present invention is generated based on this practical demand. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] The technical problem to be solved by the present invention is how to provide a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations, which can not only control the characteristics of the hash values, but also take into account the characteristics of the images, and meet the requirement of similar hash values ​​for visually similar images.

[0009] (II) Technical solution

[0010] In order to solve the above technical problems, the present invention proposes a method for calculating image hash values ​​using a convolutional neural network and orthogonal transformation, the method comprising the following steps:

[0011] S1. Neural network selection step: select an appropriate image classification convolutional neural network and train the convolutional neural network using training images to achieve a stable state with high accuracy;

[0012] S2, image feature acquisition step: input the image for which the hash value needs to be calculated into the trained convolutional neural network, and retain the input of the convolutional neural network class probability layer as the primary feature of the image;

[0013] S3, image feature transformation step: using orthogonal transformation, transforming the primary features of the image into secondary features of the image;

[0014] S4, image hash value calculation step: according to the required hash value length, cut off some components of the secondary features of the image, and binarize each component of the secondary features of the cut off image to 0 or 1 according to the average value of the partial components of the secondary features of the cut off image.

[0015] Furthermore, in step S1, the convolutional neural network is a VGGNet-16 convolutional neural network. The VGGNet-16 network has a total of 16 layers, of which the first 13 layers are convolution and pooling layers, and the last 3 layers are fully connected layers. The output of the last fully connected layer directly enters the class probability layer, i.e., the SoftMax layer.

[0016] Furthermore, in step S2, the image for which the hash value needs to be calculated is scaled to an image that meets the input requirements of the convolutional neural network and then input into the trained convolutional neural network.

[0017] Furthermore, in step S2, the input of the VGGNet-16 convolutional neural network is an original image of size 224×224, the output is 1000 categories, and the first-level feature of the image is a 1000-dimensional vector.

[0018] Furthermore, in step S3, the orthogonal transform is discrete cosine transform, Hadamard transform or Haar transform.

[0019] Furthermore, in step S3, before the transformation, if the number of components of the primary feature X of the image is not an integer power of 2, a certain number of components is added so that the number of components of X is an integer power of 2.

[0020] Furthermore, for the VGGNet-16 network, if the first-level feature of the image is a 1000-dimensional vector X = (x1, x2, ..., x 1000 ), then add 24 components with a value of 0 and expand it to a 1024-dimensional vector X'=(x1,x2,…,x 1024 ), where x 1001 to x 1024 The values ​​of are all 0; then, X' is used as the input of the orthogonal transformation, and the orthogonal transformation is performed to obtain the vector Y = (y1, y2, ..., y 1024 ), Y is the secondary feature of the image.

[0021] Furthermore, the step S4 specifically includes: according to the required hash value length N, the first N components of the secondary feature of the image are intercepted, and according to the average value of the first N components of the secondary feature of the intercepted image, each component of the secondary feature of the intercepted image is binarized to 0 or 1, that is, if the value of a component is greater than or equal to the average value, the corresponding bit in the hash value is 1, otherwise, the corresponding bit in the hash value is 0.

[0022] Furthermore, for the VGGNet-16 network, if the required image hash value is 64 bits, the 64 components of the secondary features of the image are intercepted to obtain Y'=(y1,y2,…,y 64 ); then calculate the average value y = (y1+y2+…+y 64 ) / 64; if the hash value is Z=(z1,z2,…,z 64 ), then z i The calculation method is: if y i ≥y, then z i =1, otherwise z i =0.

[0023] Furthermore, the 64 components are the first 64 components of the secondary features of the image.

[0024] (III) Beneficial effects

[0025] The present invention proposes a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformation. The method for calculating image hash values ​​proposed in the present invention can effectively utilize the feature information of the input image extracted by the convolutional neural network, so that the hash values ​​of similar images are similar, and can control the length of the hash value as needed, which is conducive to the rapid retrieval of image big data and has important application value in image big data applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations of the present invention;

[0027] Figure 2 It is a schematic diagram of a method for calculating image hash values ​​using a convolutional neural network and orthogonal transformation according to the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0029] The present invention discloses a method for calculating an image hash value using a convolutional neural network and an orthogonal transformation, the method comprising: (1) a neural network selection step. An appropriate image classification convolutional neural network is selected, and the convolutional neural network is trained using a training image so that it reaches a stable state and has high accuracy. (2) an image feature acquisition step. The image for which a hash value needs to be calculated is input into the trained convolutional neural network, and the input of the convolutional neural network class probability layer is retained as the primary feature of the image. (3) an image feature transformation step. The primary feature of the image is transformed into the secondary feature of the image using an orthogonal transformation. (4) an image hash value calculation step. According to the required hash value length, part of the secondary feature of the image is intercepted, and each component of the secondary feature of the intercepted image is binarized to 0 or 1 according to the average value of the part of the secondary feature of the intercepted image. The method for calculating the image hash value proposed by the present invention can effectively extract the feature information of the input image extracted by the convolutional neural network, so that the hash values ​​of similar images are similar, which is conducive to the rapid retrieval of image big data and has important application value in image big data applications.

[0030] The purpose of the present invention is to provide a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations, which can not only control the characteristics of the hash values, but also take into account the characteristics of the image, and meet the requirement of similar hash values ​​of visually similar images.

[0031] To achieve the above object, the present invention proposes a method for calculating image hash values ​​using a convolutional neural network and orthogonal transformation, the method comprising:

[0032] S1. Neural network selection step. Select an appropriate image classification convolutional neural network and train the convolutional neural network using training images to achieve a stable state with high accuracy.

[0033] S2, image feature acquisition step: Input the image for which the hash value needs to be calculated into the trained convolutional neural network, and retain the input of the convolutional neural network class probability layer as the primary feature of the image.

[0034] S3, image feature transformation step: using orthogonal transformation, the primary features of the image are transformed into secondary features of the image.

[0035] S4, image hash value calculation step: according to the required hash value length, some components of the secondary features of the image are intercepted, and each component of the secondary features of the intercepted image is binarized to 0 or 1 according to the average value of the partial components of the secondary features of the intercepted image.

[0036] Figure 1 It is a flow chart of a method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations of the present invention. Figure 2 This is a schematic diagram of a method for calculating image hash values ​​using a convolutional neural network and orthogonal transformation. Figure 1 and Figure 2 , illustrating the implementation mode of the present invention. Figure 1 As shown, the method includes:

[0037] (1) Neural network selection step: Select an appropriate image classification convolutional neural network and use the training images to train the convolutional neural network to achieve a stable state with high accuracy.

[0038] When implementing it, you can choose the VGGNet-16 convolutional neural network. The VGGNet-16 network has 16 layers, of which the first 13 layers are convolution and pooling layers, the last 3 layers are fully connected layers, and the output of the last fully connected layer directly enters the class probability layer (i.e., the SoftMax layer). Figure 2 In the example, the complete structure of the VGGNet-16 convolutional neural network is not given. The input of the VGGNet-16 convolutional neural network is a 224×224 size original image, and the output is 1000 categories.

[0039] (2) Image feature acquisition step: The image for which the hash value needs to be calculated is input into the trained convolutional neural network, and the input of the convolutional neural network class probability layer is retained as the primary feature of the image.

[0040] In the specific implementation, for the VGGNet-16 network, an image is scaled to a 224×224 original image, input into the trained VGGNet-16 network, and the input of the class probability layer is retained as the first-level feature of the image. Since the VGGNet-16 output is 1000 categories, the first-level feature of the image is a 1000-dimensional vector.

[0041] (3) Image feature transformation step: Using orthogonal transformation, the primary features of the image are transformed into the secondary features of the image.

[0042] In the specific implementation, for the VGGNet-16 network, if the first-level feature of the image is a 1000-dimensional vector X = (x1, x2, ..., x 1000 ), we can add 24 components with the value of 0 and expand it to a 1024-dimensional vector X'=(x1,x2,…,x 1024 ), where x 1001 to x 1024 The values ​​of are all 0. Then, X' is used as the input of discrete cosine transform, Hadamard transform, Haar transform and other orthogonal transforms, and the orthogonal transform vector Y = (y1, y2, ..., y 1024 ), Y is the secondary feature of the image.

[0043] (4) Image hash value calculation step: According to the required hash value length, some components of the secondary features of the image are intercepted, and each component of the secondary features of the intercepted image is binarized to 0 or 1 according to the average value of the partial components of the secondary features of the intercepted image.

[0044] In the specific implementation, for the VGGNet-16 network, if the required image hash value is 64 bits, the 64 components of the secondary features of the image are intercepted to obtain Y'=(y1,y2,…,y 64 Then calculate the average value y = (y1+y2+…+y 64 ) / 64. If the hash value is Z=(z1,z2,…,z 64 ), then z i The calculation method is: if y i ≥y, then z i =1, otherwise z i = 0. Specifically, the 64 components are the first 64 components of the secondary features of the image.

[0045] Embodiment 1:

[0046] A method for calculating an image hash value using a convolutional neural network and an orthogonal transformation, comprising:

[0047] (1) Neural network selection step: Select an appropriate image classification convolutional neural network and use the training images to train the convolutional neural network to achieve a stable state with high accuracy.

[0048] (2) Image feature acquisition step: Input the image for which the hash value needs to be calculated into the trained convolutional neural network, and retain the input of the convolutional neural network class probability layer as the primary feature of the image.

[0049] (3) Image feature transformation step: Using orthogonal transformation, the primary features of the image are transformed into the secondary features of the image.

[0050] (4) Image hash value calculation step: According to the required hash value length, some components of the secondary features of the image are intercepted, and each component of the secondary features of the intercepted image is binarized to 0 or 1 according to the average value of the partial components of the secondary features of the intercepted image.

[0051] Furthermore, in step (1), an appropriate convolutional neural network for image classification is selected.

[0052] Furthermore, in step (2), the image for which the hash value needs to be calculated is input into the trained convolutional neural network, and the input of the convolutional neural network class probability layer is retained as the primary feature X of the image. If the number of components of X is not an integer power of 2 (for example, 2, 4, 8, 16, 32, 64, etc.), a certain number of components can be added so that the number of components of X is an integer power of 2.

[0053] Furthermore, in step (3), the primary feature X of the image is transformed into the secondary feature Y of the image using orthogonal transformation such as discrete cosine transformation, Hadamard transformation, Haar transformation, etc.

[0054] Furthermore, in the step (4), according to the required hash value length N, the first N components of the secondary feature of the image are intercepted, and according to the average value of the first N components of the secondary feature of the intercepted image, each component of the secondary feature of the intercepted image is binarized to 0 or 1, that is, if the value of a component is greater than or equal to the average value, the corresponding bit in the hash value is 1, otherwise, the corresponding bit in the hash value is 0.

[0055] The method for calculating image hash values ​​proposed in the present invention can effectively extract feature information of the input image extracted by the convolutional neural network, so that the hash values ​​of similar images are similar, and can control the length of the hash value as needed, which is conducive to the rapid retrieval of image big data and has important application value in image big data applications.

[0056] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations, characterized in that: The method comprises the following steps: S1, neural network selection step: select an appropriate image classification convolutional neural network, and use the training images to train the convolutional neural network to achieve a stable state; S2, image feature acquisition step: input the image for which the hash value needs to be calculated into the trained convolutional neural network, and retain the input of the convolutional neural network class probability layer as the primary feature of the image; S3, image feature transformation step: using orthogonal transformation, transforming the primary features of the image into secondary features of the image; S4, image hash value calculation step: according to the required hash value length, intercepting some components of the secondary features of the image, and binarizing each component of the secondary features of the intercepted image to 0 or 1 according to the average value of the partial components of the secondary features of the intercepted image; in, In step S1, the convolutional neural network is a VGGNet-16 convolutional neural network. The VGGNet-16 network has 16 layers, of which the first 13 layers are convolution and pooling layers, the last 3 layers are fully connected layers, and the output of the last fully connected layer directly enters the class probability layer, i.e., the SoftMax layer; In step S3, the orthogonal transform is discrete cosine transform, Hadamard transform or Haar transform; In step S3, before the transformation, if the number of components of the primary feature X of the image is not an integer power of 2, a certain number of components is added so that the number of components of X is an integer power of 2; For the VGGNet-16 network, if the first-level feature of the image is a 1000-dimensional vector X = (x1, x2, ..., x 1000 ), then add 24 components with a value of 0 and expand it to a 1024-dimensional vector X'=(x1,x2,…,x 1024 ), where x 1001 to x 1024 The values ​​of are all 0; then, X' is used as the input of the orthogonal transformation, and the orthogonal transformation is performed to obtain the vector Y = (y1, y2, ..., y 1024 ), Y is the secondary feature of the image.

2. The method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations as claimed in claim 1, characterized in that: In step S2, the image for which the hash value needs to be calculated is scaled to an image that meets the input requirements of the convolutional neural network and then input into the trained convolutional neural network.

3. The method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations as claimed in claim 2, characterized in that: In step S2, the input of the VGGNet-16 convolutional neural network is an original image of size 224×224, the output is 1000 categories, and the first-level feature of the image is a 1000-dimensional vector.

4. The method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations as claimed in claim 1, characterized in that: The step S4 specifically includes: according to the required hash value length N, the first N components of the secondary feature of the image are intercepted, and according to the average value of the first N components of the secondary feature of the intercepted image, each component of the secondary feature of the intercepted image is binarized to 0 or 1, that is, if the value of a component is greater than or equal to the average value, the corresponding bit in the hash value is 1, otherwise, the corresponding bit in the hash value is 0.

5. The method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations as claimed in claim 4, characterized in that: For the VGGNet-16 network, if the required image hash value is 64 bits, the 64 components of the secondary features of the image are intercepted to obtain Y'=(y1,y2,…,y 64 ); then calculate the average value y = (y1+y2+…+y 64 ) / 64; if the hash value is Z=(z1,z2,…,z 64 ), then z i The calculation method is: if y i ≥y, then z i =1, otherwise z i =0.

6. The method for calculating image hash values ​​using convolutional neural networks and orthogonal transformations as claimed in claim 5, characterized in that: The 64 components are the first 64 components of the secondary features of the image.

Images