A method for calculating image block similarity based on deep learning
Through the deep learning twin autoencoder network model, the feature extraction and similarity calculation of image blocks is solved, the problem of noise impact in image block comparison is improved, the accuracy and model stability are improved, and parameter debugging is simplified.
Patent Information
- Application Number
- CN202010999365.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-22
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to effectively process noise-free image blocks in image block comparison, resulting in low accuracy and complex parameter debugging.
Using a deep learning-based approach, the image blocks are feature extraction and similarity calculations are performed using a twin autoencoder network model, and the hyperparameter optimization model is trained and adjusted to improve accuracy.
It significantly improves the accuracy of image block comparison, simplifies parameter debugging, reduces the complexity of manually extracting features, and obtains more stable and more accurate image block similarity calculation results.
Smart Images

Figure CN112163609B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method for calculating image block similarity based on deep learning. Background Art
[0002] Image blocks are parts of images that contain detailed texture and structural information. Although they are the description of basic image features, image block-based methods can always achieve good results in many image fields. For example, they have achieved outstanding results in image filtering, computer vision, image modeling, etc. In the field of image processing, image block comparison is also an important task, and image blocks with noise are often more difficult to compare. Summary of the invention
[0003] The purpose of the present invention is to provide a method for calculating image block similarity based on deep learning, which can at least solve some of the defects in the prior art.
[0004] To achieve the above object, an embodiment of the present invention provides the following technical solution: a method for calculating image block similarity based on deep learning, comprising the following steps:
[0005] S1, according to the training image set given by the user, add noise and divide the image block set to obtain the training image block set X train ;
[0006] S2, select the object to be tested, add noise and divide the image block set to obtain the test image block set X test ;
[0007] S3, establish a twin autoencoder network model;
[0008] S4, input the training image block set X into the twin autoencoder network model train To train the twin autoencoder network model;
[0009] S5, the test image block set X test Input the trained twin autoencoder network model, and obtain the feature information dataset corresponding to the test image block set according to the trained network weights;
[0010] S6, after obtaining the feature information data set, calculating the similarity of each pair of image blocks according to the feature information data set.
[0011] Furthermore, in step S3, the established twin autoencoder network model contains seven hyperparameters, namely, learning rate, number of iterations, type of activation function, input data size, number of neural nodes in the encoding layer, number of neural nodes in the inverse encoding layer, and output data size.
[0012] Furthermore, in step S4, the twin autoencoder network model is trained by adjusting seven hyperparameters, and finally obtaining train The cost function of the twin autoencoder network model is:
[0013]
[0014] Among them, M represents the number of valid data in the training set, W represents the weight of the twin autoencoder network model, and X train represents the effective data set in the training set, X 1i Represents the training set X train Divide into two groups of data with the same number of data. In the first group, X 2i Represents the training set X train Divide into two groups of data with the same number of data, and focus on the second group of data, F 1i Represents the first set of data X in the training set 1i The feature information dataset extracted by the encoding layer, F 2i Represents the second set of data X in the training set 2i The feature information dataset extracted by the encoding layer, β represents the regularization factor in the training process, and i represents the training set X train The ordinal number of the dataset in each group of a dataset divided into two groups of equal number.
[0015] Furthermore, the adaptive gradient descent algorithm is used to train the twin autoencoder network model so that the cost function J(W; X train ) is minimized to obtain the minimized cost function of the twin autoencoder network model.
[0016] Furthermore, before the input in step S5, the test image block set X test Divide into two sets of test image blocks X with the same number of test_1k and X test_2k , where k represents the sequence number of the data in each set of test image blocks, and k is a natural number; then X test_1k and X test_2k At the same time, the trained twin autoencoder network model is input to obtain the feature information dataset F corresponding to the two. test_1k and F test_2k .
[0017] Further, in step S6, according to the feature information dataset F test_1k and F test_2k , calculate the similarity of each pair of image patches:
[0018]
[0019] Where k represents the test set X test The ordinal number of the dataset in each group of a dataset divided into two groups of equal number.
[0020] Furthermore, in step S3, the established twin autoencoder network model is a dual-channel twin autoencoder network model, which includes dual input layers, encoding layers, de-encoding layers, and dual output layers.
[0021] Furthermore, in step S3, the learning rate of the established twin autoencoder network model is α, the number of iterations is t, the encoding layer has J (j×j) neural nodes, the inverse encoding layer has J neural nodes, the input layer receives image block data of size 1×J, the encoding layer outputs feature information data of size 1×J, the output layer outputs data of size 1×J, the two input layers receive two image block data at the same time, the two output layers output two data at the same time, and both input layers use Relu function as activation function.
[0022] Furthermore, in both step S1 and step S2, Gaussian noise with a noise intensity of σ is used, and in step S1, the division method is to divide each picture into a set of image blocks of n in size j×j, and the image block set is standardized (0~1), while in step S2, the division method is to divide each picture into a set of image blocks of m in size j×j, and the image block set is standardized (0~1).
[0023] Compared with the prior art, the present invention has the following beneficial effects:
[0024] 1. By comparing the image blocks through the similarity of the acquired image blocks, a higher accuracy can be obtained, and the parameter debugging is simple.
[0025] 2. By making full use of the advantages of parameter sharing and automatic extraction of image feature information of the dual-channel twin autoencoder network model, the complexity of manually extracting image features is significantly reduced, thereby making the optimal model structure of the learning method for image block similarity calculation based on deep learning simple.
[0026] 3. Input a set of n training image blocks into the dual-channel twin autoencoder network, and obtain the cost function of the dual-channel twin autoencoder network model of the training set by adjusting 7 hyperparameters. In multiple experiments, the model with the smallest cost function is selected as the optimal model. The optimal model is the model with the best data generalization ability of the dual-channel twin autoencoder network model, which makes the optimal model of the learning method stable.
[0027] 4. By minimizing the cost function through multiple iterations, the error of the optimal model is reduced, so that the learning method is more accurate and the image block similarity calculation results are more accurate.
[0028] 5. The optimal model has a simple structure, good stability, and the learning method for calculating image block similarity based on deep learning has more accurate similarity relationships. The obtained image block similarity can better describe the relationship between two image blocks. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A flowchart of a method for calculating image block similarity based on deep learning provided by an embodiment of the present invention;
[0030] Figure 2 A schematic diagram of a dual-channel twin autoencoder network model for an image block similarity calculation method based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] See also Figure 1 and Figure 2 The embodiment of the present invention provides a method for calculating image block similarity based on deep learning, comprising the following steps: S1, adding noise and dividing the image block set according to the training image set provided by the user to obtain the training image block set X train ; S2, select the object to be tested, add noise and divide the image block set to obtain the test image block set X test ; S3, establish a twin autoencoder network model; S4, input the training image block set X into the twin autoencoder network model train To train the twin autoencoder network model; S5, the test image block set X test Input the trained twin autoencoder network model, and obtain the feature information data set corresponding to the test image block set according to the trained network weights; S6, after obtaining the feature information data set, calculate the similarity of each pair of image blocks according to the feature information data set. In this embodiment, the established twin autoencoder network model is trained, and then the test image block set X is input. testInput the trained model, and obtain the feature information data set corresponding to the test image block set according to the trained network weights, and then calculate the similarity of each pair of image blocks according to the feature information data set, so that the image blocks can be compared by the similarity of the obtained image blocks, which can obtain a higher accuracy rate, and the parameter debugging is simple. Preferably, the learning rate of the established twin autoencoder network model is α, the number of iterations is t times, the encoding layer has J (j×j) neural nodes, the inverse encoding layer has J neural nodes, the input layer receives image block data of size 1×J, the encoding layer outputs feature information data of size 1×J, the output layer outputs data of size 1×J, the two input layers receive two image block data at the same time each time, and the two output layers output two data at the same time each time, and both input layers use Relu function as activation function. Specifically:
[0033] According to the 1000 training images of the PASCAL Vocdevkit dataset, Gaussian noise with a noise intensity of 20 is added, and each image is divided into a size of 5×5, that is, each image block has 25 features. The training image set is divided into a set of 7131304 image blocks, and the image block set is standardized (0-1). The data set consisting of 7131304 measurement valid image block data is called the training image block set X train (See Table 1 for details).
[0034] Table 1 7131304 valid data sets
[0035]
[0036] Then select the images to be tested, Boat, Fingerprint, Flowers, House, Lena, and add Gaussian noise with a noise intensity of 20. Each image is divided into 5×5 blocks, that is, each image block has 25 features, and the image block set is standardized (0-1). The data set consisting of 872784 measured valid image block data is called the test image block set X test (See Table 2 for details).
[0037] Table 2 872784 valid data sets
[0038]
[0039] As an optimization solution of the embodiment of the present invention, please refer to Figure 2The established twin autoencoder network model is a dual-channel twin autoencoder network model, which includes dual input layers, encoding layers, reverse encoding layers, and dual output layers. The established twin autoencoder network model contains seven hyperparameters, namely, learning rate, number of iterations, type of activation function, input data size, number of neural nodes in the encoding layer, number of neural nodes in the reverse encoding layer, and output data size. The way to train the twin autoencoder network model is to adjust the seven hyperparameters, and finally obtain the training image block set X based on the training image block set X. train The cost function of the twin autoencoder network model is:
[0040]
[0041] Among them, M represents the number of valid data in the training set, W represents the weight of the twin autoencoder network model, and X train represents the effective data set in the training set, X 1i Represents the training set X train Divide into two groups of data with the same number of data. In the first group, X 2i Represents the training set X train Divide into two groups of data with the same number of data, and focus on the second group of data, F 1i Represents the first set of data X in the training set 1i The feature information dataset extracted by the encoding layer, F 2i Represents the second set of data X in the training set 2i The feature information dataset extracted by the encoding layer, β represents the regularization factor in the training process, and i represents the training set X train The ordinal number of the dataset in each group of a dataset divided into two groups of equal number.
[0042] Specifically, the training image block set X is input into the dual-channel twin autoencoder network model. train , by adjusting 7 hyperparameters (see Table 3 for details).
[0043] Table 3 Adjustment of the seven hyperparameters corresponding to the dual-channel twin autoencoder network model
[0044]
[0045] Get the cost function J(W;X) of the dual-channel twin autoencoder network model of the training set train ):
[0046]
[0047] In the above formula:
[0048] M represents the number of valid data in the training set, M = 7131304;
[0049] W represents the weight of the dual-channel twin autoencoder network model, and W is obtained by iterative optimization of the dual-channel twin autoencoder network model;
[0050] X train Represents the valid data set in the training set (see Table 1 for details);
[0051] X 1i Represents the training set X train Divide into two groups of data with equal numbers, and focus on the first group of data (see rows 1 to 3565652 in Table 1 for details);
[0052] X 2i Represents the training set X train Divide into two groups of data with equal numbers and concentrate the second group of data (see rows 3565653 to 7131304 in Table 1 for details);
[0053] F 1i Represents the first set of data X in the training set 1i The feature information dataset extracted by the coding layer (see Table 4 for details); F 2i Represents the second set of data X in the training set 2i The feature information dataset extracted by the coding layer (see Table 5 for details);
[0054] β represents the regularization factor during training, β = 0.01;
[0055] i represents the training set X train The ordinal number of the data set in each of the two data sets divided into the same number of groups, i is a natural number from 1 to 3565652;
[0056] The adaptive gradient descent algorithm is used to train the cost function J(W; X train ) is minimized; accordingly, the cost function of the dual-channel twin autoencoder network model after minimization is obtained:
[0057] J(W;X train )=0.0012.
[0058] Training set data X 1i and X 2i The feature information data sets extracted by the coding layer are detailed in Table 4 and Table 5. Among them: Table 4 is the first set of data X in the training set 1i The feature information dataset extracted by the encoding layer,
[0059] Table 4 Feature information dataset F 1i
[0060]
[0061] Table 5 is the second set of data X for the training set 2i The feature information dataset extracted by the encoding layer,
[0062] Table 5 Feature information dataset F 2i
[0063]
[0064] As an optimization solution of the embodiment of the present invention, before the input in step S5, the test image block set X is test Divide into two sets of test image blocks X with the same number of test_1k and X test_2k , where k represents the serial number of the data in each set of test image blocks, k is a natural number, k is a natural number from 1 to 436392 (see Table 2 for details), and the data in rows 1 to 436392 in Table 2 are the test image block set X test_1k , 436393~872784 rows of data are the test image block set X test_2k ; Then X test_1k and X test_2k At the same time, the trained twin autoencoder network model is input to obtain the feature information dataset F corresponding to the two. test_1k and F test_2k . Training set data X 1i and X 2i The feature information datasets extracted by the coding layer are detailed in Tables 6 and 7.
[0065] in:
[0066] Table 6 shows the first set of data X in the training set test_1k Feature information dataset extracted by the encoding layer
[0067] Table 6 Feature information dataset F test_1k
[0068]
[0069] Table 7 is the second set of data X for the training set test_2k Feature information dataset extracted through the coding layer Table 7 Feature information dataset F test_2k
[0070]
[0071] Further optimize the above scheme, according to the feature information dataset F test_1k and F test_2k , calculate the similarity of each pair of image patches:
[0072]
[0073] Where k represents the test set X test The ordinal number of the data set in each group of the data set divided into two groups of equal number, k is a natural number from 1 to 436392.
[0074] S(F test_1k ,F test_2k ) is the calculated similarity of the image block (see Table 8 for details)
[0075] Table 8 Similarity S(F test_1k , F test_2k )
[0076]
[0077] As an optimization scheme for an embodiment of the present invention, Table 9 shows the image denoising performance of each algorithm, NLM represents the non-local mean denoising algorithm (Non-Local Means), NLM-AE represents the NLM algorithm using the image block similarity calculation method based on deep learning, and the peak signal-to-noise ratio PSNR (Peak signal-to-noise ratio) is used to compare the algorithm performance.
[0078] Table 9 Denoising performance comparison of various algorithms (noise intensity is 20)
[0079]
[0080] It can also be seen from Table 9 that the NLM algorithm using the image block similarity calculation method based on deep learning has improved denoising effect on the same image under the same noise intensity, and has better denoising performance. Experiments can prove that the learning method of image block similarity calculation based on deep learning of the present invention improves the accuracy of image block similarity calculation.
[0081] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for calculating image block similarity based on deep learning, characterized in that: The steps include: S1, according to the training image set given by the user, add noise and divide the image block set to obtain the training image block set X train ; S2, select the object to be tested, add noise and divide the image block set to obtain the test image block set X test ; S3, establish a twin autoencoder network model; S4, input the training image block set X into the twin autoencoder network model train To train the twin autoencoder network model; S5, the test image block set X test Input the trained twin autoencoder network model, and obtain the feature information dataset corresponding to the test image block set according to the trained network weights; S6, after obtaining the feature information data set, the similarity of each pair of image blocks is calculated according to the feature information data set. In step S3, the established twin autoencoder network model contains seven hyperparameters, namely, learning rate, number of iterations, type of activation function, input data size, number of neural nodes in the encoding layer, number of neural nodes in the reverse encoding layer, and output data size. In step S4, the twin autoencoder network model is trained by adjusting seven hyperparameters to obtain train The cost function of the twin autoencoder network model is: Among them, M represents the number of valid data in the training set, W represents the weight of the twin autoencoder network model, and X train represents the valid data set in the training set, X 1i Represents the training set X train Divide into two groups of data with the same number of data. In the first group, X 2i Represents the training set X train Divide into two groups of data with the same number of data, and focus on the second group of data, F 1i Represents the first set of data X in the training set 1i The feature information dataset extracted by the encoding layer, F 2i Represents the second set of data X in the training set 2i The feature information dataset extracted by the encoding layer, β represents the regularization factor in the training process, and i represents the training set X train The data set is divided into two groups of equal number, and the data set ordinal number in each group. In step S3, the established twin autoencoder network model is a dual-channel twin autoencoder network model, which includes dual input layers, encoding layers, inverse encoding layers, and dual output layers. In step S3, the learning rate of the established twin autoencoder network model is α, the number of iterations is t times, the encoding layer has J neural nodes, the inverse encoding layer has J neural nodes, the size of the input layer receiving image block data is 1×J, the size of the feature information data output by the encoding layer is 1×J, the size of the output layer output data is 1×J, the two input layers receive two image block data at the same time each time, and the two output layers output two data at the same time each time. Both input layers use Relu function as activation function.
2. The method for calculating image block similarity based on deep learning as claimed in claim 1, characterized in that: The twin autoencoder network model is trained using the adaptive gradient descent algorithm to make the cost function J(W; X train ) is minimized to obtain the minimized cost function of the twin autoencoder network model.
3. The method for calculating image block similarity based on deep learning as claimed in claim 1, characterized in that: Before the input in step S5, the test image block set X test Divide into two sets of test image blocks X with the same number of test_1k and X test_2k , where k represents the sequence number of the data in each set of test image blocks, and k is a natural number; then X test_1k and X test_2k At the same time, the trained twin autoencoder network model is input to obtain the feature information dataset F corresponding to the two. test_1k and F test_2k .
4. The image block similarity calculation method based on deep learning as claimed in claim 3, characterized in that: In step S6, according to the feature information dataset F test_1k and F test_2k , calculate the similarity of each pair of image patches: Where k represents the test set X test The ordinal number of the dataset in each group of a dataset divided into two groups of equal number.
5. The method for calculating image block similarity based on deep learning as claimed in claim 1, characterized in that: In both step S1 and step S2, Gaussian noise with a noise intensity of 20 is used, and in step S1, the division method is to divide each picture into a set of n image blocks of size j×j, and the image block set is 01 standardized, while in step S2, the division method is to divide each picture into a set of m image blocks of size j×j, and the image block set is 01 standardized.
Citation Information
Patent Citations
A contrast learning image quality evaluation method based on a twin network
CN109727246A
An image classification and recognition method based on a twin network
CN109840556A
