Fabric Pattern Retrieval Algorithm Based on SURF and VLAD Feature Encoding

Abstract

The present invention belongs to the technical field of fabric image retrieval, and relates to a fabric pattern retrieval algorithm based on SURF and VLAD feature coding. First, the present invention uses an image vision feature training set for clustering to obtain a visual feature codebook; then extracts the SURF features of all fabrics in the database to construct an image vision feature library, calculates the feature residuals by the obtained codebook for quantization and coding; further performs principal component analysis on the obtained coded features to achieve the effect of dimensionality reduction, reducing unnecessary overhead caused by storing visual feature vectors and calculating feature distances; constructs an index for the dimensionality-reduced fabric visual feature vectors using the ball-tree algorithm, and then uses the feature vectors of the query fabric image for querying and sorting and returning. The algorithm is based on SURF and VLAD, can adapt to a variety of complex and changeable pattern fabrics, can not only effectively detect different complex patterns, but also has a relatively fast retrieval speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of fabric image retrieval, and relates to a fabric pattern retrieval algorithm based on SURF and VLAD feature encoding. Background Art

[0002] With the digital and intelligent transformation and upgrading of the textile and garment industry, as one of the main raw materials of clothing, textile fabrics have diversified due to the improvement of consumption levels and the trend of clothing differentiation. Diversification is the only way for fabric manufacturers. The most significant change brought about by fabric diversification is the variety of fabric patterns, which has led to high inventory management costs and difficult management for enterprises. Efficiently retrieving fabrics with similar patterns from a vast fabric library is of great significance for enterprises to reduce costs and improve production efficiency. Content-Based Image Retrieval (CBIR) technology is an effective way to solve this problem and has been widely used in textile fabric retrieval.

[0003] Currently, the main content-based image retrieval methods can be divided into low-level visual feature-based retrieval and high-level semantic image retrieval. High-level semantic features generally automatically extract image features through a CNN model, which has certain robustness and generalization. However, training a complete and effective deep CNN model requires a large amount of labeled data, a long training time, and huge computing resources. Low-level visual features mainly include texture, shape, and color features. In the field of CBIR, different features or their combinations are usually used to represent images according to different needs. When low-level visual features are used for image retrieval, no manual labeled samples are required, the training time is relatively short, and the computing cost is low, and they have strong pertinence. Kang Feng et al. used SURF of maximally stable extremal regions of images to retrieve fabric patterns; Xiang Zhong proposed an image retrieval algorithm combining color and edge features, which can effectively retrieve printed fabric patterns with obvious contours; Cao Xia et al. used GLCM, GGCM, and LBP for hierarchical matching and fusion algorithms to achieve lace pattern retrieval. The above-mentioned literatures only involve simple or single fabric pattern retrieval and are not suitable for retrieving fabric patterns with complex structures and large quantities. The patterns of fabrics stored in fabric enterprises are diverse and numerous, which is not conducive to fabric enterprises efficiently retrieving fabrics with similar patterns from a vast fabric library. Liu Ying et al. proposed an algorithm that combines the local aggregation descriptor (VLAD) based on image SIFT features and global features to classify crime scene images. The combination of SIFT and VLAD has a good effect on describing the content of crime scene images and has important reference significance for image retrieval.

[0004] Therefore, in view of the problem of retrieving large quantities of fabric patterns with complex structures, the present invention proposes a fabric pattern retrieval algorithm based on SURF and VLAD feature encoding. The SURF feature also has scale invariance and rotation invariance, and its feature dimension is only half of that of SIFT, which can effectively detect and recognize the complex structures of fabric patterns; VLAD encoding can unify the feature dimensions between different images and reduce the computational complexity, thus solving the problem of low retrieval efficiency for a large number of patterns. The algorithm proposed by the present invention for large quantities of fabric patterns with complex structures has a relatively high retrieval efficiency. Summary of the Invention

[0005] The object of the present invention is to provide a fabric retrieval algorithm for large quantities of fabric with complex pattern structures, aiming to quantify and encode the visual features of fabric images to reduce the space requirements for storing visual features of images; further reduce the dimension of visual features through component analysis to reduce the overhead of calculating feature distances; and construct a hypersphere index through the ball-tree algorithm to filter out irrelevant query results, thereby further improving the retrieval speed.

[0006] The technical solution of the present invention is as follows:

[0007] A fabric pattern retrieval algorithm based on SURF and VLAD feature encoding realizes image retrieval by quantifying, encoding, constructing an index, and querying the visual features of an image. The method includes:

[0008] Extract the SURF features of the fabric image and perform clustering on them to construct a codebook.

[0009] According to the constructed codebook, quantify and encode the visual features of the image, and further perform principal component analysis on the obtained feature vectors for dimensionality reduction.

[0010] Use the visual features of the query image to query the index constructed using the ball-tree algorithm to obtain a query candidate set, calculate the distances between the candidate set and the visual features of the query image for sorting, thereby completing the retrieval of the visual features of the image.

[0011] The present invention first uses a visual feature training set of an image for clustering to obtain a visual feature codebook; then extracts the SURF features of all fabrics in the database to construct an image visual feature library, calculates the feature residuals using the obtained codebook for quantification and encoding; further performs principal component analysis on the obtained encoded features to achieve the effect of dimensionality reduction, reducing unnecessary overhead caused by storing visual feature vectors and calculating feature distances; constructs an index using the ball-tree algorithm for the dimensionality-reduced fabric visual feature vectors, and then uses the feature vectors of the query fabric image for querying and sorting and returning. The specific steps are as follows:

[0012] (1) Construction of Fabric Feature Library and Generation of Codebook

[0013] First, extract SURF features from the fabric images in the fabric image library and store their features; then, perform iterative clustering on the SURF features in the training set using the Mini Batch K-Means algorithm to generate a codebook.

[0014] (2) Quantization and Encoding of Image Visual Features

[0015] First, use the trained codebook to quantify each fabric image visual feature descriptor one by one and calculate the residuals, then splice the obtained residuals into corresponding feature vectors, and then perform dimensionality reduction on the obtained feature vectors to improve the retrieval speed and reduce the overhead.

[0016] (3) Index Construction

[0017] First, the Ball-tree recursively divides the data into nodes defined by the centroid p and the radius r, so that each point in the node is located within the defined hypersphere. As Figure 2 shown, select a point P that is farthest from the current center of the circle 1 , and a point P that is farthest from point P 1 . Assign all the points in the circle that are closest to these two points to the centers of these two clusters, then calculate the center points of each cluster and the minimum radius that contains all its affiliated observation points, and continuously recurse to obtain the nearest neighbor points, and finally construct an index. 2

[0018] (4) Image Query

[0019] First, extract the SURF features of the image to be inspected, perform VLAD encoding and principal component analysis dimensionality reduction on it to obtain a feature vector; then query the K nearest neighbors of the target node in the constructed ball-tree and return the corresponding images.

[0020] Advantages of the present invention:

[0021] The present invention designs a new fabric pattern retrieval algorithm that can effectively detect fabric patterns. The algorithm is based on SURF and VLAD, and can adapt to a variety of complex and changeable pattern fabrics. It can not only effectively detect different complex patterns, but also has a relatively fast retrieval speed. Description of the Drawings

[0022] Figure 1 is a schematic diagram of the technical route of the method of the present invention;

[0023] Figure 2 is the ball tree structure of the data set;

[0024] Figure 3 is a schematic diagram of the generation of the feature descriptor;

[0025] Figure 4 It is a schematic diagram of VLAD coding; Specific implementation manner

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] The fabric pattern retrieval algorithm based on SURF and VLAD feature coding is as follows:

[0028] Step 1, image feature extraction. First, calculate the Haar wavelet responses in the x and y directions within a circular neighborhood with a radius of 6s around the interest point. Statistically sum the horizontal and vertical Haar wavelet features of all points within a 60-degree sector. Then, rotate the sector at a certain interval and statistically sum the Haar wavelet feature values in this area again. Finally, take the direction of the sector with the largest value as the main direction of the feature point. Rotate a 20s×20s square area centered at the position of the interest point to coincide with the main direction, divide this square area into 4×4 sub-regions, and sum and take the absolute value sum of the wavelet responses dx and dy in the main direction and its perpendicular direction within each sub-region, as Figure 3 shown, to obtain a vector v with a length of 4. Therefore, a total of 64-dimensional vectors can be obtained from 16 regions;

[0029] Step 2, codebook construction. The codebook of this algorithm is generated using the Mini Batch K-Means algorithm. Instead of using all samples in each iteration, an equal amount of samples is taken each time, and then the central nodes are updated. The data is updated on each small sample set. For each small batch, the updated centroid is obtained by calculating the average value, and the data of the small batch is assigned to this centroid. As the number of iterations increases, the change of the centroid becomes smaller and smaller until the centroid is stable or the specified number of iterations is reached, and the clustering is completed and the codebook is generated;

[0030] Step 3, VLAD coding and principal component analysis. Use the NN nearest neighbor method to divide the SURF descriptor S of a fabric picture i,j into the class it belongs to, where i is the number of the clustering center, i = {1, 2,..., k}, k is the number of clustering centers, and j is the number of the descriptor in this clustering. Then, calculate the residuals in each clustering. The residuals can be calculated by the formula r i,j = s i,j - c iCalculated. In a cluster, the residual of each SURF descriptor of an image needs to be calculated. For n i descriptors, n i residuals are obtained. The VLAD of a cluster can be obtained through Equation . Putting k V i together forms a vector of length k×64, which is the VLAD. The generation process of VLAD is as shown in Figure 4 .

[0031] Since the fabric has many patterns, a lot of local details, and large structural differences, when using the SURF algorithm for extraction, more repetitive pattern units are detected. Too many descriptors lead to a relatively high-dimensional VLAD, which contains a lot of redundant information and is not conducive to computer storage and calculation. Therefore, principal component analysis is used to reduce the dimension of VLAD. The feature matrix of VLAD is V(V 1 , V 2 , …, V k ). The following objective function is obtained through optimization.

[0032] Y PCA = W T V

[0033] where V represents the VLAD feature vector of the extracted training positive samples, and Y PCA represents the VLAD feature vector after dimensionality reduction. Therefore, the feature vectors are the first K eigenvalues λ 1 , λ 2 , …, λ k corresponding feature vectors (Y PCA1 , Y PCA2 , …, Y PCAK );

[0034] Step 4, Index construction and query. The Ball-tree recursively divides the data into nodes defined by the centroid p and the radius r, so that each point in the node lies within the defined hypersphere. As shown in Figure 2 , select a point p 1 that is farthest from the current center of the circle, and a point p 1 that is farthest from point p 2 . Assign all the points in the circle that are closest to these two points to the centers of these two clusters, and then calculate the center point of each cluster and the minimum radius that contains all its affiliated observation points. Continuously recurse to obtain the nearest neighbor points, and finally construct the index.

[0035] When using the Ball-tree for query, it is used to search for the K nearest neighbors of the target point P. Assume that D S is the minimum distance between the target node and the nearest neighbor point, D S = max x∈pin|x - t|, the distance between t and the current node can be defined by the following formula, where pin represents the nearest neighbor found. If D S < D N , then expand a node. If the current node is a leaf, add each data point of this node to the result list. When the size of the result list exceeds K, remove the farthest point from the list and update D S and further execute.

[0036] D N = max{D N.parent , |t - center(N)| - radius(N)}.

Claims

1. Fabric pattern retrieval algorithm based on SURF and VLAD feature encoding, Characterized in that, The steps are as follows: First step: Construction of fabric feature library and generation of codebook Extract SURF features from fabric images in the fabric image library and store their features; Then, use the Mini Batch K-Means algorithm to iteratively cluster the SURF features in the training set to generate a codebook; Second step: Image visual feature quantization and encoding Using the trained codebook, quantize and calculate the residuals for each fabric image visual feature descriptor one by one, then splice the obtained residuals into corresponding feature vectors, and then perform dimensionality reduction processing on the obtained feature vectors; Third step: Index construction The ball-tree recursively divides the data into nodes defined by a centroid p and a radius r, such that every point in a node lies within the defined hypersphere; select a point P that is farthest from the current center of the circle 1 , and the point P 1 that is farthest from point P 2 , assign all the points in the circle that are closest to these two points to the centers of the two clusters, then calculate the center point of each cluster and the minimum radius that encloses all its associated observation points, recursively obtain the nearest neighbor points continuously, and finally construct the index; Fourth step: Image query Extract the SURF features of the image to be inspected, perform VLAD encoding and principal component analysis dimensionality reduction on it to obtain a feature vector; then query the K nearest neighbors of the target node in the constructed ball-tree and return the corresponding images; The generation of the codebook in the first step is specifically as follows: Instead of using all samples in the iteration, equal amounts of samples are sampled each time, and then the central node is updated. The data is updated on each small sample set; for each small batch, calculate the average value to obtain the updated centroid, and assign the data of the small batch to this centroid. As the number of iterations increases, the change of the centroid becomes smaller and smaller until the centroid is stable or reaches the specified number of iterations, and the clustering is completed and the codebook is generated; Calculate the Haar wavelet responses in the x and y directions within a circular neighborhood with a radius of 6s around the interest point. Statistically sum the horizontal and vertical Haar wavelet features of all points within a 60-degree sector. Then, rotate the sector at a certain interval and statistically sum the Haar wavelet feature values in the area again. Finally, take the direction of the sector with the largest value as the main direction of the feature point; rotate a 20s×20s square area centered on the position of the interest point to coincide with the main direction, divide this square area into 4×4 sub-regions, sum the wavelet responses dx and dy in the main direction and its perpendicular direction within each sub-region and perform absolute value summation to obtain a vector v with a length of 4. Therefore, a total of 64-dimensional vectors can be obtained from 16 regions.

2. The fabric pattern retrieval algorithm based on SURF and VLAD feature encoding according to claim 1, Characterized in that, The specific operation of the second step is as follows: Use the NN nearest neighbor method to classify the SURF descriptors S of a fabric image i,j into their respective classes, where i is the number of the cluster center, i = {1, 2, …, k}, k is the number of cluster centers, and j is the number of the descriptor in the cluster; then, calculate the residuals in each cluster, and the residuals are calculated by the formula r i,j = s i,j - c i ; in a cluster, the residuals of each SURF descriptor of an image need to be calculated, and n i descriptors will result in n i residuals. The VLAD of a cluster is obtained by the formula ; Use principal component analysis to reduce the dimension of VLAD. The feature matrix of VLAD is V(V 1 ,V 2 ,…,V k ); The following objective function is obtained through optimization; Y PCA = W T V Among them, V represents the extraction of the VLAD feature vector of the training positive samples, and Y PCA represents the VLAD feature vector after dimensionality reduction; therefore, the feature vectors are the first K eigenvalues λ 1 , λ 2 , …, λ k corresponding to the eigenvectors (Y PCA1 , Y PCA2 , …, Y PCAK ).

3. The fabric pattern retrieval algorithm based on SURF and VLAD feature encoding according to claim 1 or 2, Characterized in that, In the fourth step, when using ball-tree query, the specific operation is as follows: used to search for the K nearest neighbors of the target point P; assume D S is the minimum distance between the target node and the nearest neighbor point, D S = max x∈pin |x - t|, the distance D between t and the current node N can be defined by the following formula: D N = max{D N.parent , |t - center(N)| - radius(N)} where pin represents the discovered nearest neighbor; if D S <D N , then expand a node; if the current node is a leaf, add each data point of this node to the result list; When the size of the result list exceeds K, the farthest point will be removed from the list and D will be updated S Further execution is performed

Images