Fabric image retrieval method in complex scene

Abstract

The invention belongs to the field of fabric retrieval methods, and relates to a fabric image retrieval method in a complex scene. The method comprises the following steps: establishing a fabric image database; constructing a manual feature extraction model, and obtaining manual feature descriptions of the to-be-queried image and the image in the database image; building an image depth parallel representation model to process the fabric image, and obtaining depth feature descriptions of the image to be queried and the image in the database image; designing an image manual and depth feature fusion mode, fusing the obtained image manual and depth features, and comprehensively representing a fabric image; carrying out similarity measurement on the obtained fusion features of the image to be queried and the image in the database image, and outputting a retrieval result according to the similarity; and calling out fabric product details corresponding to the retrieval result image for guiding fabric production or e-commerce shopping recommendation. The method has high retrieval accuracy and robustness, and has great application potential in complex scenes such as industrial manufacturing and e-commerce platforms.

Description

Technical Field

[0001] The invention belongs to the field of fabric retrieval methods and relates to a fabric image retrieval method in complex scenes. Background Art

[0002] Image retrieval technology allows users to upload images or take photos to find similar or identical products, which is conducive to improving the efficiency and satisfaction of image queries. The image acquisition environment in industrial scenes and e-commerce platforms is usually difficult to standardize, and there are certain differences in shooting light sources, angles, resolutions and other conditions. The retrieval of such images belongs to image retrieval in complex scenes. Existing fabric image retrieval methods are usually content-based image retrieval. The images they face belong to images in standard acquisition environments. It is difficult to cope with image retrieval in complex environments, which limits the application scope of image retrieval. Existing methods are generally based on image feature extraction and similarity measurement. The extracted features mainly include manual features designed by humans and deep features extracted by deep networks. Manual features are simple to calculate and have strong interpretability, but the feature parameters are not universal. Deep features are usually automatically extracted by end-to-end training methods. They belong to high-level semantic features. The training process does not require human participation, but requires certain training data and computing resources. Existing manual and deep features are aimed at fabric images in standard acquisition environments, which are difficult to apply to fabric images collected in complex environments. Starting from the perspective of fusion of manual and deep features of fabric images, the features of fabric images can be fully characterized to achieve efficient and accurate image retrieval in complex scenes. Summary of the invention

[0003] The purpose of the present invention is to propose an efficient and accurate fabric image retrieval method in complex scenes, so as to quickly search for fabric images collected under complex conditions in scenes such as industry and e-commerce.

[0004] The technical solution of the present invention is as follows:

[0005] A fabric image retrieval method in a complex scene comprises the following steps:

[0006] S1: Establish fabric image database;

[0007] The fabric images are collected under different shooting light sources, angles, resolutions and other conditions.

[0008] S2: Build a manual feature extraction model to obtain manual feature descriptions of the query image and the database image;

[0009] The manual feature extraction model first preprocesses the fabric image, removes noise information in the image through filtering operations, and then extracts key points and direction information of the image as low-order features to describe the detail information of the fabric image.

[0010] Furthermore, the manual features include SIFT, SURF and ORB features.

[0011] S3: Build an image deep parallel representation model to process fabric images and obtain deep feature descriptions of images in the query image and database images;

[0012] The image depth parallel representation model has two parallel branches, such as Figure 2 As shown in the figure, they are used to extract scene features and category features of fabric images, describe the high-level semantic information of fabric images, ensure that the model can distinguish the same fabric in different scenes, and ensure that the retrieved images all belong to the same category.

[0013] The two parallel branches of the image deep parallel representation model use convolutional neural networks as the underlying framework to extract deep semantic features that represent fabric images in an end-to-end manner.

[0014] The convolutional neural network is a self-built compact network, which includes 1 input layer, 4 convolution and pooling layers, 3 fully connected layers and 1 output layer.

[0015] The two branches of the image depth parallel representation model are optimized using the triplet loss function, and the final loss function is formed by weight distribution. The designed loss function L can be expressed as

[0016] L=β1L1+β2L2

[0017] Where: β1 and β2 are the weights of the two loss functions respectively;

[0018] S4: Design a method to fuse manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image;

[0019] The image manual and deep feature fusion method is as follows:

[0020] A feature fusion module is introduced into the image depth parallel representation model described in S3, such as Figure 3 As shown, the manual and deep features of the image obtained by aggregating S2 and S3 are used to obtain the fusion features of the image in the query image and the database image, respectively.

[0021] The feature fusion module is based on the improved NetVLAD module, so that the manual features usually aggregated by VLAD can participate in the training in the image depth parallel representation model through the NetVLAD module, avoiding the artificial parameter design process in the manual features.

[0022] The improved NetVLAD module changes the original structure of processing feature maps in convolutional neural networks into a structure that can process feature maps and manual features at the same time, so as to achieve the purpose of fusing image manual and deep features.

[0023] S5: Measure the similarity between the fusion features of the query image and the image in the database obtained in S4, and output the retrieval result according to the size of the similarity;

[0024] The similarity measure is a distance calculation method between feature vectors, and different distance formulas can be selected for calculation.

[0025] S6: Retrieving fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

[0026] The fabric product details include product title, description and attribute information.

[0027] Beneficial effects of the present invention:

[0028] The present invention proposes a fabric image retrieval method in complex scenarios based on the retrieval needs in complex scenarios such as industry and e-commerce. The detailed information of the fabric image is captured by manual features, and the high-level semantic information of the image is extracted by deep features, so that the manual features are involved in model training, and the two features are fused to jointly characterize the intrinsic characteristics of the fabric image, so that the two features complement each other. A feature fusion module is introduced into the image deep parallel representation model, and the manual features are introduced into the deep representation model. The fabric image features are automatically extracted in an end-to-end training form, avoiding the parameter design and optimization process of manual features. This fusion method can improve the accuracy and robustness of fabric image retrieval, and can be applied to fabric retrieval in complex scenarios such as industry and e-commerce, and timely find the required fabric products, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of a fabric image retrieval method in a complex scene according to a preferred embodiment of the present invention.

[0030] Figure 2 Deep parallel representation model for fabric images.

[0031] Figure 3 Schematic diagram of the manual and deep feature fusion module for fabric images.

[0032] Figure 4 An example of fabric image retrieval in a complex scene. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0034] The embodiment of the present invention provides a fabric image retrieval method in a complex scene, comprising the following steps:

[0035] S1: Establish fabric image database;

[0036] S2: Build a manual feature extraction model to obtain manual feature descriptions of the query image and the database image;

[0037] S3: Build an image deep parallel representation model to process fabric images and obtain deep feature descriptions of images in the query image and database images;

[0038] S4: Design a method to fuse manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image;

[0039] S5: Measure the similarity between the fusion features of the query image and the image in the database obtained in S4, and output the retrieval result according to the size of the similarity;

[0040] S6: Retrieving fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

[0041] In order to explain the specific implementation of the present invention in detail, the present invention uses more than 30,000 fabric images collected from fabric production enterprises and e-commerce platforms as a database, selects images in different scenes to construct a fabric image retrieval dataset for verification, and the retrieval performance is better than the existing fabric image retrieval method. As a preferred embodiment, refer to Figure 1 , which is a flow chart of a fabric image retrieval method in a complex scene according to a preferred embodiment of the present invention.

[0042] The method of this embodiment includes the following steps:

[0043] Step S1: Establish a fabric image database.

[0044] In this step, image pairs of the same fabric in different scenes are selected from the fabric image database to construct a fabric image retrieval dataset for model training and verification, which mainly includes a training set, a verification set and a test set.

[0045] The fabrics selected in this embodiment are divided into five categories, namely stripes, plaids, prints, color spinning and dots.

[0046] Step S2: construct a manual feature extraction model to obtain manual feature descriptions of the query image and the images in the database;

[0047] In this step, the manual feature extraction model first preprocesses the fabric image, removes noise information in the image through filtering operations, and then extracts key points and direction information of the image as low-order features to describe the detail information of the fabric image.

[0048] In this embodiment, median filtering is used to filter the fabric image, the filter template size is 3×3, and the fast and stable feature point detection and extraction algorithm ORB algorithm is used to extract the key points and direction information of the fabric image.

[0049] Step S3: Building an image depth parallel representation model to process fabric images, and obtaining deep feature descriptions of images in the query image and database images;

[0050] In this step, the image depth parallel representation model has two parallel branches, such as Figure 2 As shown in the figure, they are used to extract scene features and category features of fabric images, describe the high-level semantic information of fabric images, ensure that the model can distinguish the same fabric in different scenes, and ensure that the retrieved images all belong to the same category.

[0051] In this step, the two parallel branches of the image depth parallel representation model use convolutional neural networks as the underlying framework to extract deep semantic features representing fabric images in an end-to-end manner.

[0052] The convolutional neural network is a self-built compact network, which includes 1 input layer, 4 convolution and pooling layers, 3 fully connected layers and 1 output layer.

[0053] In this step, the two branches of the image depth parallel representation model are optimized using the triplet loss function, and the final loss function is formed by weight distribution. The designed loss function L can be expressed as

[0054] L=β1L1+β2L2

[0055] Step S4: Design a method for fusing manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image;

[0056] In this step, the image manual and deep feature fusion method introduces a feature fusion module into the image deep parallel representation model described in S3, such as Figure 3 As shown, the manual and deep features of the image obtained by aggregating S2 and S3 are used to obtain the fusion features of the image in the query image and the database image, respectively.

[0057] In this step, the feature fusion module is based on the improved NetVLAD module, so that the manual features usually aggregated by VLAD can participate in the training in the image depth parallel representation model through the NetVLAD module, avoiding the artificial parameter design process in the manual features.

[0058] In this step, the improved NetVLAD module changes the original structure of processing feature maps in the convolutional neural network into a structure that can process feature maps and manual features at the same time, so as to achieve the purpose of fusing image manual and deep features.

[0059] Given N local features x of an image i and K clustering points c k As a VLAD parameter, the NetVLAD module takes the indicator function a in VLAD k (x i ) is changed to a derivable So that you can participate in model learning.

[0060]

[0061] Among them, w k =2αc k , b k =-α||c k || 2 , α is a constant.

[0062] Step S5: perform similarity measurement on the fusion features of the query image and the image in the database obtained in S4, and output the retrieval result according to the size of the similarity;

[0063] In this step, the similarity measurement method is a distance measurement method, which calculates the distance between the fusion features of the query image obtained in S4 and the image in the database, and sorts the images in the database according to the size of the distance.

[0064] This embodiment uses Euclidean distance to measure similarity.

[0065] S6: Retrieving fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

[0066] In this step, the fabric product process sheet includes product title, description and attribute information.

[0067] Those skilled in the art should understand that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fabric image retrieval method in complex scenes, characterized in that: The following steps are involved: S1: Establish fabric image database; The fabric images are collected under different shooting light sources, angles, and resolutions; S2: Build a manual feature extraction model to obtain manual feature descriptions of the query image and the database image; S3: Build an image deep parallel representation model to process fabric images and obtain deep feature descriptions of images in the query image and database images; The image deep parallel representation model has two parallel branches, which are used to extract scene features and category features of fabric images, describe high-level semantic information of fabric images, ensure that the model can distinguish the same fabric in different scenes, and ensure that the retrieved images belong to the same category; The two parallel branches of the image deep parallel representation model use convolutional neural networks as the underlying framework to extract deep semantic features representing fabric images in an end-to-end manner; The convolutional neural network is a self-built compact network, which includes 1 input layer, 4 convolution and pooling layers, 3 fully connected layers and 1 output layer; The two branches of the image depth parallel representation model are optimized using the triplet loss function, and the final loss function is formed by weight distribution. The designed loss function L is expressed as L=β1L1+β2L2 S4: Design a method to fuse manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image; The image manual and deep feature fusion method is as follows: Introducing a feature fusion module into the image deep parallel representation model described in S3, aggregating the image manual and deep features obtained in S2 and S3, and obtaining fusion features of the image in the query image and the database image respectively; The feature fusion module is based on the improved NetVLAD module, so that the manual features usually aggregated by VLAD can participate in the training in the image depth parallel representation model through the NetVLAD module; The improved NetVLAD module changes the original structure of processing feature maps in convolutional neural networks into a structure that can process feature maps and manual features at the same time, so as to achieve the purpose of fusing image manual and deep features; S5: Measure the similarity between the fusion features of the query image and the image in the database obtained in S4, and output the retrieval result according to the size of the similarity; S6: Retrieving fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

2. The fabric image retrieval method in a complex scene as claimed in claim 1, characterized in that: In the step S2, the manual feature extraction model first preprocesses the fabric image, removes noise information in the image through filtering operations, and then extracts key points and direction information of the image as low-order features to describe the detail information of the fabric image.

3. The fabric image retrieval method in a complex scene as described in claim 1 or 2, characterized in that: In the step S2, the manual features include SIFT, SURF and ORB features.

4. The fabric image retrieval method in a complex scene as described in claim 1 or 2, characterized in that: In the step S5, the similarity measure is a distance calculation method between feature vectors, and different distance formulas are selected for calculation.

5. The fabric image retrieval method in a complex scene as claimed in claim 3, characterized in that: In the step S5, the similarity measure is a distance calculation method between feature vectors, and different distance formulas are selected for calculation.

6. The fabric image retrieval method in a complex scene as described in claim 1, 2 or 5, characterized in that: In step S6, the fabric product details include product title, description and attribute information.

7. The fabric image retrieval method in a complex scene as claimed in claim 3, characterized in that: In step S6, the fabric product details include product title, description and attribute information.

8. The fabric image retrieval method in a complex scene as claimed in claim 4, characterized in that: In step S6, the fabric product details include product title, description and attribute information.

Images