A fabric image retrieval method in complex scenes

Abstract

The present invention belongs to the field of fabric retrieval methods, and relates to a fabric image retrieval method in complex scenarios. The steps are as follows: establish a fabric image database; construct a manual feature extraction model to obtain manual feature descriptions of the image to be queried and the image in the database; build an image depth parallel representation model to process the fabric image, and obtain the depth feature descriptions of the image to be queried and the image in the database; design an image manual and depth feature fusion method, and fuse the obtained image manual and depth features to comprehensively represent the fabric image; measure the similarity between the features of the obtained fusion features of the image to be queried and the image in the database, and output the retrieval results according to the size of the similarity; call out the fabric product details corresponding to the retrieval result image to guide fabric production or e-commerce shopping recommendations. The present invention has high retrieval accuracy and robustness, and has great application potential in complex scenarios such as industrial manufacturing and e-commerce platforms.

Description

Technical Field

[0001] The present invention belongs to the field of fabric retrieval methods and relates to a fabric image retrieval method in complex scenes. Background Art

[0002] Image retrieval technology allows users to upload images or take photos to find similar or identical products, improving image search efficiency and satisfaction. Image acquisition environments in industrial settings and e-commerce platforms are often difficult to standardize, with varying lighting, angles, and resolutions. Retrieval of such images falls under the category of complex image retrieval. Existing fabric image retrieval methods typically rely on content-based image retrieval, targeting images acquired under standard conditions. These methods struggle to address complex environments, limiting their application. Existing methods typically rely on image feature extraction and similarity metrics. The extracted features primarily consist of handcrafted features and deep features extracted using deep networks. Handcrafted features are computationally simple and highly interpretable, but their parameters are not universally applicable. Deep features are typically automatically extracted through end-to-end training. They are high-level semantic features that require no human intervention, but require significant training data and computing resources. Existing handcrafted and deep features target fabric images acquired under standard conditions and are therefore difficult to apply to fabric images acquired under complex environments. Fusion of handcrafted and deep features for fabric images can comprehensively characterize fabric image features, enabling efficient and accurate image retrieval in complex environments. Summary of the Invention

[0003] The purpose of this invention is to propose an efficient and accurate fabric image retrieval method in complex scenes, which can quickly search for fabric images collected under complex conditions in scenes such as industry and e-commerce.

[0004] The technical solutions of the present invention are as follows:

[0005] A fabric image retrieval method in complex scenes includes the following steps:

[0006] S1: Establish fabric image database;

[0007] The fabric images are collected under different shooting light sources, angles, resolutions and other conditions.

[0008] S2: Build a manual feature extraction model to obtain manual feature descriptions of the query image and the database images;

[0009] The manual feature extraction model first preprocesses the fabric image, removes noise information in the image through filtering operations, and then extracts key points and direction information of the image as low-order features to describe the detail information of the fabric image.

[0010] Furthermore, the manual features include SIFT, SURF and ORB features.

[0011] S3: Build a deep parallel image representation model to process fabric images and obtain deep feature descriptions of the query image and the database images;

[0012] The image depth parallel representation model has two parallel branches, such as Figure 2 As shown in the figure, they are used to extract scene features and category features of fabric images, describe the high-level semantic information of fabric images, ensure that the model can distinguish the same piece of fabric in different scenes, and ensure that the retrieved images all belong to the same category.

[0013] The two parallel branches of the image depth parallel representation model use convolutional neural networks as the underlying framework to extract deep semantic features representing fabric images in an end-to-end manner.

[0014] The convolutional neural network is a self-built compact network, which includes 1 input layer, 4 convolution and pooling layers, 3 fully connected layers and 1 output layer.

[0015] The two branches of the image depth parallel representation model are optimized using the triplet loss function, and the final loss function is formed in a weight distribution manner. The designed loss function L can be expressed as

[0016] L=β1L1+β2L2

[0017] Where: β1 and β2 are the weights of the two loss functions respectively;

[0018] S4: Design a method to fuse manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image;

[0019] The image manual and deep feature fusion method is as follows:

[0020] The feature fusion module is introduced into the image depth parallel representation model described in S3, such as Figure 3 As shown in FIG, the manual and deep features of the image obtained by S2 and S3 are aggregated to obtain the fusion features of the image in the query image and the database image respectively.

[0021] The feature fusion module is based on the improved NetVLAD module, so that the manual features usually aggregated by VLAD can participate in the training in the image depth parallel representation model through the NetVLAD module, avoiding the artificial parameter design process in the manual features.

[0022] The improved NetVLAD module changes the original structure of processing feature maps in convolutional neural networks into a structure that can process feature maps and manual features at the same time, so as to achieve the purpose of fusing image manual and deep features.

[0023] S5: Measure the similarity between the fusion features of the query image and the database image obtained in S4, and output the retrieval results according to the size of the similarity;

[0024] The similarity measure is a distance calculation method between feature vectors, and different distance formulas can be selected for calculation.

[0025] S6: Retrieve the fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

[0026] The fabric product details include product title, description and attribute information.

[0027] Beneficial effects of the present invention:

[0028] The present invention is based on the retrieval needs in complex scenarios such as industry and e-commerce, and proposes a fabric image retrieval method in complex scenarios. The detailed information of the fabric image is captured by manual features, and the high-level semantic information of the image is extracted by deep features, so that the manual features are involved in model training, and the two features are fused to jointly characterize the intrinsic characteristics of the fabric image, so that the two features form complementary advantages. A feature fusion module is introduced into the image deep parallel representation model, and the manual features are introduced into the deep representation model. The fabric image features are automatically extracted in an end-to-end training form, avoiding the parameter design and optimization process of manual features. This fusion method can improve the accuracy and robustness of fabric image retrieval, and can be applied to fabric retrieval in complex scenarios such as industry and e-commerce, to find the required fabric products in a timely manner, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of a fabric image retrieval method in complex scenarios according to a preferred embodiment of the present invention.

[0030] Figure 2 A deep parallel representation model for fabric images.

[0031] Figure 3 Schematic diagram of the manual and deep feature fusion module for fabric images.

[0032] Figure 4 An example of fabric image retrieval in complex scenes. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0034] An embodiment of the present invention provides a fabric image retrieval method in a complex scene, comprising the following steps:

[0035] S1: Establish fabric image database;

[0036] S2: Build a manual feature extraction model to obtain manual feature descriptions of the query image and the database images;

[0037] S3: Build a deep parallel image representation model to process fabric images and obtain deep feature descriptions of the query image and the database images;

[0038] S4: Design a method to fuse manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image;

[0039] S5: Measure the similarity between the fusion features of the query image and the database image obtained in S4, and output the retrieval results according to the size of the similarity;

[0040] S6: Retrieve the fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

[0041] To illustrate the specific implementation of the present invention, the present invention uses more than 30,000 fabric images collected from fabric production enterprises and e-commerce platforms as a database, selects images in different scenes to construct a fabric image retrieval dataset for verification, and the retrieval performance is better than the existing fabric image retrieval method. As a preferred embodiment, refer to Figure 1 , which is a flow chart of a fabric image retrieval method under complex scenarios according to a preferred embodiment of the present invention.

[0042] The method of this embodiment includes the following steps:

[0043] Step S1: Establish a fabric image database.

[0044] In this step, image pairs of the same fabric in different scenes are selected from the fabric image database to construct a fabric image retrieval dataset for model training and verification, which mainly includes a training set, a verification set and a test set.

[0045] The fabrics selected in this embodiment are divided into five categories, namely stripes, plaids, prints, color-spun and dots.

[0046] Step S2: construct a manual feature extraction model to obtain manual feature descriptions of the query image and the images in the database;

[0047] In this step, the manual feature extraction model first preprocesses the fabric image, removes noise information in the image through filtering operations, and then extracts the key points and direction information of the image as low-order features to describe the detailed information of the fabric image.

[0048] In this embodiment, median filtering is used to filter the fabric image. The filter template size is 3×3. The fast and stable feature point detection and extraction algorithm ORB algorithm is used to extract the key points and direction information of the fabric image.

[0049] Step S3: Build an image depth parallel representation model to process the fabric image and obtain the deep feature description of the image to be queried and the image in the database;

[0050] In this step, the image depth parallel representation model has two parallel branches, such as Figure 2 As shown in the figure, they are used to extract scene features and category features of fabric images, describe the high-level semantic information of fabric images, ensure that the model can distinguish the same piece of fabric in different scenes, and ensure that the retrieved images all belong to the same category.

[0051] In this step, the two parallel branches of the image depth parallel representation model use convolutional neural networks as the underlying framework to extract deep semantic features representing fabric images in an end-to-end manner.

[0052] The convolutional neural network is a self-built compact network, which includes 1 input layer, 4 convolution and pooling layers, 3 fully connected layers and 1 output layer.

[0053] In this step, both branches of the image depth parallel representation model are optimized using the triplet loss function, and the final loss function is formed in a weighted distribution manner. The designed loss function L can be expressed as

[0054] L=β1L1+β2L2

[0055] Step S4: Design a method to fuse manual and deep features of the image, fuse the manual and deep features of the image obtained in S2 and S3, and comprehensively represent the fabric image;

[0056] In this step, the image manual and deep feature fusion method introduces a feature fusion module into the image depth parallel representation model described in S3, such as Figure 3 As shown in FIG, the manual and deep features of the image obtained by S2 and S3 are aggregated to obtain the fusion features of the image in the query image and the database image respectively.

[0057] In this step, the feature fusion module is based on the improved NetVLAD module, so that the manual features usually aggregated by VLAD can participate in the training in the image depth parallel representation model through the NetVLAD module, avoiding the artificial parameter design process in the manual features.

[0058] In this step, the improved NetVLAD module changes the original structure of processing feature maps in convolutional neural networks into a structure that can process feature maps and manual features at the same time, so as to achieve the purpose of fusing image manual and deep features.

[0059] Given N local features x of an image i and K clustering points c k As a VLAD parameter, the NetVLAD module takes the indicator function a in VLAD k (x i ) is changed to a guide So that you can participate in model learning.

[0060]

[0061] Among them, w k =2αc k , b k =-α||c k || 2 , α is a constant.

[0062] Step S5: Measure the similarity between the fusion features of the query image and the database image obtained in S4, and output the retrieval results according to the similarity.

[0063] In this step, the similarity measurement method is a distance measurement method, which calculates the distance between the query image obtained in S4 and the fusion features of the images in the database, and sorts the images in the database according to the size of the distance.

[0064] This embodiment uses Euclidean distance to measure similarity.

[0065] S6: Retrieve the fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

[0066] In this step, the fabric product process sheet includes product title, description and attribute information.

[0067] Those skilled in the art should understand that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A fabric image retrieval method in complex scenes, characterized by: The following steps are involved: S1: Establish fabric image database; The fabric images are collected under different shooting light sources, angles, and resolutions; S2: Build a manual feature extraction model to obtain manual feature descriptions of the query image and the database images; S3: Build a deep parallel image representation model to process fabric images and obtain deep feature descriptions of the query image and the database images; The image deep parallel representation model has two parallel branches, one for extracting scene features and the other for extracting category features of fabric images, describing the high-level semantic information of fabric images, ensuring that the model can distinguish the same fabric in different scenes and that all retrieved images belong to the same category. The two parallel branches of the image depth parallel representation model use convolutional neural networks as the underlying framework to extract deep semantic features representing fabric images in an end-to-end manner; The convolutional neural network is a self-built compact network consisting of 1 input layer, 4 convolution and pooling layers, 3 fully connected layers and 1 output layer; The two branches of the image depth parallel representation model are optimized using the triple loss function, and the final loss function is formed in a weight distribution manner. The designed loss function L is expressed as L=β1L1+β2L2 S4: Design a method to fuse manual and deep features of images, fuse the manual and deep features of images obtained in S2 and S3, and comprehensively represent the fabric image; The image manual and deep feature fusion method is as follows: Introducing a feature fusion module into the image depth parallel representation model described in S3, aggregating the image manual and deep features obtained in S2 and S3, and obtaining fused features of the query image and the image in the database respectively; The feature fusion module is based on the improved NetVLAD module, so that the manual features usually aggregated by VLAD can participate in the training of the image depth parallel representation model through the NetVLAD module; The improved NetVLAD module changes the original structure of processing feature maps in convolutional neural networks to a structure that can process feature maps and manual features at the same time, so as to achieve the purpose of fusing image manual and deep features; S5: Measure the similarity between the fusion features of the query image and the database image obtained in S4, and output the retrieval results according to the size of the similarity; S6: Retrieve the fabric product details corresponding to the search result image to guide fabric production or e-commerce shopping recommendations.

2. The fabric image retrieval method in a complex scene according to claim 1, characterized in that: In step S2, the manual feature extraction model first preprocesses the fabric image, removes noise information in the image through filtering operations, and then extracts key points and direction information of the image as low-order features to describe the detail information of the fabric image.

3. The fabric image retrieval method in a complex scene according to claim 1 or 2, characterized in that: In the step S2, the manual features include SIFT, SURF and ORB features.

4. The fabric image retrieval method in a complex scene according to claim 1 or 2, characterized in that: In the step S5, the similarity measure is a distance calculation method between feature vectors, and different distance formulas are selected for calculation.

5. The fabric image retrieval method in complex scenes according to claim 3, characterized in that: In the step S5, the similarity measure is a distance calculation method between feature vectors, and different distance formulas are selected for calculation.

6. The fabric image retrieval method in a complex scene according to claim 1, 2 or 5, characterized in that: In step S6, the fabric product details include product title, description and attribute information.

7. The fabric image retrieval method in complex scenes according to claim 3, characterized in that: In step S6, the fabric product details include product title, description and attribute information.

8. The fabric image retrieval method in complex scenes according to claim 4, characterized in that: In step S6, the fabric product details include product title, description and attribute information.

Images