Lattice fabric image retrieval method based on depth representation model

Abstract

The invention relates to a grid fabric image retrieval method based on a depth representation model, and belongs to the field of textile fabric retrieval and computer vision. The multi-scale feature fusion method has the advantages that the model extracts low-order visual attributes and high-order semantic features of the fabric at the same time, and the problem of feature redundancy or information splitting in a traditional method is solved; (2) a quaternary loss joint optimization mechanism: introducing global comparison loss into a dynamic margin regulator driven by local similarity, adaptively contracting / expanding a decision boundary according to the similarity of image blocks, and improving the fine-grained distinguishing capability of similar textures; the local alignment loss strengthens the consistency of the microstructure, the periodic constraint loss explicitly models the spatial rule of dyed yarn arrangement, and the problem that CNN is insensitive to periodic characteristics is solved. According to the method, an efficient and reliable image retrieval tool is provided for the textile industry, the application prospects in e-commerce recommendation, production quality inspection and supply chain management scenes are wide, and remarkable economic and social benefits are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of lattice fabric image retrieval method based on deep representation model, belong to textile fabric retrieval and computer vision field. BACKGROUND

[0002] Lattice fabric is widely used in clothing, home, industry and other fields due to its rich patterns, color combinations and periodic structures. Lattice fabric retrieval technology can quickly obtain the product images required by users, helping e-commerce platforms to make accurate recommendations, textile enterprises to manage products, and consumers to improve shopping experience. The low-level visual attribute information such as texture and color in different types of lattice fabric and the high-level semantic information such as style and pattern semantics have different performances. For example, college lattice not only contains low-level information such as fabric organization and colored yarn lines, but also contains high-level semantic features such as lattice type and style. How to design lattice fabric image representation and retrieval algorithm according to the characteristics of lattice fabric images in the subfield is an urgent problem for lattice fabric production enterprises.

[0003] Existing fabric image retrieval methods mainly include low-level feature-based retrieval methods and high-level feature-based representation methods. Low-level feature-based retrieval methods usually require manual design of handcrafted features to extract texture, color or shape features of fabric images. Such methods are simple, easy to implement, and have strong interpretability, but have poor parameter universality. High-level feature-based representation methods use deep network models to automatically extract features of fabric images in an end-to-end manner. Such methods avoid the need for manual selection of feature parameters and can represent high-level semantic information in fabric images. The differences between lattice fabric images are reflected in key information such as organization, lattice type, and colored yarn arrangement. The features extracted by traditional high-level representation models are usually too redundant and difficult to focus on key information in lattice fabric images. The present application focuses on key features in lattice fabric images from a global and local perspective through the construction of a deep representation model. In addition, to learn features that sufficiently represent lattice fabric images, the present application designs a multi-loss joint training strategy, including global contrast loss, local alignment loss, periodic constraint loss, and adaptive decoupling loss, to constrain the model from different angles to achieve optimal model training results. SUMMARY

[0004] The present application addresses the shortcomings of the prior art by providing a lattice fabric image retrieval method based on a deep representation model, aiming to improve the efficiency and accuracy of lattice fabric image retrieval, and to help e-commerce platforms make accurate recommendations, textile enterprises manage products, and consumers improve their shopping experience.

[0005] The technical solution of the present application is as follows:

[0006] A lattice fabric image retrieval method based on a deep representation model, comprising the following steps:

[0007] Step S01: Establishing a plaid fabric image retrieval dataset;

[0008] The established dataset includes a training set, a validation set and a test set;

[0009] The plaid fabric includes four types: color block plaid, college plaid, window plaid and Welsh plaid.

[0010] Step S02: Constructing a plaid fabric image deep representation model;

[0011] The constructed plaid fabric image deep representation model adopts a double-flow architecture, including global and local branches, respectively used for extracting global and local features of the plaid fabric image.

[0012] The global branch adopts ResNet50 as the backbone network, and designs a periodic perception convolutional layer (PAC) to capture the unique periodic texture in the plaid fabric image, such as color yarn arrangement and fabric organization.

[0013] Suppose x represents the input feature map, K represents the number of periodic kernels, and ω k represents the weight parameters of the kth kernel, then PAC is defined as follows

[0014]

[0015] Where Γ k represents a periodic sensitive convolution kernel function, and ΔT k represents an adaptive periodic parameter.

[0016] The local branch adopts an improved Swin Transformer to represent the local information of the plaid fabric image. The improved Swin Transformer adopts direction-aware window partitioning to avoid uniform window damage to the continuous texture of the plaid fabric. Suppose (x, y) represents the pixel coordinates on the feature map, s x (θ) and s y (θ) represent the partitioning step size of the x-axis and y-axis in the θ direction, respectively, and the index coordinate set W θ of the window partitioned in the θ direction can be represented as

[0017]

[0018] Since the global branch focuses on the overall style and the local branch focuses on the texture details, there is a misalignment between the two, and the global features lose the local arrangement rules, and the local features lack global constraints. In order to solve the representation conflict between global features and local features and realize the collaborative optimization of multi-granularity information, the present application designs a cross-branch feature calibration module to align the global and local features.

[0019] Suppose denotes the calibrated global feature vector, F g denotes the original global feature vector, F l denotes the local feature matrix, then

[0020]

[0021] where max(F l ) denotes the max-pooling operation, MLP(x) denotes the fully connected network, and σ denotes the Sigmoid activation function.

[0022] Step S03: training the grid fabric image deep representation model;

[0023] The grid fabric image deep representation model is trained in a multi-loss joint manner, including a global contrast loss, a local alignment loss, a periodic constraint loss, and an adaptive decoupling loss.

[0024] The global contrast loss is used to ensure that grid fabric images of different categories are significantly distinguished in the overall feature space, while the overall features of images of the same texture category are pulled closer. By introducing a dynamic margin mechanism, the robustness of the model to local structure changes of fabrics is significantly improved, and the discrimination ability for similar textures is strengthened. It is a core discriminant component in the multi-loss joint training framework. Assuming that P O and N E denote the positive sample set and the negative sample set, respectively, f i g and denote the global feature vectors of samples i and j, respectively, then the global contrast loss L g can be expressed as

[0025]

[0026] wherein and denote the dynamic positive sample margin and the negative sample margin, respectively, and ‖·‖2 denotes the L2 norm.

[0027]

[0028] wherein μ denotes the basic margin, σ denotes the Sigmoid function, sim(F i ,F j ) denotes the similarity of local features F i and F j , sim(F i ,F k ) denotes the similarity of local features F i and F k , and η denotes the margin scaling factor.

[0029] The local alignment loss is used to ensure that small regions with highly similar local structures in a lattice fabric image are mapped to similar positions in the feature space, enhancing the model's ability to perceive local texture details. Assuming that T represents the optimal transport matrix, M and N represent the number of regions divided in the horizontal and vertical directions of the image, respectively, the local alignment loss L l may be defined as

[0030]

[0031] where C represents the cost function, represents the mth local feature of sample i, F l jn represents the nth local feature of sample j, H represents the matrix entropy, and λ represents the entropy regularization coefficient.

[0032] The periodic constraint loss L p is used to explicitly model and utilize the periodic characteristics inherent in the lattice texture, so that the feature map learned by the model exhibits the same periodicity as the original texture in the spatial dimension, or the feature itself can encode periodic information. Assuming that F l represents the local feature sequence, L represents the number of frequency components, and the periodic constraint loss L p may be defined as

[0033]

[0034] where P represents the self-learning period parameter, FFT represents the fast Fourier transform, and ‖·‖1 represents the L1 norm.

[0035] There are global and local feature aggregations in the lattice fabric image, and the adaptive decoupling loss is used to separate them, as much as possible to reflect the essence of the features and avoid the insufficient representation accuracy caused by the mixing of global and local features. Assuming that F g represents the global feature, represents the ith local feature, and the adaptive decoupling loss L d may be represented as

[0036]

[0037] where N P represents the number of local regions, and τ represents the decoupling threshold.

[0038] Step S04: calling the deep representation model to extract the features of the lattice fabric image;

[0039] After the deep representation model is trained, the model is called to extract the features of the lattice fabric image, including the features of the input lattice fabric image to be searched and the features of the lattice fabric images in the database.

[0040] Step S05: calculating the similarity between the image feature to be searched and the database image feature;

[0041] The feature similarity measurement method is a distance measurement method, and the Euclidean distance between the image feature to be searched and the database image feature is calculated as the similarity.

[0042] Step S06: outputting the corresponding image in the database as the search result through similarity sorting.

[0043] The similarity sorting method is to sort according to the distance from small to large, and output the database image corresponding to the feature as the search result.

[0044] The effective effects of the present application are:

[0045] The present application significantly improves the retrieval accuracy and efficiency of the lattice fabric image by constructing a global-local collaborative deep representation model combined with an innovative multi-loss joint training strategy. Specifically embodied as:

[0046] (1) Multi-scale feature fusion advantage: the model extracts the low-order visual attributes (yarn density, texture) and high-order semantic features (lattice style, pattern semantics) of the fabric at the same time, solving the problem of feature redundancy or information fragmentation in traditional methods;

[0047] (2) Four-element loss joint optimization mechanism: the global contrast loss introduces a local similarity driven dynamic margin regulator, which adaptively shrinks / expands the decision boundary according to the image block similarity, improves the fine-grained discrimination ability of similar textures, and strengthens the microscopic structure consistency by local alignment loss, and the periodic constraint loss explicitly models the spatial law of color yarn arrangement, solving the problem that CNN is not sensitive to periodic features; the adaptive decoupling loss strips off interference factors such as light and shooting angle, and still maintains high representation ability in noisy scenes;

[0048] (3) Technical universality extension: the model can be migrated to jacquard fabric, lace and other textile products with periodic structure for retrieval.

[0049] The present application provides an efficient and reliable image retrieval tool for the textile industry, which has a wide application prospect in e-commerce recommendation, production quality inspection and supply chain management scenarios, and has significant economic and social benefits. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The flowchart for the present application.

[0051] Figure 2 The lattice fabric image deep representation model constructed by the present application.

[0052] Figure 3 The search result example of the present application. DETAILED DESCRIPTION

[0053] The present application is described in detail below with reference to the accompanying drawings and embodiments. The present application proposes a lattice fabric image retrieval method based on a deep representation model, the overall flowchart of which is shown in Figure 1 The execution steps are as follows:

[0054] Step S01: Establish a lattice fabric image retrieval dataset;

[0055] The lattice fabric image database established in this embodiment has a total of 44,000 images. The constructed dataset includes a training set, a validation set and a test set, wherein the training set has 16,000 images, the validation set has 2,000 images, and the test set has 400 images, ensuring uniform distribution of images of different categories.

[0056] The lattice fabric includes four categories: color block, college, window and Welsh.

[0057] Step S02: Construct a deep representation model of lattice fabric images;

[0058] The constructed deep representation model of lattice fabric images adopts a dual-stream architecture, including global and local branches, which are respectively used to extract global and local features of lattice fabric images, as shown in Figure 2 .

[0059] The global branch adopts ResNet50 as the backbone network, and designs a periodic perception convolutional layer (PAC) to capture the unique periodic texture in the lattice fabric image, such as yarn arrangement and fabric organization.

[0060] Assuming x represents the input feature map, K represents the number of periodic kernels, ω k represents the weight parameters of the kth kernel, then PAC is defined as follows

[0061]

[0062] where Γ k represents the periodic sensitive convolution kernel function, and ΔT k represents the adaptive periodicity parameter.

[0063] The local branch uses an improved Swin Transformer to represent the local information of the lattice fabric image. The improved Swin Transformer adopts direction-aware window partitioning to avoid uniform window damage to the continuous texture of the lattice fabric. Assuming (x, y) represents the pixel coordinates on the feature map, s x (θ) and s y (θ) represent the partitioning step size of the x-axis and y-axis in the θ direction, respectively, and the index coordinate set W θ of the window partitioned in the θ direction can be represented as

[0064]

[0065] Since the global branch focuses on the overall style and the local branch focuses on the texture details, the two exist misalignment cases, and the global feature loses the local arrangement rule and the local feature lacks global constraints. In order to solve the representation conflict of global features and local features and realize the collaborative optimization of multi-granularity information, the application designs a cross-branch feature calibration module to align the global and local features.

[0066] Assume represents the calibrated global feature vector, F g represents the original global feature vector, F l represents the local feature matrix, then

[0067]

[0068] wherein, max (F l ) represents the max-pooling operation, MLP (x) represents the fully connected network, and sigma represents the Sigmoid activation function.

[0069] Step S03: training the lattice fabric image depth representation model;

[0070] The lattice fabric image depth representation model is trained in a multi-loss joint manner, including global contrast loss, local alignment loss, periodic constraint loss and adaptive decoupling loss.

[0071] The global contrast loss is used to ensure that different categories of lattice fabric images are significantly distinguished in the overall feature space, while the overall features of images of the same texture category are pulled closer. By introducing a dynamic margin mechanism, the robustness of the model to local structure changes of the fabric is significantly improved, and the discrimination ability for similar textures is strengthened. It is the core discriminant component in the multi-loss joint training framework. Assuming P O and N E represent the positive sample set and the negative sample set respectively, f i g and represent the global feature vectors of samples i and j respectively, then the global contrast loss L g can be represented as

[0072]

[0073] wherein, and represent the dynamic positive sample margin and the negative sample margin respectively, and ‖·‖2 represents the L2 norm.

[0074]

[0075] where μ denotes the base margin, σ denotes the Sigmoid function, sim(F i ,F j ) denotes the similarity of local features F i and F j , sim(F i ,F k ) denotes the similarity of local features F i and F k , and η denotes the margin scaling factor.

[0076] The local alignment loss is used to ensure that small regions with highly similar local structures in a lattice fabric image are mapped to close positions in the feature space, enhancing the model's ability to perceive local texture details. Assuming that T denotes the optimal transport matrix, M and N denote the number of regions divided in the horizontal and vertical directions of the image, respectively, the local alignment loss L l can be defined as

[0077]

[0078] where C denotes the cost function, F l jn denotes the m-th local feature of sample i, F p denotes the n-th local feature of sample j, H denotes the matrix entropy, and λ denotes the entropy regularization coefficient.

[0079] The periodicity constraint loss L l is used to explicitly model and utilize the inherent periodic characteristics of lattice textures, so that the feature maps learned by the model exhibit the same periodicity as the original texture in the spatial dimension, or the features themselves can encode periodic information. Assuming that F p denotes the local feature sequence, and L denotes the number of frequency components, the periodicity constraint loss L l can be defined as

[0080]

[0081] where P denotes the self-learning periodicity parameter, FFT denotes the fast Fourier transform, and ||·||1 denotes the L1 norm.

[0082] There are global and local features aggregated in the lattice fabric image, and the adaptive decoupling loss is used to separate them, as much as possible to reflect the essence of the features and avoid the insufficient representation accuracy caused by the mixing of global and local features. Assuming that F g denotes the global feature, denotes the i-th local feature, the adaptive decoupling loss L d can be represented as

[0083]

[0084] wherein N P denotes the number of local regions, and τ denotes the decoupling threshold.

[0085] Step S04: calling the deep representation model to extract the features of the plaid image;

[0086] After the training of the deep representation model is completed, the model is called to extract the features of the plaid image, including the features of the input to-be-retrieved plaid image and the features of the plaid image in the database.

[0087] Step S05: calculating the similarity between the features of the to-be-retrieved image and the features of the database image;

[0088] The feature similarity measurement method is a distance measurement method, and the Euclidean distance between the features of the to-be-retrieved image and the features of the database image is calculated as the similarity.

[0089] Step S06: outputting the corresponding image in the database as the retrieval result through similarity sorting.

[0090] The similarity sorting method is to sort according to the distance from small to large, and output the database image corresponding to the feature as the retrieval result, and the retrieval result example is shown in Figure 3 .

[0091] The proposed plaid image retrieval method can be widely applied to e-commerce recommendation, production quality inspection, supply chain management and other scenes, reduce the labor intensity of enterprise workers, and improve the user experience.

Claims

1. A method for retrieving a lattice fabric image based on a deep representation model, characterized in that, Comprise the following steps: Step S01: lattice fabric image retrieval dataset is established; Step S02: lattice fabric image deep feature model is constructed; The constructed lattice fabric image deep feature model adopts a double-flow architecture, including global and local branches, which are respectively used for extracting global and local features of the lattice fabric image; The global branch adopts ResNet50 as the backbone network, and designs a periodic perception convolutional layer to capture the unique periodic texture in the lattice fabric image; Let x denote the input feature map, K denote the number of periodic kernels, ω k denote the weight parameters of the kth kernel, then PAC is defined as follows: where Γ k represents a periodic sensitive convolution kernel function, ΔT k represents an adaptive periodic parameter; The local branch adopts an improved Swin Transformer to represent the local information of the lattice fabric image; the improved Swin Transformer adopts direction-aware window division, which avoids uniform window from destroying the continuous texture of the lattice fabric; let (x, y) represent the pixel coordinates on the feature map, s x (θ) and s y (θ) represent the division step of the x-axis and the y-axis in the direction of θ respectively, and the index coordinate set W θ is represented as In order to solve the representation conflict between global features and local features, and realize multi-granularity information collaborative optimization, the application designs a cross-branch feature calibration module to align the global and local features; Let denote the calibrated global feature vector, F g denote the original global feature vector, F l denote the local feature matrix, then where max(F l ) denotes the max-pooling operation, MLP(x) denotes the fully connected network, and σ denotes the Sigmoid activation function. Step S03: lattice fabric image deep feature model is trained; The lattice fabric image deep feature model is trained in a multi-loss joint manner, including global contrast loss, local alignment loss, periodic constraint loss and adaptive decoupling loss; Step S04: call the deep feature model to extract the lattice fabric image features; After the training of the deep feature model is completed, the model is called to extract the features of the lattice fabric image, including the features of the input lattice fabric image to be retrieved and the features of the lattice fabric image in the database; Step S05: calculate the similarity between the features of the image to be retrieved and the features of the database image; The feature similarity measurement method is distance measurement, and the Euclidean distance between the features of the image to be retrieved and the features of the database image is calculated as the similarity; Step S06: output the corresponding image in the database as the retrieval result through similarity sorting.

2. The lattice fabric image retrieval method based on a deep representation model according to claim 1, wherein, In step S01, the established dataset includes training set, validation set and test set;The lattice fabric includes four types: color block lattice, college lattice, window lattice and Wales lattice.

3. The lattice fabric image retrieval method based on a deep representation model according to claim 1, wherein, In step S03, the global contrast loss L g is represented as wherein, and denote the dynamic positive and negative sample margins, respectively, and || · ||2denotes the L2norm. Where μ represents the basic margin, σ represents the Sigmoid function, and sim(F i ,F j ) represents local feature F i and F j Similarity, sim(F i ,F k ) represents local feature F i and F k The similarity, where η represents the margin scaling factor.

4. The lattice fabric image retrieval method based on a deep representation model according to claim 1, wherein, The local alignment loss L l is defined as where C denotes a cost function, denotes the mth local feature of sample i, denotes the nth local feature of sample j, H denotes a matrix entropy, and λ denotes an entropy regularization coefficient.

5. The lattice fabric image retrieval method based on a deep representation model according to claim 1, wherein, In step S03, the periodic constraint loss L p is defined as Wherein, P represents a self-learning cycle parameter, FFT represents fast Fourier transform, and ‖·‖1 represents L1 norm.

6. The lattice fabric image retrieval method based on a deep representation model according to claim 1, wherein, In step S03, the adaptive decoupling loss L d is represented as where N P denotes the number of local regions, and τ denotes the decoupling threshold.

7. The lattice fabric image retrieval method based on a deep representation model according to claim 1, wherein, In step S06, the similarity sorting method is to sort according to the distance from small to large, and output the database image corresponding to the feature as the retrieval result.

Images