A garment style match retrieval method and system

Abstract

The present application relates to the technical field of computer vision and garment digitization intelligent manufacturing, in particular to a garment style matching retrieval method and system, comprising: collecting three types of features of garment visual profile, local structure and pattern process parameters, and establishing two-level index by adopting hierarchical K-means; building an improved ResNet network with split-channel attention to distinguish between shaping structure and decorative features; setting two-level adaptive threshold, calculating comprehensive matching degree through a weighting formula, and realizing model closed-loop fine-tuning by using triplets and MSE loss; supporting AES hierarchical encryption and cloud-edge retrieval system. The same model retrieval accuracy is improved, and the retrieval delay of million samples is greatly reduced.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital intelligent manufacturing technology for clothing, specifically to a method and system for matching and retrieving clothing styles. Background Technology

[0002] Currently, the apparel e-commerce, flexible intelligent manufacturing, and apparel design intellectual property protection industries are developing rapidly, making image-based intelligent apparel retrieval a necessity for these industries. Online retail platforms, apparel pattern-making factories, and intellectual property rights protection agencies all need to quickly match similar patterns based on real-life apparel images. However, most existing deep learning-based apparel image retrieval solutions rely solely on convolutional neural networks to extract shallow visual features such as color, texture, and outline to achieve similarity matching. In industrial applications, these solutions suffer from significant drawbacks, including low accuracy, high computational costs, and an inability to meet the specialized needs of apparel pattern making.

[0003] CN110825899B discloses a method for retrieving clothing images by fusing color features and residual network deep features. The method involves inputting a training dataset into a ResNet50-based network model; fusing deep features and color features as global feature representations of the image; using the K-Means algorithm to cluster vectors in the feature library; inputting the clothing image to be retrieved into the same neural network as the dataset to obtain the global feature vector of the queried clothing image; calculating the distance between the cluster center vector and the vector of the retrieval image; and using distance comparison to measure similarity and obtain the retrieval result. This existing technology can combine multiple feature information of images, resulting in high retrieval efficiency and low time overhead; the extracted deep features have a certain degree of effectiveness and hierarchy; and it possesses strong robustness and practicality.

[0004] However, the feature dimensions are limited to visual information in image pixels, deviating from professional standards for judging garment patterns. Relying solely on appearance color matching and fabric texture for matching easily leads to mismatches where "appearance colors are similar, but internal patterns are completely different," resulting in a high rate of pattern confusion and false detections, failing to meet the professional needs of factory pattern adjustments and patent infringement comparisons. The similarity calculation uses equal weight for defining areas such as collars, shoulders, waists, and plackets, as well as decorative areas such as prints and buttons. When real-life images have variations in lighting, fabric wrinkles, or obstructions, decorative textures dominate the matching results, significantly reducing search accuracy.

[0005] A single-layer global clustering retrieval architecture is used, and clustering is only used for coarse image segmentation. During retrieval, it is still necessary to traverse and calculate distances within the entire feature library. When the database stores hundreds of thousands or millions of clothing samples, the latency of a single retrieval is high and the server computing power consumption is huge, making it difficult to support high-concurrency scenarios for real-time product searches in e-commerce.

[0006] Based on this, the present invention proposes a clothing style matching retrieval method and system. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, a method and system for matching and retrieving clothing styles is provided, which achieves high-precision, low-computing-power, universal applicability across all categories, and commercially viable intelligent matching and retrieval of clothing styles.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is a clothing style matching and retrieval method, comprising the following steps:

[0009] S1. Collect and label clothing samples, extract three types of features: visual silhouette features, local structural features, and pattern process parameters of clothing, store the associated sample labels, and establish first- and second-level retrieval indexes by hierarchical K-means clustering according to clothing category;

[0010] S2. Collect original images of the garment from multiple perspectives, and preprocess them by median filtering, background removal, perspective correction, and size normalization.

[0011] S3. An improved ResNet50 network based on an embedded clothing channel attention module outputs three fixed-dimensional fused feature vectors in parallel: visual silhouette, local structure, and pattern technology.

[0012] S4. Two-level hierarchical matching retrieval:

[0013] S41. Coarse-grained retrieval: Match the primary category index of the database, calculate the visual profile similarity, and filter the candidate sample set that is greater than the first adaptive threshold;

[0014] S42. Fine-grained weighted matching: Differentiated feature weights are assigned according to the priority rules of clothing shape structure, the comprehensive matching degree is calculated by weighting, and samples below the second adaptive threshold are removed.

[0015] S5. Sort the results in descending order of overall matching degree and output the results visually with layout parameters.

[0016] The above-mentioned clothing style matching and retrieval method uses visual silhouette features: clothing outer contour, six standard silhouette labels (H / X / A / T / O / S); local structural features: seven fixed parts silhouettes (collar, sleeve, waist, shoulder, placket, pocket, hem); and pattern process parameters: bust ease, shoulder width, garment length, seam lines, pleats, splicing, and quantitative pattern making parameters for grading.

[0017] The above-mentioned clothing style matching retrieval method has the following differential weight constraints: the total weight of clothing shape structure features is ≥60%, and the weight of print and button decoration features is <10%.

[0018] The formula for calculating the overall matching degree in the above-mentioned clothing style matching retrieval method is as follows:

[0019] ;

[0020] in, For overall matching accuracy, For visual profile similarity, For local structural similarity, For the matching degree of pattern and process parameters; , , These are the corresponding feature weight coefficients, and .

[0021] The above-mentioned clothing style matching retrieval method, the visual corridor similarity Based on normalized Euclidean distance conversion, the value range is [0, 1]; the local structural similarity Seven key points were extracted from different parts, and the cosine similarity of each point was weighted and averaged to obtain the pattern pattern process parameters matching degree. It is calculated by fusion of normalized difference of continuous parameters and discrete label matching.

[0022] The above-mentioned clothing style matching retrieval method also includes a closed-loop iterative optimization step: constructing a triplet dataset by manually selecting the same style as positive samples and the different styles as negative samples, using triplet and MSE joint loss to supervise and fine-tune the network, and adding styles to incrementally update the clustering index.

[0023] The above clothing style matching retrieval method improves ResNet50 by removing the original fully connected classification layer and embedding the attention module into the back end of the fourth residual block; it also fine-tunes the freezing of the first three residual blocks, uses a training batch size of 32, and an initial learning rate of 10. -4 Iterate 10 to 30 times.

[0024] The above clothing style matching retrieval method has fixed weights based on different scenarios: women's clothing scenario. =0.3, =0.55, =0.15, suit pattern making scene =0.25, =0.45, =0.3; the coarse / fine matching threshold switches adaptively according to the business scenario.

[0025] A clothing style matching and retrieval system for executing the above-mentioned clothing style matching and retrieval method includes: an image acquisition module, an image preprocessing module, a multi-dimensional feature extraction module, a database management module, a hierarchical matching and retrieval module, a weight adaptive configuration module, a result output interaction module, and a model iteration optimization module.

[0026] The aforementioned clothing style matching and retrieval system also includes an AES-128 encrypted storage module, which encrypts the pattern quantification parameters in segments and implements hierarchical control of hardware keys with three levels of permissions.

[0027] The system adopts a collaborative architecture of edge-end acquisition and preprocessing, and cloud-based high-performance computing, enabling lightweight image transmission and local storage of high-definition original images.

[0028] The beneficial effects of the clothing style matching retrieval method and system of the present invention are that it integrates three types of features: visual silhouette of clothing, local fixed structure, and professional pattern quantitative process parameters. Through a clothing-specific channel attention mechanism, it actively suppresses interference from irrelevant decorative features such as fabric printing, color, shooting light and shadow, and model posture, and distinguishes between the fixed structure of the clothing body and decorative accessories. It solves the defect of the existing technology that relies solely on image pixels and processes structure and decoration with equal weight, resulting in high false detection.

[0029] The system employs a hierarchical K-means two-level clustering partitioning retrieval architecture. First, it performs coarse-grained filtering through a first-level category index, and then performs fine-grained weighted matching only on highly similar candidate samples, without needing to traverse the feature vectors of the entire database. The single-item retrieval response time in the millions of clothing pattern sample database is only 160~200ms.

[0030] A closed-loop system of manual feedback supervision is established. Manually selected identical samples are used as positive samples and samples with different patterns are used as negative samples to construct a triplet training set. The network is fine-tuned using a joint loss function of triplet and MSE. During the fine-tuning stage, the backbone residual blocks are frozen and only the attention layer is updated. When new clothing samples are added, three types of feature vectors are automatically extracted and the clustering index is updated. The recognition accuracy is continuously optimized as the number of business samples increases. Attached Figure Description

[0031] Figure 1 : Overall flowchart of the clothing style matching retrieval method;

[0032] Figure 2 : Overall architecture diagram of the clothing style matching and retrieval system. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, a clothing style matching retrieval method includes the following steps:

[0035] S1. Collect clothing samples from multiple channels, including online e-commerce and offline pattern making inventory, and label them through machine recognition and manual verification; extract three types of features: visual silhouette features, local structural features, and pattern process parameters, and bind and store these three types of features with unique sample identifiers; use hierarchical K-means clustering to partition according to clothing categories and establish a first-level category retrieval index and a second-level silhouette tag retrieval index.

[0036] Hierarchical K-means clustering implementation logic: Clustering is divided into two dimensions. The first dimension clusters based on clothing categories (dresses / suits / trousers / knitwear), and the second dimension clusters based on six basic silhouette labels. Euclidean distance is used as the clustering metric, where the change in cluster centers between two consecutive intervals is less than 10. -3 The process converges iteratively; the first-level index stores the mapping between the category number and the corresponding sample database address, and the second-level index stores the pointers to the feature vectors of all samples under the profile label; when the cumulative number of new samples reaches 500 sets, the cluster center and the second-level index are automatically updated; during retrieval, the first-level index is matched to lock the sub-database, and then the second-level index is called to quickly traverse, without having to traverse the entire sample set.

[0037] S2. Image Acquisition and Standardized Preprocessing: Acquire original images of the garment to be retrieved from multiple perspectives, including the front, side, and back. Then, sequentially perform median filtering for noise reduction, GrabCut adaptive background removal, perspective distortion correction, and bilinear interpolation for size normalization. Output a standardized RGB image with a uniform 256×256 pixel resolution and a value range of 0~255.

[0038] Preprocessing algorithm implementation logic: 1. Median filtering: A fixed 3×3 sliding window is used to take the median value of image pixels to remove salt and pepper noise and fully preserve the edge contour of the clothing; 2. GrabCut adaptive background removal: The foreground area of ​​the clothing is manually selected, and the foreground / background pixels are divided into foreground and background pixels by a fixed 15 iterations to remove irrelevant backgrounds such as walls, mannequins, and clutter; 3. Perspective distortion correction: Four sets of contour key points at the four corners of the clothing are identified, and the tilted image is mapped to a front view through a 3×3 homography matrix transformation; 4. Bilinear interpolation is used to uniformly scale the image to a 256×256 three-channel RGB standard image.

[0039] S3. Multi-dimensional fusion feature extraction: Based on a convolutional neural network with an embedded clothing structure attention mechanism, the visual silhouette, local structure, and pattern process parameters of the clothing to be retrieved are extracted as fusion features.

[0040] The logic for calculating attention across different channels in clothing structure is as follows: The output feature map of the fourth residual block is divided into three independent channels. Channel 1 learns the global features of the outer contour of the clothing, channel 2 learns the rigid structural features of the collar / shoulder / waist, and channel 3 learns the decorative features of prints and fabric textures. Channel 3 is subjected to a weight suppression of 0~0.3, while channels 1 and 2 are assigned high weights of 0.7~1.0. The channel weights are used to generate a weight mask using the Sigmoid function. The channel fusion calculation formula is as follows:

[0041] ;

[0042] Output fused feature map; Profile channel characteristics, Rigid structure channel Decorative passageway; / / ) represents the weights of each channel. A weight mask is generated by convolution and the Sigmoid function. The weights of the decoration channels are constrained to 0~0.3, and the weights of the structure channels are 0.7~1.0.

[0043] The feature map is output by weighted fusion, which weakens the decoration and strengthens the pattern structure; the original classification fully connected layer is removed from the network, and a 128-dimensional global profile vector, a 256-dimensional local structure vector, and a 64-dimensional pattern process parameter fixed dimension feature vector are output in parallel.

[0044] S4. Two-level hierarchical matching retrieval:

[0045] S41. Coarse-grained retrieval: Call the first-level category index of the database, calculate the similarity using visual profile features, and filter the candidate sample set whose similarity is greater than the first preset threshold; the first threshold is based on 75, and is lowered to 70 in the suit pattern making scenario. The system automatically switches according to the business mode.

[0046] S42, Fine-grained weighted matching: Differentiated feature weights are assigned based on the importance of clothing structure, and the comprehensive matching degree is calculated by integrating three types of features. Invalid samples with a comprehensive matching degree lower than the second preset threshold are removed. The base value of the second threshold is 85, and it is increased to 88 in the infringement comparison scenario. The system is configured adaptively.

[0047] S5. Sort candidate samples from high to low according to the overall matching degree, associate them with the template number, product ID, and complete set of pattern process parameters, and output the search results in a visual format.

[0048] The visual silhouette features include the outer contour boundary of the garment and six standard pattern silhouette labels: H / X / A / T / O / S.

[0049] The local structural features are the outline features of key parts for garment shaping, including the shapes of seven parts: collar, sleeves, waist, shoulders, placket, pockets, and hem.

[0050] The pattern-making process parameters are the bust ease, shoulder width, garment length, dividing lines, pleats, splicing, and grading quantitative pattern-making parameters.

[0051] Differential weighting constraint rule: The total weight of garment shape and structural features is ≥60%, and the weight of decorative accessories such as prints and buttons is <10%.

[0052] The formula for calculating the overall matching degree is as follows:

[0053] ;

[0054] in, For overall matching accuracy, For visual profile similarity, For local structural similarity, For the matching degree of pattern and process parameters; , , These are the corresponding feature weight coefficients, and .

[0055] Visual profile similarity Based on normalized Euclidean distance conversion, the value range is constrained to [0, 1]; local structural similarity Key points of seven categories were extracted, and the cosine similarity of each category was calculated and then weighted averaged; pattern and process parameter matching degree. Normalized difference calculation is performed on continuous quantization parameters, and discrete version labels are directly matched. The arithmetic mean of the results yields a score of 0 to 1.

[0056] It also includes a closed-loop iterative optimization step: constructing a supervised dataset based on the best matching samples selected manually as positive samples and low-match irrelevant samples as negative samples, and fine-tuning the network under supervision.

[0057] Complete calculation rules for the joint loss of triplet and MSE: Construct a triplet (anchor sample a, positive sample p, negative sample n), triplet loss formula:

[0058] ;

[0059] Anchor sample, The same sample Differential negative samples; is the Euclidean distance of the feature vectors; 0.2 is the fixed margin; gradient backpropagation is only performed when the loss is greater than 0.

[0060] MSE loss constrains the eigenvector error of the template: ;

[0061] Pattern feature vector dimension; Real-world pattern parameter vector; The network predicts the pattern vector; the mean square error is used to constrain the fitting accuracy of the pattern features.

[0062] Total loss: ;

[0063] 0.5 is the MSE loss balancing coefficient, which balances feature discrimination and pattern fitting accuracy. This fully explains the role of the coefficient.

[0064] During the fine-tuning phase, the first three residual blocks of the network are frozen, and only the attention module and the output fully connected layer are updated; the network training batch size is 32, and the initial learning rate is 10. -4 The number of fine-tuning iterations is controlled between 10 and 30 rounds.

[0065] Standard configuration with fixed weights for different scenarios:

[0066] Women's apparel e-commerce retail scenarios: =0.3, =0.55, =0.15; Suit industrial pattern making scenario: =0.25, =0.45, =0.3.

[0067] like Figure 2 As shown, another aspect of the present invention provides a clothing style matching and retrieval system for executing any of the aforementioned clothing style matching and retrieval methods, comprising eight basic functional modules: image acquisition module, image preprocessing module, multi-dimensional feature extraction module, database management module, hierarchical matching and retrieval module, weight adaptive configuration module, result output interaction module, and model iteration optimization module.

[0068] The system adds an encrypted storage module, which uses the AES-128 symmetric encryption algorithm to store classified template data;

[0069] AES-128 Encryption Field and Key Management Process: The encrypted object is only a floating-point pattern parameter array such as bust ease, shoulder slope, and dividing line. The array is serialized and encrypted in segments. The original garment pattern and silhouette label are stored in plaintext. The key is divided into three levels: ordinary operators can only read the similarity report, the pattern supervisor can adjust the data of a single set of patterns, and the system administrator holds the hardware root key. The key is permanently stored in the local encryption chip and is not uploaded to the cloud server.

[0070] The system adopts an edge-cloud collaborative deployment architecture;

[0071] Cloud-edge data interaction and transmission process: After the edge preprocesses the 256×256 image, it is compressed into WEBP format and uploaded to the cloud; the cloud only sends the sample ID and matching score back to the edge; the incremental model dataset is sent to the edge cache by the cloud on a daily schedule; the feature vectors of newly added styles are uploaded to the database in batches, and the high-definition original images are permanently stored locally on the edge, reducing the transmission bandwidth usage.

[0072] The system can output standardized multi-dimensional similarity detection reports, fully recording the matching scores of each item, which can be used as evidence of infringement of clothing appearance patents; it supports multi-terminal deployment on mobile phones, industrial computers, and cloud servers, and can be connected to e-commerce back-ends, clothing CAD, and inventory management systems.

[0073] Example 1

[0074] This embodiment is applied to a full-category search scenario in women's clothing e-commerce, and provides a clothing style matching search method, including the following steps:

[0075] S1. Collect a total of 10,000 samples of dresses, suits, knitwear, and skirts; label six silhouette types (H / X / A / T / O / S), 12 collar types, 8 sleeve types, and 28 women's clothing pattern parameters; establish primary and secondary retrieval indexes using hierarchical K-means clustering, with a cluster center update threshold of 10. -3 .

[0076] S2. A dress with floral patterns and a complex indoor background was photographed with a mobile phone. The image was then processed using 3×3 median filtering, 15 GrabCut iterations for image cutout, and perspective correction to create a unified 256×256 RGB image.

[0077] S3. Improved ResNet50 embedding of clothing attention module, channel weight constraint decoration channel ≤ 0.3, output 128 / 256 / 64-dimensional three-class feature vectors.

[0078] S4, coarse-grained threshold 75, filters 21 candidate samples; fine-grained weights. =0.3, =0.55, =0.15, second threshold 85, retain 6 highly matched samples.

[0079] S5. Display in descending order of 98.2%, 95.7%, and 92.1%, and output the complete set of CAD template parameters simultaneously.

[0080] S6. The best manually selected styles are used as positive samples to construct a triplet dataset. The first three residual blocks are frozen and fine-tuned for 20 rounds. The feature vectors of newly added styles are added to the database, and the clustering index is updated synchronously.

[0081] Example 2

[0082] This embodiment applies to a men's suit factory pattern making search scenario, adjusting the adaptive weight coefficient. =0.25, =0.45, =0.3; coarse-grained threshold lowered to 70, fine-grained second preset threshold raised to 88; special attention module increases the weight of lapel, bust dart, and armhole pattern parameter channels; single search latency 160ms, can accurately match different sizes of the same suit pattern, adapting to flexible production lines for rapid pattern adjustment.

[0083] Example 3

[0084] This embodiment is applied to the scenario of clothing style infringement comparison. The system automatically amplifies the weight of the unique structural features of clothing and reduces the weight of the general basic pattern; it retrieves and outputs a sub-item similarity detection report, which fully records the matching scores of the silhouette, seven types of local features, and each pattern parameter.

[0085] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for matching and retrieving clothing styles, characterized in that: Includes the following steps: S1. Collect and label clothing samples, extract three types of features: visual silhouette features, local structural features, and pattern process parameters of clothing, store the associated sample labels, and establish first- and second-level retrieval indexes by hierarchical K-means clustering according to clothing category; S2. Collect original images of the garment from multiple perspectives, and preprocess them by median filtering, background removal, perspective correction, and size normalization. S3. An improved ResNet50 network based on an embedded clothing channel attention module outputs three fixed-dimensional fused feature vectors in parallel: visual silhouette, local structure, and pattern technology. S4. Two-level hierarchical matching retrieval: S41. Coarse-grained retrieval: Match the primary category index of the database, calculate the visual profile similarity, and filter the candidate sample set that is greater than the first adaptive threshold; S42. Fine-grained weighted matching: Differentiated feature weights are assigned according to the priority rules of clothing shape structure, the comprehensive matching degree is calculated by weighting, and samples below the second adaptive threshold are removed. S5. Sort the results in descending order of overall matching degree and output the results visually with layout parameters.

2. The clothing style matching and retrieval method according to claim 1, characterized in that, Visual silhouette features: outer outline of the garment, six standard silhouette labels (H / X / A / T / O / S); local structural features: outlines of seven shaped parts (collar, sleeves, waist, shoulders, placket, pockets, and hem); pattern making parameters: bust ease, shoulder width, garment length, seam lines, pleats, splicing, and grading parameters.

3. The clothing style matching and retrieval method according to claim 2, characterized in that, Differential weight constraint: The total weight of garment shape and structure features is ≥60%, and the weight of print and button decoration features is <10%.

4. The clothing style matching and retrieval method according to claim 3, characterized in that, The formula for calculating the overall matching degree is as follows: ； in, For overall matching accuracy, For visual profile similarity, For local structural similarity, For the matching degree of pattern and process parameters; , , These are the corresponding feature weight coefficients, and .

5. The clothing style matching and retrieval method according to claim 4, characterized in that, visual corridor similarity Based on normalized Euclidean distance conversion, the value range is [0, 1]; the local structural similarity Seven key points were extracted from different parts, and the cosine similarity of each point was weighted and averaged to obtain the pattern pattern process parameters matching degree. It is calculated by fusion of normalized difference of continuous parameters and discrete label matching.

6. The clothing style matching and retrieval method according to claim 5, characterized in that, It also includes a closed-loop iterative optimization step: constructing a triplet dataset by manually selecting similar styles as positive samples and different styles as negative samples, using triplet and MSE joint loss to supervise and fine-tune the network, and incrementally updating the clustering index with new styles.

7. The clothing style matching and retrieval method according to claim 6, characterized in that, Improved ResNet50 by removing the original fully connected classification layer and embedding the attention module into the back end of the fourth residual block; fine-tuned by freezing the first three residual blocks, training batch size of 32, and initial learning rate of 10. -4 Iterate 10-30 times.

8. The clothing style matching and retrieval method according to claim 7, characterized in that, Fixed weights based on different scenarios: Women's clothing scenario =0.3, =0.55, =0.15, suit pattern making scene =0.25, =0.45, =0.3; the coarse / fine matching threshold switches adaptively according to the business scenario.

9. A clothing style matching and retrieval system, characterized in that, The method for performing the clothing style matching retrieval method according to any one of claims 1 to 8 includes: an image acquisition module, an image preprocessing module, a multi-dimensional feature extraction module, a database management module, a hierarchical matching retrieval module, a weight adaptive configuration module, a result output interaction module, and a model iteration optimization module.

10. The clothing style matching and retrieval system according to claim 9, characterized in that, It also includes an AES-128 encrypted storage module, which encrypts the form quantization parameters in segments and implements hierarchical control of hardware keys with three levels of permissions; The system adopts a collaborative architecture of edge-end acquisition and preprocessing, and cloud-based high-performance computing, enabling lightweight image transmission and local storage of high-definition original images.

Images