Clothing material identification method and device

Abstract

The application discloses a clothing material identification method and device, and relates to the technical field of material identification. The method uses a multi-scale fusion network model for material identification. The multi-scale fusion network model uses an input layer to pre-process a real object image to improve the image quality, sets a backbone network to extract features in a shallow layer, a middle layer and a deep layer, and identifies based on the features of each layer, thereby improving the accuracy of clothing material identification.

Description

Technical Field

[0001] This application relates to the field of material identification technology, and in particular to a method and apparatus for identifying clothing materials. Background Technology

[0002] E-commerce platforms use models to identify ingredients in product images uploaded by sellers and in product reviews, comparing them with product ingredient labels to detect fraudulent sales. This is crucial for platform merchant supervision.

[0003] The main challenges in extracting product components are the low recognizability and confusion caused by the visual characteristics of the materials. For example, the surface textures and luster of natural fibers (cotton / viscose) and synthetic fibers (polyester / nylon) are highly similar, making them difficult to distinguish with just a visual image. Furthermore, fabric processing techniques (brushing, wrinkle-resistant treatments) can alter the surface morphology, making the treated synthetic fabrics visually resemble natural fibers, increasing the probability of misjudgment.

[0004] Current mainstream material recognition methods on e-commerce platforms include: using the ResNet50 (Residual Network with 50 layers) model to extract macroscopic features and then performing recognition based on these macroscopic features; and using CNN (Convolutional Neural Network) to extract shallow texture features and then performing recognition based on these shallow texture features.

[0005] Recognition methods based on the ResNet50 model rely solely on global information such as overall gloss and wrinkles. CNN-based recognition methods rely solely on shallow texture features and lack the ability to capture mesoscopic structures and macroscopic attributes. Existing recognition methods are based on features at a single level and cannot take into account both microscopic textures and macroscopic attributes, resulting in a misclassification rate of up to 30% for highly confusing materials such as cotton / viscose and polyester / nylon. Summary of the Invention

[0006] The purpose of this application is to provide a method and apparatus for identifying clothing materials, so as to improve the accuracy of clothing material identification.

[0007] To achieve the above objectives, this application provides the following solution.

[0008] Firstly, this application provides a method for identifying clothing materials, including: Obtain a physical image of the garment to be identified; The physical image is input into a multi-scale fusion network model for material recognition to obtain the recognition result; The multi-scale fusion network model includes an input layer, a backbone network, a fusion layer, and a fully connected output layer. The input layer is used to preprocess the physical image to obtain an input feature map. The backbone network is used to extract multi-scale features from the input feature map to obtain a shallow texture feature map, a mid-level structure feature map, and a deep attribute feature map. The fusion layer is used to fuse the shallow texture feature map, the mid-level structure feature map, and the deep attribute feature map to obtain a fused feature map. The fully connected output layer is used to perform material recognition based on the fused feature map to obtain a recognition result.

[0009] Optionally, the backbone network includes a texture feature extraction subnetwork, a structural feature extraction subnetwork, and an attribute feature extraction subnetwork arranged in parallel; The texture feature extraction sub-network includes: a MobileNetV3-Large network, an LBP network, and a GLCM network; the MobileNetV3-Large network is used to generate a first texture feature map and a second texture feature map based on dynamically separable convolutional layers and SE modules; the LBP network is used to generate a third texture feature map using LBP operators; and the GLCM network is used to calculate and generate a fourth texture feature map through contrast features and correlation features; the shallow texture feature map is obtained by fusing the first texture feature map, the second texture feature map, the third texture feature map, and the fourth texture feature map. The structural feature extraction subnetwork includes: an improved U-Net network; the improved U-Net network is obtained by improving the U-Net network, which is done by: establishing skip connections between corresponding layers of the encoder and decoder in the U-Net network, and setting a double cross-attention mechanism between the encoder and decoder in the U-Net network. The attribute feature extraction subnetwork includes: the Swing Transformer-Tiny network and the spectral feature simulation module.

[0010] Optionally, the MobileNetV3-Large network includes a dynamically separable convolutional layer and an SE module connected in sequence; The dynamically separable convolutional layer comprises a depthwise convolutional layer and a pointwise convolutional layer connected in sequence.

[0011] Optionally, the output feature map of the dynamically separable convolutional layer is: ; ; In the formula, To produce texture features for the output feature map of the deep convolutional layer. , These are the row and column numbers of the pixel space coordinates, respectively. This refers to the channel index of the input feature map. The material features are used as input to the feature map. and These are the x and y coordinates within the convolution kernel of the deep convolutional layer, respectively. The size of the convolution kernel of the depthwise convolutional layer. For the convolution kernel of the c-th channel The weight value of the position. To produce texture features for the output feature map of the pointwise convolutional layer, C The total number of channels in the input feature map. This refers to the channel index of the output feature map of a dynamically separable convolutional layer. The weights are for channel fusion; the output feature map of the pointwise convolutional layer is the first texture feature map.

[0012] Optionally, the texture feature extraction process of the SE module is as follows: The input feature map is subjected to global average pooling using the following formula to obtain the channel statistics; ; In the formula, For channel statistics, This refers to the channel index of the input feature map. and These represent the height and width of the input feature map, respectively. The material features are used as input to the feature map. , These are the row and column numbers of the pixel space coordinates, respectively; Based on the channel statistics, channel weights are generated using two fully connected layers and a Sigmoid activation function. Based on the channel weights, the input feature map is enhanced using the following formula to obtain a second texture feature map; ; In the formula, This is the second texture feature map. For the input feature map, This is the channel weight vector.

[0013] Optionally, the formula for the skip connection is: ; in, and The decoder's first Layer and first The output feature map of the layer, For the encoder's first The output feature map of the layer, For upsampling operation, This is a feature splicing operation.

[0014] Optionally, the dual cross-attention mechanism includes a channel cross-attention mechanism and a spatial cross-attention mechanism; The channel cross-attention mechanism is as follows: ; ; ; ; in, The h-th layer feature token is output by the channel cross-attention mechanism. For the encoder's first Layer feature token, This is the concatenation result of feature tokens from all layers of the encoder. For the encoder's first Query vectors of layer features , These are the key vector and value vector of the global features, respectively. This is the sum of the number of all channels in the output feature maps of each layer of the encoder; The spatial cross-attention mechanism is as follows: ; ; in, For spatial cross-attention mechanism, The number of pixels in the h-th layer feature token output by the channel cross-attention mechanism. The h-th layer enhancement feature output by the spatial cross-attention mechanism. This is the output feature map of the h-th layer of the encoder. For normalization function, For upsampling operation, This is a feature splicing operation.

[0015] Optionally, a texture feature filter, a structure feature filter, and an attribute feature filter are respectively provided after the texture feature extraction subnetwork, the structure feature extraction subnetwork, and the attribute feature extraction subnetwork.

[0016] Optionally, the product images are obtained from the product display area and / or product review area of ​​the seller's online store; The preprocessing includes median filtering and bilinear interpolation.

[0017] In a first aspect, this application provides a clothing material identification device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the above-mentioned clothing material identification method.

[0018] According to the specific embodiments provided in this application, this application has the following technical effects.

[0019] This application provides a method and apparatus for identifying clothing materials. This application utilizes a multi-scale fusion network model for material identification. The multi-scale fusion network model uses the input layer to preprocess the real object image to improve the image quality, sets up a backbone network to extract features of shallow, medium and deep layers, and performs identification based on the features of each layer, thereby improving the accuracy of clothing material identification. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a method for identifying clothing materials according to an embodiment of this application.

[0022] Figure 2 This is a schematic diagram illustrating the principle of a clothing material identification method provided in one embodiment of this application.

[0023] Figure 3 This is a schematic diagram of the structure of a clothing material identification device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] This application aims to address the challenge of distinguishing between highly confusing materials such as cotton vs. viscose and polyester vs. nylon in low-resolution / blurred / obscured images in e-commerce review sections. By employing multi-scale feature complementarity, lightweight design, and a robust architecture, it improves the accuracy and efficiency of identifying false ingredients, helping e-commerce platforms effectively verify whether the advertised ingredients of products match the actual items and protecting consumer rights.

[0027] In one exemplary embodiment, a method for identifying clothing materials is provided, such as... Figure 1 and Figure 2 As shown, it includes the following steps 101-102.

[0028] Step 101: Obtain a physical image of the garment to be identified.

[0029] Step 102: Input the physical image into a multi-scale fusion network model for material recognition and obtain the recognition result.

[0030] The multi-scale fusion network model includes an input layer, a backbone network, a fusion layer, and a fully connected output layer. The input layer is used to preprocess the physical image to obtain an input feature map. The backbone network is used to extract multi-scale features from the input feature map to obtain a shallow texture feature map, a mid-level structure feature map, and a deep attribute feature map. The fusion layer is used to fuse the shallow texture feature map, the mid-level structure feature map, and the deep attribute feature map to obtain a fused feature map. The fully connected output layer is used to perform material recognition based on the fused feature map to obtain a recognition result.

[0031] In the above embodiments, different "levels" of the model are used to capture multi-dimensional features of materials from micro to macro, and then this information is integrated through a specific fusion strategy to eliminate the blind spots of single-scale judgment, while also adapting to defects such as blurring and occlusion in real-world images in the comment section. The core logic is "layered extraction - feature alignment - fusion enhancement", which achieves multi-scale feature complementarity through a hybrid architecture and targeted extraction methods.

[0032] In another exemplary embodiment, the above-described input layer is used to preprocess the physical images in the comment section, using median filtering to remove salt and pepper noise, and bilinear interpolation to resize to 224×224 pixels to retain as much detail as possible.

[0033] In this embodiment of the application, the distinguishing points of clothing materials are dispersed in visual information of different granularities, realizing the extraction of features at three scales: micro, meso and macro. This overcomes the defect that a single scale can only capture a certain type of information (for example, if you only look at the macro gloss, you will misjudge brushed polyester as cotton), and achieves "all-round scanning". Then, all information is integrated to make a judgment, thereby improving the recognition accuracy.

[0034] Microscopic scale: the thickness and cross-sectional shape of fibers (e.g., cotton is a hollow fiber, while polyester is a solid circle) can be indirectly captured through high-frequency information of texture.

[0035] Mesoscale: Weaving method of fabric (plain weave, twill weave), warp and weft linear density (e.g., the coarse texture of denim vs. the fine texture of silk).

[0036] Macro scale: the overall luster of the material (such as the soft luster of wool vs. the strong reflectivity of nylon), and the shape of the wrinkles (the natural wrinkles of cotton vs. the stiff wrinkles of synthetic fibers).

[0037] In another exemplary embodiment, in clothing material recognition, the multi-scale feature extraction of the above-mentioned backbone network is mainly based on the "CNN+Transformer hybrid architecture". The core is "hierarchical extraction + feature alignment + fusion enhancement", which is divided into 3 core levels, corresponding to micro, meso and macro feature extraction respectively. The feature extraction information of each scale is shown in Table 1.

[0038] Table 1 Multi-scale feature extraction information

[0039] In another exemplary embodiment, the backbone network described above includes a texture feature extraction subnetwork, a structural feature extraction subnetwork, and an attribute feature extraction subnetwork configured in parallel. The texture feature extraction subnetwork includes a MobileNetV3-Large network, an LBP network, and a GLCM network. The MobileNetV3-Large network generates a first texture feature map and a second texture feature map based on dynamically separable convolutional layers and an SE (Squeeze-and-Excitation) module. The LBP network generates a third texture feature map using the LBP operator. The GLCM network... The M-network is used to calculate and generate a fourth texture feature map through contrast and correlation features; the shallow texture feature map is obtained by fusing the first texture feature map, the second texture feature map, the third texture feature map, and the fourth texture feature map; the structural feature extraction sub-network includes an improved U-Net network; the improved U-Net network is obtained by improving the U-Net network by establishing skip connections between corresponding layers of the encoder and decoder in the U-Net network, and setting a double cross-attention mechanism between the encoder and decoder in the U-Net network; the attribute feature extraction sub-network includes a Swing Transformer-Tiny network and a spectral feature simulation module.

[0040] In another exemplary embodiment, the texture feature extraction sub-network described above adopts the MobileNetV3 + traditional texture feature enhancement (LBP (Local Binary Pattern) network and GLCM (Gray Level Co-occurrence Matrix) network) approach.

[0041] Model selection: MobileNetV3-Large (input size 224×224).

[0042] Core function: By fusing deep convolution with traditional texture features, it accurately captures fiber-level micro-details, reducing computational load while preserving key texture information.

[0043] In another exemplary embodiment, the MobileNetV3-Large network described above includes a dynamically separable convolutional layer and an SE module connected in sequence.

[0044] Dynamically separable convolutions extract texture features, such as the gaps between cotton fibers, within a local 3×3 area through channel-wise convolution. This dynamically separable convolutional layer consists of sequentially connected depthwise convolutional layers and pointwise convolutional layers.

[0045] Among them, the depthwise convolutional layer is used to process the input feature map. ,use The convolutional kernel processes each channel sequentially to output a feature map. : In the formula, The material features of the input feature map correspond to the preprocessed material features of the real object image in the comment section, such as texture and color information; : Output features of deep convolutional layers (local texture features extracted channel by channel, such as the gaps between cotton fibers). Texture features are output from the feature map of the deep convolutional layer; Pixel space coordinates , These are the row and column numbers of the pixel space coordinates, respectively; c: Channel index of the input feature map (each channel corresponds to a basic feature, such as brightness or texture edge); K: The size of the convolution kernel of the deep convolutional layer, typically 3, i.e., a 3×3 convolution kernel, used to capture texture within a local 3×3 area; : The internal coordinates of the convolution kernel (0 to K-1, traversing each pixel of the convolution kernel). and These are the x and y coordinates within the convolution kernel of the deep convolutional layer, respectively; , which are the parameters learned during model training and used to match material texture features.

[0046] Pointwise convolution is used to... Channel fusion is performed using convolutional kernels to output feature maps. : The computational cost of this pointwise convolution is reduced to that of traditional convolution. ,like It reduces by about 90% at times.

[0047] Pointwise convolution outputs feature maps (comprehensive features that fuse multi-channel textures). To provide texture features for the output feature map of the pointwise convolutional layer; , which represents the new number of channels, such as compressing / expanding from 96 dimensions to 256 dimensions; C: The total number of channels in the input feature map, the number of channels retained after depthwise convolution, such as 96 channels in MobileNetV3; The weight matrix of pointwise convolution (dimension C×C', used to fuse texture features from different channels, such as merging "fiber thickness" and "pore density" features). The weights for channel fusion.

[0048] The output feature map of the pointwise convolutional layer is the first texture feature map.

[0049] In another exemplary embodiment, the SE module described above processes the input feature map. Channel statistics are obtained through global average pooling. : ; Channel weights are generated through two fully connected layers (FC) and sigmoid activation. Final output This enhances key channel texture information (such as the fiber pores of cotton) and suppresses irrelevant interference.

[0050] Channel statistics, i.e., the global average pooling result of the c-th channel, the average value of the channel feature, reflecting the strength of the channel feature, such as the average response of the "cotton texture" channel; : Input the height and width of the feature map, such as 14×14 pixels, which corresponds to the size of the local area of ​​the material; s: Channel weight vector, dimension C×1, each element corresponds to the importance score of a channel, such as "fiber texture" channel weight 0.8, "background color" channel weight 0.2; The Sigmoid activation function is used to map weight values ​​to the range of 0-1, thereby achieving channel filtering. The ReLU activation function is used to introduce non-linearity and enhance the expressive power of the model. , The weight matrix of the fully connected layer. : C×(C / 16), (C / 16)×C, used to learn the correlation between channels; The second texture feature map, i.e., the output feature map of the SE module, is used to enhance key texture channels and suppress irrelevant interference. Element-wise multiplication is used to multiply the feature map of each channel with its corresponding weight to achieve channel enhancement.

[0051] In another exemplary embodiment, Local Binary Pattern (LBP) is used for each pixel of the input feature map. The sign of the difference between its 8 neighboring pixels and the center pixel is calculated to generate a binary code that represents the difference between the rough texture of cotton and the fine texture of viscose.

[0052] , : These are the row and column numbers of the pixel space coordinates, respectively; P: Number of neighboring pixels, typically 8, i.e., 8-neighborhood, which consists of 8 pixels surrounding the center pixel; R: Neighborhood radius, typically 1, meaning the distance from a neighboring pixel to the center pixel is 1; : The grayscale value of the center pixel (the brightness of that point in the material map, such as the light grayscale of cotton and the dark grayscale of synthetic fibers); The gray value of the p-th neighboring pixel; : The sign function is used to determine whether the gray level of neighboring pixels is greater than or equal to that of the center pixel, and outputs 0 or 1; : Binary encoded weights (encoding the 0 / 1 results of 8 neighborhoods into 8-bit binary numbers, i.e., LBP values, used to characterize local texture) In another exemplary embodiment, the gray-level co-occurrence matrix (GLCM) is used to calculate contrast and correlation features: in, It is a grayscale pair The probability of occurrence, the elements of the gray-level co-occurrence matrix (the probability that gray values ​​i and j appear simultaneously in a specific direction and at a certain distance, such as the high probability of "light gray - light gray" co-occurrence in the texture of cotton); Grayscale index (the grayscale value level of the material map, such as 0-255); Grayscale The mean of all The weighted average of i; Grayscale The mean of all middle The weighted average; Grayscale Standard deviation; Grayscale Standard deviation; Contrast: Reflects the clarity of material texture (e.g., denim has high contrast, while silk has low contrast). Correlation: Reflects the regularity of material texture (e.g., plain weave fabric has high correlation, irregular texture has low correlation).

[0053] In another exemplary embodiment, the encoder and decoder of the improved U-Net network described above are both 4 layers, which are used to complete the features of the occluded area and enhance the spatial correlation of the woven structure through skip connections and double cross attention.

[0054] In another exemplary embodiment, the above-described skip connection connects the encoder's first... Layer features and decoder Layer feature stitching, ensuring size alignment (e.g., zero padding) and Dimensions are consistent.

[0055] The decoder's first The output feature map of the layer (mesoscopic structural features after fusing encoder details, such as the plain / twill structure of a fabric). The encoder's first The output feature map of the layer (local structural features extracted during U-Net downsampling, such as the intersection of latitude and longitude lines); The decoder's first The output feature map of the layer (deep abstract structural features, such as the overall weaving style); Upsampling operation (to Enlarged to the size of Consistent (e.g., scaling up from 8×8 to 16×16). Feature concatenation operation (merging the encoder's detailed features with the decoder's abstract features along the channel dimension, preserving structural details). In another exemplary embodiment, the above-described dual cross-attention mechanism includes a channel cross-attention mechanism and a spatial cross-attention mechanism.

[0056] Features output from each layer of the encoder Flattened token spliced ​​together .

[0057] In another exemplary embodiment, the above-described channel cross-attention mechanism is as follows: Calculate attention weights and output: .

[0058] The encoder's first Layer feature token (flattening the feature map into a vector, dimension) P is the number of pixels. (Number of channels); The concatenated result of feature tokens from all layers of the encoder (dimensions) , (This is the sum of the number of all channels; for example, the total number of channels in a 4-layer encoder is 512.) The encoder's first The query vector for layer features (generated by 1D depthwise convolution, used to match global features); K: Key vector of global features (used for...) Calculate similarity); V: The value vector of the global features (used to generate attention-weighted features); 1D depthwise convolution (channel-wise convolution, reducing computation while preserving channel specificity). : The transpose of the matrix; The sum of the number of channels in the output feature maps of each layer of the encoder (used for normalization to avoid excessively high similarity values). Normalization function (converts similarity into 0-1 attention weights to highlight important feature associations).

[0059] In another exemplary embodiment, the above-described spatial cross-attention mechanism is as follows: The h-th layer feature token (dimension) output by the channel cross-attention mechanism. (It has already incorporated global channel features). P: The number of pixels of the h-th layer feature token output by the channel cross-attention mechanism (used for normalization to avoid overflow in spatial similarity calculation). : The transpose matrix (used to calculate the spatial correlation between different pixels, such as the spatial correlation of pixels in twill fabric). Output: Weighted feature tokens (enhancing spatially regular structural features, such as the continuity of woven textures).

[0060] In another exemplary embodiment, the attribute feature extraction subnetwork described above is simulated using SwingTransformer-Tiny+ spectral features.

[0061] Model selection: Swing Transformer-Tiny (input size 224×224).

[0062] Core function: By using window attention and spectral simulation, it captures macroscopic properties such as overall gloss and wrinkles, amplifying the essential differences in materials.

[0063] Core algorithm formula: Window Attention is used to divide the feature map into... The window, for each token in the window Calculate attention: Where B is the relative position encoding matrix (dimension N×N, used to supplement the spatial position information of pixels, such as the positional association of "top left - bottom right"), which is pre-calculated using a sine function: , : No. The and the first The relative position encoding value of each pixel (calculated by a sine function to simulate the effect of spatial position on features).

[0064] Computational complexity from Down to ; : Query, key, and value vectors, derived from feature maps, with dimensions N×C, where N is the number of pixels within the window and C is the number of channels; : Total number of pixels in the window (M is the window size, typically 7, i.e., a 7×7 window, N=49); C: Number of feature channels (typically 768 for Swin-Tiny, used to capture macroscopic attribute features); : and The transpose product (calculates the similarity between the query and the key, reflecting the feature associations of different pixels). Normalization factor (to avoid excessively high similarity values, which could cause the Softmax gradient to vanish); B: Relative position encoding matrix; Normalization function (converts similarity into attention weights to highlight globally relevant macroscopic features, such as the overall gloss of a material). The spectral feature simulation employs Gabor filtering, which is used to simulate the near-infrared spectral response after converting the RGB image to HSV space using a two-dimensional Gabor filter. in, The parameters are set as follows: , , , .

[0065] Formula parameter description: : The original coordinates of the physical image (material image pixels in HSV space); : Rotated coordinates (used to adjust the filtering direction and match the directionality of the material texture, such as the texture direction of twill fabric); Filter direction (typically 45°, used to capture diagonal textures, such as the twill of denim). Gaussian window width (typical value 10, controls the range of filtering; the larger the value, the wider the texture area covered). The aspect ratio of the filter (typically 0.5, which makes the filter elliptical to fit fine texture features). Filter wavelength (typical value 10, controls the capture of texture period, such as the coarse texture of cotton corresponding to a longer wavelength). Phase offset (typical value 0, controls the starting position of the cosine wave, does not affect the overall extraction of texture features); : Output value of Gabor filter (simulates near-infrared spectral response, enhances the differences in chemical characteristics of materials, such as the spectral differences between cotton and chemical fibers).

[0066] In another exemplary embodiment, a texture feature filter, a structure feature filter, and an attribute feature filter are respectively provided after the texture feature extraction subnetwork, the structure feature extraction subnetwork, and the attribute feature extraction subnetwork.

[0067] Texture feature filters are used for shallow feature filtering, eliminating invalid texture channels with channel response values ​​below 0.3; The structural feature filter is used for mid-level feature filtering to remove structural features with a double cross-attention (DCA) weight of less than 0.1; The attribute feature filter is used for deep feature filtering, and effective macroscopic attribute features are selected by using a spectral feature similarity threshold (0.2).

[0068] In another exemplary embodiment, the aforementioned fusion layer is used to integrate feature maps from different levels into a unified-dimensional "multi-scale feature vector." The fusion method is as follows: shallow texture features (256-dimensional LBP+GLCM), mid-level structural features (64-dimensional DCA output), and deep attribute features (768-dimensional Swin-T output) are concatenated into a 1088-dimensional vector. in, Shallow texture feature vector (256 dimensions, including texture features of LBP and GLCM); : Mid-level structure feature vector (64-dimensional, including the braided structure features output by DCA); : Deep attribute feature vector (768 dimensions, including macroscopic gloss and wrinkle features output by Swin-T); : The fused feature vector (1088 dimensions, the total vector integrating multi-scale features).

[0069] In another exemplary embodiment, the fully connected output layer described above reduces the dimensionality using two fully connected layers (512→256) and Dropout (0.5), outputting component probabilities: in, Use the ReLU activation function (to introduce non-linearity and improve the model's classification ability). : Weight matrices of the fully connected layers (W1: 1088×512, W2: 512×256, used for feature dimensionality reduction and classification mapping); Material type r (e.g., r=0 represents cotton, r=1 represents polyester fiber, for a total of R types of materials); : The probability of predicting the material as the r-th type (between 0 and 1, such as P(y=0)=0.95 representing a 95% probability of cotton); Normalization function (adjusts the sum of probabilities of all categories to 1, making it easier to determine the most likely material).

[0070] Based on the above embodiments, the differences between the multi-scale fusion network model used in this application and existing models are shown in Table 2.

[0071] Table 2 Comparison of Model Structures

[0072] This application employs a hybrid architecture of MobileNetV3 (shallow layer), Swin Transformer-Tiny (deep layer), and U-Net (feature fusion), balancing lightweight design with multi-scale feature capture capabilities. The model has approximately 12M parameters and achieves an accuracy of 94.3% on the Fashion-MNIST material subset and a dataset of 100,000 real-world e-commerce review images (compared to 81.2% for traditional methods). Its specific applications in material obfuscation scenarios include: 1. Resolve the confusion of "macroscopic similarities but microscopic differences" (e.g., cotton vs. viscose): Macroscopically, both have a soft luster and natural wrinkles, making them difficult to distinguish in deep networks; shallow networks capture that "cotton has a coarser texture and obvious warp and weft lines," while viscose has "a finer texture and uniform pores," and can be distinguished by microscopic details after fusion.

[0073] 2. Resolve confusion regarding "microscopic similarities but macroscopic differences" (e.g., polyester vs. nylon): Microscopically, both have fine textures and no obvious fiber gaps, and shallow networks are easily misjudged; deep networks capture "polyester's reflection is more concentrated and the wrinkles are stiffer", while nylon's "reflection is more dispersed and the wrinkles are elastic", and after fusion, they can be distinguished by macroscopic features.

[0074] 3. Adapt to the defects of images in the comment section (such as obstruction or blurring): When a certain area is occluded, shallow networks cannot capture local textures, but the macroscopic features of deep networks (such as overall gloss) can fill the gap; when the image is blurry, the woven structure features of middle-layer networks (such as diagonal and flat patterns) are more resistant to interference than shallow textures, and multi-scale fusion can reduce misjudgments caused by blur.

[0075] In one exemplary embodiment, a clothing material identification device is provided. This device can be a server or a terminal, and its internal structure diagram can be as follows: Figure 3 As shown, this clothing material recognition device includes a processor, memory, input / output (I / O) interface, and communication interface. The processor, memory, and I / O interface are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage medium. The I / O interface is used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a clothing material recognition method.

[0076] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the clothing material identification device to which the present application is applied. A specific clothing material identification device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] In one exemplary embodiment, a clothing material identification device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0078] In an exemplary embodiment, the following experiment was conducted to illustrate the technical effects of the technical solution provided in the embodiments of this application.

[0079] 1. Experimental Objective This study verifies the superiority of the technical solution of "multi-level hybrid architecture, multi-scale feature fusion, and combination of traditional and deep features" in terms of lightweight, high-obfuscation material differentiation, and anti-interference capability, and proves its effectiveness in solving the defects of existing technologies.

[0080] 2. Experimental Environment See Table 3 for the experimental environment.

[0081] Table 3 Experimental Environment

[0082] 3. Compare experimental design and results (to verify overall superiority) Three mainstream models in the prior art were selected as control groups and compared with the scheme of this application to verify the technical advantages, as shown in Table 4.

[0083] Table 4 Comparison Results

[0084] 4. Ablation Experiment Design and Results (Verifying the Necessity of the Core Module) Through a control experiment involving the removal of a core module, the necessity of shallow traditional texture features (LBP+GLCM), mid-layer dual cross attention (DCA), deep spectral feature simulation (Gabor), and 3D fusion strategy is verified, as shown in Table 5.

[0085] Table 5 Ablation Experiment Results

[0086] 5. Key Conclusions Significant advantages of lightweight design: This application has 12.0M parameters, which is only 46.9% of ResNet50 and 13.8% of ViT-Base; the inference speed is 12FPS, which is 50% higher than ResNet50, meeting the real-time verification needs of e-commerce platform edge devices.

[0087] Strong ability to distinguish highly confusing materials: the accuracy rates for cotton vs. viscose and polyester vs. nylon reach 92.1% and 90.7% respectively, which is 18%-22 percentage points higher than existing technologies, solving the problem of material confusion due to "macroscopic similarity / microscopic similarity".

[0088] It has outstanding anti-interference capabilities: the accuracy of blurred and occluded images reaches 89.5% and 82.1% respectively, which is 19.2-21.4 percentage points higher than the existing technology, and it is suitable for low-resolution and defective image scenarios in the comment section.

[0089] Core modules are indispensable: removing any core module (LBP+GLCM, DCA, Gabor, 3D fusion) reduces accuracy by 3.1-5.6 percentage points, demonstrating the crucial supporting role of each module in performance.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0091] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0092] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0093] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying clothing materials, characterized in that, include: Obtain a physical image of the garment to be identified; The physical image is input into a multi-scale fusion network model for material recognition to obtain the recognition result; The multi-scale fusion network model includes an input layer, a backbone network, a fusion layer, and a fully connected output layer. The input layer is used to preprocess the object image to obtain an input feature map. The backbone network is used to extract multi-scale features from the input feature map to obtain a shallow texture feature map, a mid-level structure feature map, and a deep attribute feature map. The fusion layer is used to fuse the shallow texture feature map, the mid-level structure feature map, and the deep attribute feature map to obtain a fused feature map. The fully connected output layer is used to perform material recognition based on the fused feature map to obtain a recognition result. The backbone network includes a texture feature extraction subnetwork, a structural feature extraction subnetwork, and an attribute feature extraction subnetwork that are configured in parallel. The texture feature extraction sub-network includes: a MobileNetV3-Large network, an LBP network, and a GLCM network; the MobileNetV3-Large network is used to generate a first texture feature map and a second texture feature map based on dynamically separable convolutional layers and SE modules; the LBP network is used to generate a third texture feature map using LBP operators; and the GLCM network is used to calculate and generate a fourth texture feature map through contrast features and correlation features; the shallow texture feature map is obtained by fusing the first texture feature map, the second texture feature map, the third texture feature map, and the fourth texture feature map. The structural feature extraction subnetwork includes: an improved U-Net network; the improved U-Net network is obtained by improving the U-Net network, which is done by: establishing skip connections between corresponding layers of the encoder and decoder in the U-Net network, and setting a double cross-attention mechanism between the encoder and decoder in the U-Net network. The attribute feature extraction subnetwork includes: a Swin Transformer-Tiny network and a spectral feature simulation module; The texture feature extraction subnetwork, the structural feature extraction subnetwork, and the attribute feature extraction subnetwork are respectively followed by a texture feature filter, a structural feature filter, and an attribute feature filter; Texture feature filters are used to remove invalid texture channels with channel response values ​​below 0.3; The structural feature filter is used to remove structural features with a double cross-attention weight of less than 0.1; The attribute feature filter is used to screen effective macroscopic attribute features by using a spectral feature similarity threshold.

2. The method for identifying clothing materials according to claim 1, characterized in that, The MobileNetV3-Large network includes a dynamically separable convolutional layer and an SE module connected in sequence; The dynamically separable convolutional layer comprises a depthwise convolutional layer and a pointwise convolutional layer connected in sequence.

3. The garment material identification method according to claim 2, characterized in that, The output feature map of the dynamically separable convolutional layer is: ； ； In the formula, To produce texture features for the output feature map of the deep convolutional layer. , These are the row and column numbers of the pixel space coordinates, respectively. This refers to the channel index of the input feature map. The material features are used as input to the feature map. and These are the x and y coordinates within the convolution kernel of the deep convolutional layer, respectively. The size of the convolution kernel of the depthwise convolutional layer. For the convolution kernel of the c-th channel The weight value of the position. To produce texture features for the output feature map of the pointwise convolutional layer, C The total number of channels in the input feature map. This refers to the channel index of the output feature map of a dynamically separable convolutional layer. The weights are for channel fusion; the output feature map of the pointwise convolutional layer is the first texture feature map.

4. The method for identifying clothing materials according to claim 2, characterized in that, The texture feature extraction process of the SE module is as follows: The input feature map is subjected to global average pooling using the following formula to obtain the channel statistics; ； In the formula, For channel statistics, This refers to the channel index of the input feature map. and These represent the height and width of the input feature map, respectively. The material features are used as input to the feature map. , These are the row and column numbers of the pixel space coordinates, respectively; Based on the channel statistics, channel weights are generated using two fully connected layers and a Sigmoid activation function. Based on the channel weights, the input feature map is enhanced using the following formula to obtain a second texture feature map; ； In the formula, This is the second texture feature map. For the input feature map, This is the channel weight vector.

5. The method for identifying clothing materials according to claim 1, characterized in that, The formula for the jump connection is: ； in, and The decoder's first Layer and first The output feature map of the layer, For the encoder's first The output feature map of the layer, For upsampling operation, This is a feature splicing operation.

6. The method for identifying clothing materials according to claim 1, characterized in that, Dual cross-attention mechanisms include channel cross-attention mechanisms and spatial cross-attention mechanisms; The channel cross-attention mechanism is as follows: ； ； ； ； in, The h-th layer feature token is output by the channel cross-attention mechanism. For the encoder's first Layer feature token, This is the concatenation result of feature tokens from all layers of the encoder. For the encoder's first Query vectors of layer features , These are the key vector and value vector of the global features, respectively. This is the sum of the number of all channels in the output feature maps of each layer of the encoder; The spatial cross-attention mechanism is as follows: ； ； in, For spatial cross-attention mechanism, The number of pixels in the h-th layer feature token output by the channel cross-attention mechanism. The h-th layer enhancement feature output by the spatial cross-attention mechanism. This is the output feature map of the h-th layer of the encoder. For normalization function, For upsampling operation, This is a feature splicing operation.

7. The method for identifying clothing materials according to claim 1, characterized in that, The product images are obtained from the product display area and / or product review area of ​​the seller's online store; The preprocessing includes median filtering and bilinear interpolation.

8. A clothing material identification device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the clothing material identification method according to any one of claims 1-7.

Images