A clothing image classification method and system based on dynamic decision fusion

Abstract

The application discloses a clothing image classification method and system based on dynamic decision fusion, and belongs to the technical field of computer vision and image classification. Clothing image data sets are acquired and preprocessed; parallel convolutional neural network branches and Transformer branches are constructed to extract local texture features and global structure features of the clothing image respectively; cross-attention mechanisms are used to bidirectionally interact and enhance the double-flow features; the enhanced features are spliced to obtain fusion features; a dynamic weight network is constructed to adaptively generate three decision weights corresponding to the features obtained by the two branches and the fusion features; three parallel classifiers are set to classify and predict the three kinds of features; and the outputs of the three classifiers are weighted and fused by using the dynamic weights to obtain a final classification result. The application effectively improves the precision and stability of clothing image classification through complementary extraction of double-flow features, cross-attention interaction enhancement and adaptive distribution of dynamic weights.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image classification technology, specifically involving a clothing image classification method and system based on dynamic decision fusion, which is particularly suitable for fine-grained classification tasks of clothing images in e-commerce scenarios where clothing datasets have centralized labeling problems. Background Technology

[0002] In recent years, with the rapid development of internet technology, the e-commerce industry has flourished. Clothing, as a core category on e-commerce platforms, has seen its online transaction volume continue to expand, generating massive amounts of clothing image data. How to efficiently and accurately classify and manage these images has become a pressing technical problem for e-commerce platforms.

[0003] Clothing image classification faces numerous challenges in practical applications. First, clothing itself possesses diverse styles, significant color variations, and complex textures, making feature learning difficult. Second, during the imaging process, factors such as shooting angle, lighting conditions, and background environment vary considerably, further increasing the complexity of the classification task. Furthermore, folding, stretching, and other morphological changes that may occur during the shooting process can also interfere with the model's effective recognition.

[0004] Even more challenging is the issue of overlapping and severe occlusion in real-world e-commerce scenarios and complex clothing combinations. For instance, a model might wear a coat, shirt, and underwear simultaneously, causing key areas of the inner garments (such as the collar, cuffs, and hem) to be largely obscured by the outer garments. This occlusion not only leads to a loss of visual information but also causes a serious "attribute labeling gap" at the dataset construction level. Taking the well-known Fashionpedia dataset as an example, annotation experts often cannot accurately determine fine-grained attributes (such as "opening type," "waistline position," or "length") when faced with severely occluded or viewpoint-limited instances, and can only label them as "not sure" or leave them blank. Statistics show that this dataset contains a large number of attribute labeling gaps caused by occlusion, and the same image often contains multiple instances of the same category (such as two layered tops), with each instance's attribute labeling being independent and often incomplete.

[0005] Early clothing image feature extraction mainly employed handcrafted feature methods, such as color histograms, directional gradient histograms, and local binary patterns. However, these traditional handcrafted features can only capture the underlying visual information of the image, making it difficult to express the semantic category of clothing. Furthermore, they exhibit poor robustness under interference from changes in shooting angle, cluttered backgrounds, and clothing deformation, resulting in generally low classification accuracy.

[0006] In recent years, the development of deep learning technology has provided a more effective technical approach for clothing image classification. Convolutional neural networks can extract local features from images with fine precision, while the Transformer architecture has significant advantages in global information modeling and capturing long-distance dependencies. Researchers have attempted to combine the two, proposing a two-stream fusion architecture. However, most existing methods employ simple feature concatenation or weighted summation, which is insufficient for fully exploring the semantic relationships and information interactions between the two-stream features. More importantly, existing techniques typically assume that the feature information of the input image is complete, or treat the weights of all regions as equal, lacking the ability to adapt to "occlusion perception." When faced with the attribute loss problem caused by multi-instance occlusion, existing models often fail to perform effective inference and completion based on the local texture and overall structural constraints of the visible parts, leading to decreased or unstable classification confidence.

[0007] Furthermore, stochastic vector functional chains (SVAs), as a single-hidden-layer feedforward neural network, have the advantages of fast training speed and good generalization ability. However, their randomized generation of input weights and hidden layer biases leads to unstable classification performance. Existing optimization methods suffer from problems such as low optimization efficiency and susceptibility to getting trapped in local optima.

[0008] Therefore, how to effectively integrate the dual-stream features of CNN and Transformer to improve the stability and generalization ability of classifiers, especially how to use dynamic decision-making mechanisms to adaptively integrate local visible cues and global structural constraints in complex scenarios with multiple instance occlusion and uncertain attribute labels, and achieve robust inference of missing attributes, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] To address the problems existing in the prior art, the present invention provides a clothing image classification method and system based on dynamic decision fusion.

[0010] The technical solution adopted in this invention is as follows: A clothing image classification method based on dynamic decision fusion aims to solve the problems of multi-instance overlay occlusion and missing attribute annotations commonly found in datasets like Fashionpedia. The method includes the following steps: Step S1: Obtain the clothing image dataset and perform preprocessing; Step S2: Construct parallel convolutional neural network branches and Transformer branches to extract local texture features and global structural features of the clothing image as two-stream features, respectively; the local texture features include the texture features of the unoccluded local areas; the global structural features include the global structural features of the partially occluded entire garment. Step S3: The dual-stream features extracted in step S2 are enhanced bidirectionally through a cross-attention mechanism to address the semantic gap between local texture features and global structural features caused by occlusion, resulting in enhanced CNN features and enhanced Transformer features. Step S4: Concatenate the enhanced CNN features and the enhanced Transformer features obtained in step S3 to obtain the fused features; Step S5: Construct a dynamic weight network. Input the fused features obtained in step S4 into the dynamic weight network to adaptively generate the first decision weights corresponding to the CNN features, the second decision weights corresponding to the Transformer features, and the third decision weights corresponding to the fused features. The more severe the clothing occlusion in the image, the greater the second decision weight; the less severe the clothing occlusion in the image, the greater the first decision weight. Step S6: Set up three parallel classifiers, taking the enhanced CNN features obtained in step S3, the enhanced Transformer features, and the fused features obtained in step S4 as inputs, to obtain the first prediction result, the second prediction result, and the third prediction result. Step S7: Using the first decision weight, second decision weight and third decision weight generated in step S5, the first prediction result, second prediction result and third prediction result obtained in step S6 are weighted and fused to obtain the final classification result.

[0011] Preferably, in step S2, the convolutional neural network branch uses EfficientNetV2-M as the backbone network; the Transformer branch uses Swing Large Transformer as the backbone network.

[0012] Preferably, in step S3, the cross-attention mechanism adopts a bidirectional cross-attention structure, including: performing a first cross-attention calculation using local texture features as the query and global structural features as the key and value; performing a second cross-attention calculation using global structural features as the query and local texture features as the key and value; and using residual connections in both cross-attention calculations.

[0013] Preferably, in step S5, the dynamic weight network is composed of a feature gating unit and a meta-network connected in series; the feature gating unit adaptively filters the fused features; the meta-network is composed of three fully connected layers, and its output is normalized by the Softmax function to obtain three decision weights, the sum of the three decision weights being 1.

[0014] Preferably, in step S6, the three parallel classifiers are all classification heads composed of two fully connected networks.

[0015] Preferably, in step S6, all three parallel classifiers employ regularized stochastic vector functional chain networks (SVMs), whose outputs are represented as a weighted combination of the hidden layer node outputs and the original inputs. The hyperparameters of the SVMs are optimized using a Bayesian optimization algorithm, and the hyperparameters include the number of hidden layer nodes, regularization coefficients, sparsity, and ensemble weights.

[0016] Preferably, in step S1, the preprocessing includes scaling the image size to a uniform size and performing normalization; during the training process, a weighted binary cross-entropy loss function is used to handle the sample imbalance problem, and the class weight is inversely proportional to the number of class samples.

[0017] The present invention also provides a clothing image classification system based on dynamic decision fusion, for executing the clothing image classification method based on dynamic decision fusion, comprising: The data acquisition and preprocessing module is used to acquire clothing image datasets and perform preprocessing. The dual-stream feature extraction module includes parallel convolutional neural network branches and Transformer branches, which are used to extract local texture features and global structural features of clothing images, respectively. The cross-attention enhancement module is used to perform bidirectional interactive enhancement of two-stream features; The feature concatenation module is used to concatenate the enhanced CNN features and the enhanced Transformer features to obtain fused features; The dynamic weight generation module contains a dynamic weight network, which is used to adaptively generate three decision weights based on the fusion features. The parallel classification module contains three parallel classifiers, which are used to classify and predict the enhanced CNN features, enhanced Transformer features, and fused features, respectively. The weighted fusion output module is used to weight and fuse the prediction results of the three classifiers using three decision weights, and output the final classification result.

[0018] Preferably, the dynamic weight network in the dynamic weight generation module is composed of a feature gating unit and a meta-network connected in series; the feature gating unit contains two fully connected layers for adaptive filtering of fused features; the meta-network consists of three fully connected layers, and the output is normalized by the Softmax function to obtain three decision weights.

[0019] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the clothing image classification method based on dynamic decision fusion.

[0020] Compared with the prior art, the present invention has the following beneficial effects: First, the complementary features of the two streams and deep interaction effectively solve the problem of information loss in occluded scenarios.

[0021] This invention employs a dual-stream architecture that parallelizes CNN and Transformer. The CNN branch is responsible for capturing local texture and detail features of the unoccluded parts of the garment (such as exposed neckline embroidery and cuff texture), while the Transformer branch is responsible for modeling the global structural features of the entire garment (such as the silhouette and the proportions of the top and bottom). Specifically, this invention achieves deep semantic interaction between the two-stream features through a bidirectional cross-attention mechanism, enabling locally visible cues to actively "query" the global structural context, and vice versa. This mechanism overcomes the information barriers of existing simple splicing or weighted summation fusion methods. When faced with multiple instances overlapping or severe local occlusion, it can infer missing local information through global structural constraints, significantly improving the model's reasoning ability for incomplete visual signals.

[0022] Second, dynamic decision fusion of occlusion perception enables adaptive inference in scenarios with missing attributes.

[0023] To address the issues of missing attribute annotations (e.g., "not sure") and imbalanced samples in real-world datasets like Fashionpedia, this invention designs an innovative dynamic weight network. This network adaptively generates three decision weights corresponding to CNN features, Transformer features, and the fused features based on the fusion features. When a clothing instance in an image is severely occluded, the dynamic weight network automatically increases the weight of the global Transformer branch, relying on structural priors for inference; when visible details are abundant, it emphasizes texture recognition in the CNN branch. This "sample-adaptive" fusion strategy, compared to traditional fixed-weight fusion, can more intelligently handle varying degrees of occlusion and annotation uncertainty, achieving robust recovery of missing attributes.

[0024] Third, the three-level parallel classification and consistency regularization greatly improve the stability and generalization ability of the classifier.

[0025] This invention constructs a unique dual mechanism of "feature-level fusion + decision-level fusion". By setting up three parallel classifiers to classify and predict the enhanced CNN features, Transformer features, and fused features respectively, and introducing consistency regularization loss to force the output distribution of the two-stream branches to converge. This design not only retains the advantages of each stream but also avoids misclassification drift of a single branch under severe occlusion through regularization constraints. Experimental results show that the method of this invention achieves a mean accuracy (mAP) of 0.8787 to 0.8804 on the Fashionpedia dataset, with a standard deviation of only 0.0015 for three independent runs, demonstrating its extremely high stability in complex occlusion scenarios and outperforming various existing fusion strategies and classic classification models.

[0026] Fourth, the potential for enhanced visual-ontology collaborative reasoning capabilities.

[0027] Although the basic implementation does not explicitly introduce a knowledge graph, this invention essentially constructs a collaborative reasoning framework that integrates "local details (visual)" and "global structure (context)" through cross-attention mechanisms and dynamic weight allocation. This lays the structural foundation for subsequently introducing attribute logical relationships from the Fashionpedia ontology (such as the coexistence of "opening type" and "button" attributes), enabling the model to not only rely on image features when processing ambiguous samples, but also implicitly utilize the structured prior knowledge contained in the dataset, further improving the accuracy of fine-grained classification.

[0028] Experimental results show that the method of the present invention achieves an average accuracy of 0.8787 to 0.8804 on the Fashionpedia dataset, with a standard deviation of only 0.0015 for three independent runs, outperforming various existing fusion strategies and classic classification models. Attached Figure Description

[0029] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the structure of the dual-stream feature extraction network of the present invention; Figure 3 This is a schematic diagram of the cross-attention interaction enhancement module of the present invention; Figure 4 This is a schematic diagram of the structure of the dynamic weight network of the present invention; Figure 5 This is a diagram of the overall model architecture of the present invention; Figure 6 This is a comparison diagram of the box plots of this invention and other models. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1 This embodiment provides a clothing image classification method based on dynamic decision fusion. The overall flowchart of the method is as follows: Figure 1 Specifically, it includes the following steps: Step 1: Dataset preparation and preprocessing.

[0032] This embodiment uses the Fashionpedia dataset, which contains 46 clothing categories. The training set contains 45,623 images, and the validation set contains 1,158 images. Each image is labeled with multiple category labels, making it a multi-label classification dataset. The input images are preprocessed by scaling them to 224×224 pixels and normalizing them so that pixel values ​​are distributed between 0 and 1.

[0033] Due to the severe long-tail distribution and attribute label gaps caused by occlusion in the dataset, this embodiment uses a weighted binary cross-entropy loss function during training, where the class weight is inversely proportional to the number of class samples.

[0034] Step 2: Construct a two-stream feature extraction network.

[0035] like Figure 2 As shown, parallel CNN and Transformer branches are constructed. The CNN branch uses EfficientNetV2-M as the backbone network, fine-tuned based on weights pre-trained on the ImageNet dataset. After the input image passes through the EfficientNetV2-M network, it outputs a 1280-dimensional local texture feature vector. For example, for a shirt heavily obscured by a coat, the CNN branch is used to extract texture features from the unobscured local areas (such as exposed cuffs and collars).

[0036] The Transformer branch uses the Swin Large Transformer as the backbone network, and is also initialized using ImageNet pre-trained weights. After the input image passes through the Swin Large Transformer network, it outputs a 1536-dimensional global structural feature vector. Utilizing the window attention mechanism of the Swin Large Transformer network, this branch can capture the global structure of the entire garment (such as the length ratio and shoulder width contour), and the global structural information remains relatively complete even if some parts are occluded.

[0037] Since the output dimensions of the two branches are inconsistent, a projection layer is introduced to map the two sets of features to a unified 512-dimensional space.

[0038] Step 3: Enhance cross-attention interaction.

[0039] like Figure 3 As shown, the aligned features are first input into a multi-head self-attention layer for self-enhancement, with the number of attention heads set to 8.

[0040] Due to occlusion, a semantic gap exists between CNN features (local) and Transformer features (global). This gap is addressed through bidirectional cross-attention to achieve complementarity. The first cross-referencing (local query global): uses CNN features as the query and Transformer features as the key and value. For example, when the CNN captures details such as "lace texture" but cannot determine its location, this computation allows "lace texture" to query the global features for "Does this usually appear at the neckline or the cuff?", thus supplementing the local features with occluded contextual information.

[0041] The second cross-referencing (global query for local): Transformer features are used as the query, and CNN features are used as the key and value. For example, when the Transformer sees the outline of a "high-waisted" style but the waist details are covered by the top, it queries the CNN features for "whether there are exposed belt or trouser loop details?", thereby enriching the detailed representation of the global structure.

[0042] Both sets of cross-attention employ residual connections to ensure that the original, valid information of the unoccluded portion extracted from the occluded clothing is not lost due to interaction.

[0043] Step 4: Dynamic weight allocation.

[0044] like Figure 4 As shown, the two enhanced sets of features are concatenated to obtain a 1024-dimensional fused feature, which is then input into a dynamic weight network. The dynamic weight network is used to simulate the human cognitive process when faced with occlusion. The dynamic weight network first performs adaptive filtering through feature gating: the gating network consists of two fully connected layers, which are activated by a sigmoid function and then multiplied element-wise with the original fused features. This step suppresses noise features caused by occlusion (such as blur artifacts in occluded areas) and enhances effective feature channels. The first layer reduces the 1024-dimensional feature to 256-dimensional feature and activates it with ReLU. The second layer maps the 256-dimensional feature back to 1024-dimensional feature, activates it with a sigmoid function, and then multiplies it element-wise with the original fused features. The gated features are then input into the meta-network, which consists of three fully connected layers with dimensions of 1024, 256, and 3 respectively. The output is normalized by Softmax to obtain three decision weights, the sum of which is 1. When clothing occlusion is severe in the image, the confidence of local details in the CNN of the fused features is low, and the dynamic weight network tends to increase the weights of the Transformer branch, relying on the global structure for inference; when visible details are abundant, the weights of the CNN branch are increased.

[0045] Step 5: Classification and weighted fusion.

[0046] Three parallel classifiers receive CNN-enhanced features, Transformer-enhanced features, and fused features, respectively. Each classifier consists of two fully connected layers. The predicted probabilities output by the three classifiers are dynamically weighted and summed to obtain the final classification result. This mechanism ensures that even if one branch fails due to severe occlusion, the other branches can still provide reliable prediction support.

[0047] Step 6: Model training.

[0048] The AdamW optimizer was used, with an initial learning rate of 5×10⁻⁶. -5 The weight decay is set to 1×10. -4 The batch size was set to 32, and the number of training rounds was set to 50. The learning rate was dynamically adjusted using a cosine annealing strategy.

[0049] To address the incompleteness of attribute annotations in Fashionpedia, this embodiment introduces Consistency Loss to minimize the mean squared error of predictions from the CNN and Transformer branches. This forces the two branches to learn complementary and aligned representations in occluded scenarios—that is, predictions for the same attribute should tend to be consistent regardless of whether the view is local or global, thereby significantly improving the model's robustness to annotation noise (such as "not sure").

[0050] In the total loss function, the balance coefficient for consistency regularization loss is set to 0.001, and the L2 regularization coefficient is set to 1×10. -5 .

[0051] Step 7: Model evaluation and prediction.

[0052] The trained model was evaluated on the Fashionpedia validation set. Mean precision, Top-3 accuracy, F1 score, precision, and recall were used as evaluation metrics. Experimental results show that the method in this embodiment achieves a mean precision of 0.8804 and a standard deviation of 0.0015 across three independent runs, outperforming comparative methods such as simple concatenation fusion, sequential progressive fusion, iterative cross fusion, and gated fusion.

[0053] Example 2 Based on Example 1, this embodiment addresses the problem of extremely imbalanced fine-grained attribute classification in the Fashionpedia dataset by introducing an optimized stochastic vector function chain classifier to replace the original fully connected classifier, and using a Bayesian optimization algorithm to optimize its hyperparameters.

[0054] Because Fashionpedia contains 294 fine-grained attributes, and some attributes have very few samples, such as the special "open-front type", traditional fully connected classifiers are prone to overfitting or getting stuck in local optima.

[0055] The output of the stochastic vector function chain (SVC) classifier is represented as a weighted combination of the hidden layer node outputs and the original input. A regularized SVC is employed, incorporating an L2 regularization term into the optimization objective to control model complexity.

[0056] A Bayesian optimization algorithm is used to optimize the hyperparameters of a stochastic vector function chain. The search space includes: the number of hidden layer nodes ranging from 400 to 1200, the regularization coefficient ranging from 0.05 to 0.3, the sparsity ranging from 0.1 to 0.4, and the ensemble weights ranging from 0.4 to 0.7. A Gaussian process is used as the probabilistic surrogate model, and the expectation improvement is used as the acquisition function to search for the optimal parameter combination within a finite number of iterations.

[0057] The experimental results on the Fashionpedia dataset in this embodiment show that the Bayesian-optimized stochastic vector function chain classifier achieves the highest mean accuracy and the smallest standard deviation, outperforming comparative methods such as swarm intelligence optimization algorithms, random search, and grid search.

[0058] Model architecture description as follows Figure 5 As shown, the overall model architecture of this invention is as follows: The input image is simultaneously fed into a CNN branch and a Transformer branch to extract local texture features and global structural features, respectively. After alignment by a projection layer, the two sets of features are enhanced bidirectionally through a cross-attention module. The enhanced features are then input into three parallel classifiers, while the concatenated fused features are input into a dynamic weight network to generate adaptive weights. Finally, the outputs of the three classifiers are weighted and fused using dynamic weights to obtain the final classification result.

[0059] Experimental data analysis and verification To verify the effectiveness of the clothing image classification method based on dynamic decision fusion proposed in this invention, this section presents a comprehensive experimental test based on the Fashionpedia dataset and compares it with various existing technologies.

[0060] Performance Comparison Analysis of Fusion Strategies As shown in Table 1, the method of this invention (i.e., "the method of this paper" in the table) performs excellently in all evaluation metrics. Specifically, the mAP (mean accuracy) of the method of this invention reaches 0.8804, which is significantly higher than that of EfficientNetV2-M (0.7984) and Swin Large Transformer (0.8465), verifying the complementary advantages of the two-stream architecture.

[0061] Table 1: Performance Comparison of Fusion Strategies

[0062] The advantages of the method of this invention are particularly evident when compared with existing fusion strategies: Compared to simple splicing and fusion (0.8389), this invention improves by approximately 4.15%, demonstrating the necessity of cross-attention mechanism in mining semantic associations, rather than simple feature stacking.

[0063] Compared to serial progressive fusion (0.8631) and iterative cross fusion (0.8579), this invention further unleashes the potential of dual-stream features by introducing a dynamic weight network for decision-level fusion.

[0064] Compared to gated fusion (0.8544), the dynamic weighting mechanism of this invention not only considers the importance of feature channels, but also combines the global context of fused features to generate decision weights, thereby achieving better performance.

[0065] Model stability analysis Stability is an important indicator for measuring the maturity of a patented technology solution. As shown in Table 2, the standard deviation of the method of this invention in three independent training sessions is only 0.0015.

[0066] Table 2: Statistics of each model's three runs

[0067] In comparison, the standard deviation of simple splicing fusion is as high as 0.0188, while that of gated fusion is 0.0030. Figure 6The box plot visually demonstrates that the method of this invention has the narrowest box, indicating its insensitivity to random initialization and its extremely high stability for industrial applications. This stability benefits from the alignment effect of the consistency regularization loss on the two-stream features and the robustness of the dynamic weight network to noisy samples.

[0068] Learning rate sensitivity analysis To verify the robustness of the model to hyperparameters, a learning rate sensitivity experiment was conducted (see Table 3). The results show that the performance fluctuation of the method of this invention is extremely small (maximum standard deviation of only 0.0029) within the range of 1e-5 to 1e-4, and it reaches its optimum (0.8787±0.0015) when the learning rate is 5e-5. This indicates that the method of this invention is insensitive to the choice of learning rate and is easy to reproduce and deploy.

[0069] Table 3: Results of Learning Rate Sensitivity Analysis

[0070] Comparison of RVFL classifier optimization algorithms Table 4 shows the performance of different optimization algorithms for the Bayesian optimized RVFL classifier used in Example 2. Bayesian optimization outperforms the GSO algorithm, random search, and grid search in both average mAP (0.8838) and stability (standard deviation 0.0012). In particular, Bayesian optimization can efficiently find the optimal combination of hyperparameters such as the number of hidden layer nodes and regularization coefficients, avoiding the high computational cost of traditional grid search, demonstrating its superiority in handling fine-grained attribute classification tasks.

[0071] Table 4: Comparison Results of Optimization Algorithms

[0072] 8.5 Comprehensive Comparison with Advanced Models As shown in Tables 5 and 6, the method of this invention not only significantly outperforms classic models such as ResNet, VGG, and DenseNet (with an mAP lead of approximately 5%-8%), but also demonstrates competitiveness in comparison with recent state-of-the-art (SOTA) models. The method of this invention (0.8787) is superior to ConvNeXt-Base (0.8588), Swin-Base (0.8601), and Next-ViT-Base (0.8541). This proves that the dual mechanism of "feature-level interaction (cross-attention) + decision-level fusion (dynamic weights)" is more adaptable to the multi-scale, multi-label characteristics of clothing images than a single advanced backbone network.

[0073] Table 5: Comparison Results of Classical Models

[0074] Table 6: Comparison Results of Advanced Models

[0075] In summary, the experimental data fully demonstrate the significant technological advancements of this invention in classification accuracy, model stability, and anti-interference capabilities. It is particularly suitable for solving complex scenarios in the Fashionpedia dataset, such as multi-label, fine-grained, and missing attribute annotations.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A clothing image classification method based on dynamic decision fusion, characterized in that, Includes the following steps: Step S1: Obtain the clothing image dataset and perform preprocessing; Step S2: Construct parallel convolutional neural network branches and Transformer branches to extract local texture features and global structural features of the clothing image as two-stream features, respectively; the local texture features include the texture features of the unoccluded local areas; the global structural features include the global structural features of the partially occluded entire garment. Step S3: The dual-stream features extracted in step S2 are enhanced bidirectionally through a cross-attention mechanism to address the semantic gap between local texture features and global structural features caused by occlusion, resulting in enhanced CNN features and enhanced Transformer features. Step S4: Concatenate the enhanced CNN features and the enhanced Transformer features obtained in step S3 to obtain the fused features; Step S5: Construct a dynamic weight network. Input the fused features obtained in step S4 into the dynamic weight network to adaptively generate the first decision weights corresponding to the CNN features, the second decision weights corresponding to the Transformer features, and the third decision weights corresponding to the fused features. The more severe the clothing occlusion in the image, the greater the second decision weight; the less severe the clothing occlusion in the image, the greater the first decision weight. Step S6: Set up three parallel classifiers, taking the enhanced CNN features obtained in step S3, the enhanced Transformer features, and the fused features obtained in step S4 as inputs, to obtain the first prediction result, the second prediction result, and the third prediction result. Step S7: Using the first decision weight, second decision weight and third decision weight generated in step S5, the first prediction result, second prediction result and third prediction result obtained in step S6 are weighted and fused to obtain the final classification result.

2. The clothing image classification method based on dynamic decision fusion according to claim 1, characterized in that, In step S2, the convolutional neural network branch uses EfficientNetV2-M as the backbone network; the Transformer branch uses Swing Large Transformer as the backbone network.

3. The clothing image classification method based on dynamic decision fusion according to claim 1, characterized in that, In step S3, the cross-attention mechanism adopts a bidirectional cross-attention structure, including: performing a first cross-attention calculation with local texture features as the query and global structural features as the key and value; performing a second cross-attention calculation with global structural features as the query and local texture features as the key and value; and using residual connections in both cross-attention calculations.

4. The clothing image classification method based on dynamic decision fusion according to claim 1, characterized in that, In step S5, the dynamic weight network is composed of a feature gating unit and a meta-network connected in series; the feature gating unit performs adaptive filtering of the fused features; the meta-network is composed of three fully connected layers, and its output is normalized by the Softmax function to obtain three decision weights, the sum of the three decision weights is 1.

5. The clothing image classification method based on dynamic decision fusion according to claim 1, characterized in that, In step S6, the three parallel classifiers are all classification heads composed of two fully connected networks.

6. The clothing image classification method based on dynamic decision fusion according to claim 1, characterized in that, In step S6, all three parallel classifiers use regularized stochastic vector functional chain networks (SVMs), whose outputs are weighted combinations of hidden layer node outputs and original inputs. The hyperparameters of the SVMs are optimized using a Bayesian optimization algorithm, and the hyperparameters include the number of hidden layer nodes, regularization coefficient, sparsity, and ensemble weights.

7. The clothing image classification method based on dynamic decision fusion according to claim 1, characterized in that, In step S1, the preprocessing includes scaling the image size to a uniform size and performing normalization processing; During training, a weighted binary cross-entropy loss function is used to address the imbalanced sample problem, where the class weight is inversely proportional to the number of samples in each class.

8. A clothing image classification system based on dynamic decision fusion, used to execute the clothing image classification method based on dynamic decision fusion as described in any one of claims 1 to 7, characterized in that, include: The data acquisition and preprocessing module is used to acquire clothing image datasets and perform preprocessing. The dual-stream feature extraction module includes parallel convolutional neural network branches and Transformer branches, which are used to extract local texture features and global structural features of clothing images, respectively. The cross-attention enhancement module is used to perform bidirectional interactive enhancement of two-stream features; The feature concatenation module is used to concatenate the enhanced CNN features and the enhanced Transformer features to obtain fused features; The dynamic weight generation module contains a dynamic weight network, which is used to adaptively generate three decision weights based on the fusion features. The parallel classification module contains three parallel classifiers, which are used to classify and predict the enhanced CNN features, enhanced Transformer features, and fused features, respectively. The weighted fusion output module is used to weight and fuse the prediction results of the three classifiers using three decision weights, and output the final classification result.

9. The clothing image classification system based on dynamic decision fusion according to claim 8, characterized in that, The dynamic weight generation module consists of a dynamic weight network composed of a feature gating unit and a meta-network connected in series. The feature gating unit contains two fully connected layers for adaptive filtering of fused features. The meta-network consists of three fully connected layers, and the output is normalized by the Softmax function to obtain three decision weights.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 7.

Images