Fusion attention enhanced unbalanced garment fine-grained classification method and system thereof

Abstract

This invention relates to a fine-grained classification method and system for unbalanced clothing based on attention enhancement, belonging to the field of computer vision and image recognition technology. It extracts a first feature map and a second feature map to obtain a first spatial response map and a second spatial response map. Based on the pixel activation intensity distribution and pixel activation statistical characteristics of the response maps, it obtains an overlap rate index and an adaptive threshold. Spatial filtering is performed on the first feature map to obtain a filtered first feature map, and cross-level residual enhancement processing is performed on it to output a target discrimination feature vector. The classification result and its feedback optimization parameters are obtained to update the adaptive threshold. This method helps to eliminate the interference of background noise and spurious activation on fine-grained discrimination, significantly improving the robustness and practical value of the classification system.

Description

Technical Field

[0001] This application belongs to the field of computer vision and image recognition technology, specifically an attention-enhanced fine-grained classification method and system for unbalanced clothing. Background Technology

[0002] With the rapid development of e-commerce and smart retail, fine-grained image classification of clothing has become an increasingly popular research topic in the field of computer vision. It aims to distinguish subcategories with high visual similarity within the same major category. Unlike general object recognition, clothing categories naturally have a tree-like semantic hierarchy. This hierarchy provides prior knowledge for classification and also places higher demands on feature extraction algorithms. Existing technologies typically combine convolutional neural networks with end-point attention mechanisms. These mechanisms achieve fine-grained classification by extracting discriminative activation regions from high-level feature maps. However, when these methods delve into the fine-grained subclass discrimination level, their attention weights become overly focused on the global contour, losing the fine-grained discriminative anchor points located in the mid-to-low-level feature maps. This results in a systematic error in the model, where the parent class is accurately identified but the subclass is randomly identified, severely limiting the practical value of the classification system. Therefore, how to accurately activate the discriminative features of the middle-level subclasses while preserving the semantic constraints of the high-level parent class, achieve non-equitable feature fusion through cross-level causal guidance, and effectively protect the fine-grained discriminative ability of the minority classes under imbalanced data conditions has become an urgent technical problem to be solved. Existing technologies need to be improved to address these issues. Summary of the Invention

[0003] The purpose of this application is to provide a fine-grained classification method and system for unbalanced clothing that incorporates attention enhancement, aiming to eliminate the interference of background noise and spurious activation on fine-grained discrimination, and significantly improve the robustness and practical value of the classification system.

[0004] The objective of this application can be achieved through the following technical solution: Firstly, a fine-grained classification method for unbalanced clothing that incorporates attention enhancement is provided, comprising the following steps: Acquire the image of the clothing to be identified, and extract the first feature map representing its local detail features and the second feature map representing its global contour features; The first feature map and the second feature map are subjected to saliency compression processing of the channel dimension to obtain the first spatial response map and the second spatial response map, and spatial scale alignment processing is performed on the two. Based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map, the overlap rate index, which characterizes the spatial attribution relationship between the local detail region and the global contour region of the clothing image, is obtained. An adaptive threshold for extracting the activation region of the clothing image is generated based on the pixel activation statistics of the second spatial response map, and a spatial mask matrix of the activation region is generated. Spatial filtering is performed on the first feature map based on the spatial mask matrix to obtain the filtered first feature map. Based on the overlap rate index, cross-level residual enhancement processing guided by the second feature map is performed on the filtered first feature map to output the target discrimination feature vector; The classification result of the clothing image is obtained based on the target discrimination feature vector, a feedback optimization parameter representing the confidence level of the classification result is obtained, and the adaptive threshold is updated based on the parameter.

[0005] Secondly, an attention-enhanced fine-grained classification system for unbalanced clothing includes the following modules: The image acquisition module is used to acquire the image of the clothing to be identified and extract a first feature map representing its local detail features and a second feature map representing its global contour features. The first processing module is used to perform saliency compression processing on the first feature map and the second feature map respectively to obtain a first spatial response map and a second spatial response map, and to perform spatial scale alignment processing on the two. The image evaluation module is used to obtain an overlap rate index that characterizes the spatial attribution relationship between the local detail region and the global contour region of the clothing image based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map. The second processing module is used to generate an adaptive threshold for extracting the activation region of the clothing image based on the pixel activation statistics of the second spatial response map, and generate a spatial mask matrix of its activation region, and perform spatial filtering processing on the first feature map based on the spatial mask matrix to obtain the filtered first feature map. The image classification module is used to perform cross-level residual enhancement processing based on the second feature map on the filtered first feature map based on the overlap rate index, so as to output a target discrimination feature vector, obtain the classification result of the clothing image according to the target discrimination feature vector, obtain feedback optimization parameters characterizing the confidence level of the classification result, and update the adaptive threshold according to them.

[0006] Thirdly, a computer storage medium storing computer-executable instructions, which, when executed, implement the attention-enhanced fine-grained classification method for unbalanced clothing described in the first aspect.

[0007] Compared with the prior art, the beneficial effects of this application are: This application uses the global contour semantics of the high-level second feature map as the anchor point, and performs legality verification and gating filtering on the detail activation points of the middle-level first feature map through a spatial mask matrix. Only the subclass detail features whose physical location conforms to the semantic inclusiveness of the parent class are retained, which fundamentally eliminates the interference of background noise and pseudo-activation on fine-grained discrimination. This application proposes an overlap rate index and uses it as a dynamic gain coefficient to accurately calculate residual details unique to subclasses but not possessed by parent classes, significantly improving the classification accuracy of minority classes. Through a closed-loop feedback mechanism, the classification system achieves self-correction to form a complete closed-loop correction link, enabling the system to dynamically optimize the classification boundary for different image qualities, lighting conditions, and class distributions, significantly improving the robustness and practical value of the classification system. Attached Figure Description

[0008] Figure 1 This is a schematic diagram illustrating the steps of the attention-enhanced fine-grained classification method for unbalanced clothing proposed in this application. Figure 2 This is a schematic diagram of the module of the attention-enhanced unbalanced clothing fine-grained classification system of this application. Detailed Implementation

[0009] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but only to illustrate selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item has been defined in one figure, it does not need to be further defined and explained in subsequent figures. The terms "first", "second", etc. are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0010] In traditional fine-grained clothing classification systems, mainstream solutions rely on single-scale attention mechanisms at the ends of deep convolutional networks to extract discriminative features, ignoring the inherent tree-like semantic hierarchy of clothing categories. Because there is a lack of explicit causal feedback pathways between high-level and mid-level feature maps, the semantic perception scope of deep attention mechanisms is limited to the global contour level, failing to effectively guide the precise activation of mid-level fine-grained discriminative anchor points. Under imbalanced data conditions, this problem is further amplified; the fine-grained features of minority classes have weak responses at deeper levels and are easily overwhelmed by the common features of the parent classes of the majority classes, leading to systematic failures in subclass discrimination.

[0011] For example, in an imbalanced clothing dataset, the visual difference between stand-up collar shirts and turn-down collar shirts mainly lies in the details of the collar structure (belonging to mid-level local features). However, existing attention mechanisms can only perceive the global outline of a shirt in deep feature maps, failing to effectively capture the unique details and textures of the collar. When stand-up collar shirts are a minority class in the training set, the deep activation of their fine-grained features is extremely weak, often leading the system to misclassify them as turn-down collar shirts, exhibiting a systematic defect of correctly classifying the parent class while misclassifying the child class. Without addressing these issues, the discriminative power of fine-grained levels will be inversely proportional to the depth of the class hierarchy, severely limiting the application value of fine-grained classification systems in real-world long-tail distribution scenarios.

[0012] To address the aforementioned challenges, this application first establishes a cross-level semantic causal constraint mechanism. By extracting high-level global contour features and mid-level local detail features, an explicit spatial attribution reference system is constructed between the two. Using the high-level parent class semantics as the legitimacy verification anchor point, mid-level detail features are gating and filtered. Furthermore, an overlap rate index quantifying the spatial attribution relationship between the two is introduced, using a dynamic gain coefficient to drive cross-level residual enhancement, precisely amplifying fine-grained differences unique to the subclass but not possessed by the parent class. Finally, through a closed-loop adaptive threshold feedback mechanism, the spatial filtering boundary is continuously optimized based on the confidence level of the classification results, achieving the self-correction capability of the classification system.

[0013] Therefore, such as Figure 1 As shown, this application provides a fine-grained classification method for unbalanced clothing that incorporates attention enhancement, including the following steps: Acquire the image of the clothing to be identified, and extract the first feature map representing its local detail features and the second feature map representing its global contour features; The first feature map and the second feature map are subjected to saliency compression processing of the channel dimension to obtain the first spatial response map and the second spatial response map, and spatial scale alignment processing is performed on the two. Based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map, the overlap rate index, which characterizes the spatial attribution relationship between the local detail region and the global contour region of the clothing image, is obtained. An adaptive threshold for extracting the activation region of the clothing image is generated based on the pixel activation statistics of the second spatial response map, and a spatial mask matrix of the activation region is generated. Spatial filtering is performed on the first feature map based on the spatial mask matrix to obtain the filtered first feature map. Based on the overlap rate index, cross-level residual enhancement processing guided by the second feature map is performed on the filtered first feature map to output the target discrimination feature vector; The classification result of the clothing image is obtained based on the target discrimination feature vector, a feedback optimization parameter representing the confidence level of the classification result is obtained, and the adaptive threshold is updated based on the parameter.

[0014] This invention further proposes that the process of acquiring a clothing image to be identified and extracting a first feature map representing its local detail features and a second feature map representing its global contour features includes: inputting the clothing image to be identified into a preset feature extraction network. The feature extraction network can be constructed using a Residual Network (ResNet), a Visual Transformer (ViT), or other pre-trained convolutional neural networks. Its function is to abstract the image in the original pixel space layer by layer into feature representations with different semantic levels through multi-layer nonlinear transformations. The feature extraction network, as a basic component of this invention (not an innovation), can adopt existing standard pre-trained models in the field. The focus of this invention is on cross-level causal constraint processing of its output features at different levels.

[0015] Based on the feature extraction network, a feature map with a first spatial resolution and a first number of channels is extracted from its middle layer and used as the first feature map to characterize the local detail features of the garment, including subclass discrimination information such as neckline shape, cuff texture, and button arrangement; a feature map with a second spatial resolution and a second number of channels is extracted from its end layer and used as the second feature map to characterize the global category outline features of the garment, including parent category semantic information such as overall garment body distribution and garment category type.

[0016] The first spatial resolution is higher than the second spatial resolution, and the first number of channels is less than the second number of channels. This design follows the inherent characteristics of convolutional neural networks: as network depth increases, the spatial resolution of feature maps gradually decreases while the number of channels gradually increases. High-level feature maps have stronger semantic abstraction capabilities but weaker spatial detail resolution, while mid-level feature maps retain richer spatial detail information but have relatively limited semantic abstraction capabilities. By simultaneously extracting features from both levels, this invention lays the data foundation for subsequent cross-level causal constraints.

[0017] To illustrate with a specific example: When using ResNet-50 as the feature extraction network, the first feature map can be taken from the output of layer 3, with a spatial resolution of 28×28 and 1024 channels; the second feature map can be taken from the output of layer 4, with a spatial resolution of 14×14 and 2048 channels. The spatial resolution of the first feature map (28×28) is higher than that of the second feature map (14×14), and the number of channels in the first feature map (1024) is less than that in the second feature map (2048).

[0018] The present invention further proposes a process in which the first feature map and the second feature map are subjected to saliency compression processing of the channel dimension to obtain a first spatial response map and a second spatial response map, and spatial scale alignment processing is performed on the two, including: The purpose of channel-dimensional saliency compression is to compress a high-dimensional multi-channel feature map into a two-dimensional spatial saliency distribution map for subsequent position alignment and spatial relationship quantization. The specific implementation is as follows: For both the first and second feature maps, obtain the set of activation intensities for each pixel across all channel directions. Record the maximum activation intensity value in each set as the activation intensity value of that pixel in the corresponding spatial response map. Through the above channel max pooling operation, the first feature map (size...) The first spatial response map (size ) is compressed into a two-dimensional shape. ), second feature map (size is The second spatial response map is compressed into a two-dimensional shape (size: ).

[0019] The physical significance of this step is that, for each spatial coordinate location, the most significant activation response across all feature channels is retained, resulting in a two-dimensional heatmap reflecting the overall saliency intensity of that location. This removes redundant channel information interference, laying the foundation for subsequent spatial location comparison. Compared to channel average pooling, channel maximum pooling better preserves sparse but important local activation peaks, which is beneficial for subsequent precise localization of fine-grained discrimination anchor points.

[0020] Because the first and second feature maps come from different layers of the network, their corresponding spatial response maps have different resolutions. and ,in, , Since pixel-by-pixel position comparison cannot be performed directly, spatial scale alignment is required. A bilinear interpolation algorithm is used to upsample the second spatial response map, ensuring its spatial dimensions (height and width) are identical to the first spatial response map. The bilinear interpolation algorithm estimates the activation intensity of each pixel at the target resolution by weighted averaging the activation intensities of neighboring pixels. This maintains the spatial continuity of the response map while increasing resolution, avoiding the checkerboard artifacts caused by simple nearest-neighbor interpolation.

[0021] After completing bilinear interpolation, pixel coordinate calibration is performed on the upsampled second spatial response map according to the downsampling step size of the preset feature extraction network. This includes necessary center cropping or edge padding operations to ensure that each pixel in the aligned second spatial response map corresponds completely to the physical spatial position of the first spatial response map in the original clothing image. This establishes an accurate spatial inclusion reference system between layers, providing an accurate positional benchmark for subsequent overlap rate index calculation.

[0022] Continuing with the specific embodiments described above: The initial resolution of the second spatial response map is 14×14. After bilinear interpolation upsampling, it is expanded to 28×28, and it achieves pixel-by-pixel spatial coordinate correspondence with the first spatial response map. After alignment, it is recorded as the aligned second spatial response map.

[0023] The present invention further proposes a process for obtaining an overlap index characterizing the spatial attribution relationship between local detail regions and global contour regions of a clothing image, based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map. This process includes: The overlap rate index is the core quantitative indicator of this invention. Its physical meaning is to measure the spatial topological affiliation between the local detail regions extracted by the first feature map and the global contour regions represented by the second feature map. In other words, it quantitatively characterizes whether the activation of local details in the middle layer physically falls within the contour range of the higher-level major categories. The introduction of this index is a significant innovation of this invention. Existing feature fusion schemes do not explicitly quantify the spatial affiliation of cross-level features, while this invention uses this index as a link to transform cross-level spatial topological relationships into dynamic weights that can participate in feature synthesis operations, thus realizing a mathematical expression of causal relationships between levels.

[0024] The specific calculation process of the overlap rate index is as follows: First, the aligned second spatial response map is multiplied pixel by pixel with the first spatial response map to obtain an interaction matrix representing the common regions of the hierarchy. Each element in the interaction matrix is ​​equal to the product of the activation intensity values ​​of the two response maps at the corresponding coordinate position. A high value means that the coordinate shows significant activation response in both levels, that is, it has both contour salience and detail salience, and belongs to the common activation region of the hierarchy; a value close to zero means that there is a significant inconsistency in the activation of the two levels at that coordinate, which may be a background region or a pseudo-activation point.

[0025] Subsequently, based on the interaction matrix and the pixel activation intensities of the two response maps, the overlap rate index is calculated according to the idea of ​​Intersection-over-Union (IoU). This index uses the sum of the pixel-wise products of the activation intensities of the two response maps as the numerator, and the sum of the pixel-wise union energies of the activation intensities of the two response maps (i.e., the sum of the pixel activation intensities minus their product, to avoid double-counting common regions) as the denominator, thus obtaining a normalized spatial overlap rate metric. Mathematically, let... Represents the interaction matrix (where Equal to the element-wise product of the aligned second spatial response map and the first spatial response map, with the symbol... This represents the summation of the activation intensity values ​​of all pixels within the matrix. This represents the aligned second spatial response map, with Representing the first spatial response map, the overlap index is... for: ; This overlap ratio index has an intuitive causal logic at the physical level: if A value close to 1 indicates a high degree of overlap between the activated areas of local details in the middle layer and the activated areas of the global outline in the upper layer. The detailed features are fully attached to the main body of the garment, creating a strong sense of spatial belonging. If... If the value is close to 0, it indicates that the mid-level detail feature is largely distributed outside the main body of the garment, and may be background impurities or false activations, which should be suppressed. This index will serve as a dynamic weighting coefficient for subsequent cross-level residual enhancement, driving the feature synthesis process to adaptively adjust the enhancement magnitude based on the spatial assignment reliability of the detail feature.

[0026] The present invention further proposes a process for generating an adaptive threshold for extracting the activation region of the clothing image based on the pixel activation statistical characteristics of the second spatial response map, generating a spatial mask matrix of the activation region, and performing spatial filtering on the first feature map based on the spatial mask matrix to obtain a filtered first feature map, comprising: This invention uses the average activation intensity value of all pixels in the aligned second spatial response map as an adaptive threshold. The core significance of introducing an adaptive threshold lies in the significant differences in the activation intensity distribution of feature maps for different clothing images—the average activation value of a dark coat on the feature map might be 0.8, while that of a light-colored shirt might be only 0.3. Using a fixed threshold (such as 0.5) could lead to an excessively large mask for a dark coat (mistakenly including the background in the subject's activation region), while the mask for a light-colored shirt might disappear entirely (over-filtering). By using the statistical mean of pixel activation in the current image as the adaptive threshold, it can be ensured that regardless of image quality, lighting conditions, or clothing color, the mask boundary is always dynamically anchored to the physical boundary of the "subject activation region" in the current image, guaranteeing the robustness of spatial filtering.

[0027] In the aligned second spatial response map, the activation intensity value is greater than the adaptive threshold. The set of coordinates is defined as the activation region (i.e., the activation region of the main body of the clothing), and the activation intensity value is less than or equal to the adaptive threshold. The set of coordinates is defined as the background region, and the pixels in the active region are assigned a higher weight value than the pixels in the background region, thus forming the logical basis for the subsequent generation of the spatial mask matrix.

[0028] To obtain the mask value corresponding to the pixel at coordinate (i,j) in the spatial mask matrix, this invention utilizes the Logistic mapping function, based on the activation intensity value of the second spatial response map aligned at that coordinate relative to an adaptive threshold. The offset is used to generate continuous weight values ​​in a non-linear manner. Specifically, let... Let be the mask value of the spatial mask matrix at coordinates (i,j). The value of the activation intensity at coordinates (i,j) in the aligned second spatial response map. If the preset smoothing control coefficients are used, the formula for generating the spatial mask matrix is ​​as follows: ; The logistic mapping function exhibits smooth nonlinear saturation characteristics: when the activation intensity at a certain coordinate position is much higher than the adaptive threshold... When the mask value approaches 1, it indicates that the location belongs to the activation region, allowing the corresponding mid-level detail features to pass through completely; when the activation intensity at a certain coordinate position is much lower than the adaptive threshold... When the mask value approaches 0, it indicates that the location belongs to the background region, and the corresponding mid-level detail features are strongly filtered out; when the activation intensity equals the adaptive threshold... At this point, the mask value is precisely 0.5, achieving a smooth boundary transition.

[0029] Smoothing control coefficient Controlling the hardness or softness of the mask boundaries: When the value is large, the mask boundary tends to be hard truncated (binarized distribution of 0 or 1), which is suitable for images with clear subject outlines; When the size is small, the mask boundaries become smoother and more gradual, making it suitable for images with more diffuse feature activation distribution. Compared to a simple hard thresholding scheme, the logistic mapping assigns a gradual weight between 0 and 1 to the boundary region, effectively avoiding the step noise introduced by the binarized mask at the boundary, thus forming a "soft space filter" based on broad semantics, enabling a smooth gradual transition between the activation region and the background region.

[0030] Obtain the spatial mask matrix (size is) After that, it is broadcast and expanded along the channel dimension so that its dimension is the same as the first feature map before performing spatial filtering. (size is) The results are consistent. Then, element-wise multiplication (i.e., the Adama product, denoted as symbol ) is used. The spatial mask matrix after broadcast expansion is multiplied point-by-point at corresponding coordinate positions by the first feature map to obtain the filtered first feature map. Let... For the first feature map after filtering, then ; From a physical perspective, when the mask matrix When the value at a certain position in the mask matrix approaches 0 (background region), all channel activation values ​​of the first feature map at that position are multiplied by a weight close to zero, effectively erasing them; when the mask matrix... When the value at a certain position approaches 1 (activation region), the activation value of the first feature map at that position is almost completely preserved. This step physically achieves spatial filtering of mid-level local detail features based on the high-level parent class semantics as the validity standard, retaining only the detail features within the main body of the clothing, effectively eliminating interference from background impurities and spurious activations, and providing a high-quality filtering feature foundation for subsequent cross-level residual enhancement.

[0031] This application further proposes that, based on the overlap rate index, the process of performing cross-level residual enhancement processing guided by the second feature map on the filtered first feature map to output a target discrimination feature vector includes: After completing the spatial filtering process, the first feature map after filtering is obtained. Only the detailed features of subclasses that conform to the semantic constraints of the parent class have been retained. However, relying solely on the mid-level features after spatial filtering is insufficient to fully distinguish fine-grained subclasses with high visual similarity. To address this, the present invention further introduces a cross-level residual enhancement mechanism, utilizing the overlap rate index calculated in step three. As a dynamic gain coefficient, it explicitly calculates and amplifies residual details that are unique to the subclass but not available in the parent class.

[0032] First, a dimensionality transformation is performed on the second feature map. This is because the second feature map has a limited number of channels (…). ) is usually more than the first feature map ( To align dimensions in subsequent feature operations, a 1×1 convolution or linear transformation matrix is ​​used to change the channel dimension of the second feature map from... Mapped to the same number of channels as the filtered first feature map The second feature map after dimension alignment is obtained. 1×1 convolution, without changing the spatial resolution of the feature map, achieves flexible dimensionality transformation by learning the linear combination relationship between channels, and is a commonly used technique in deep learning feature alignment.

[0033] Subsequently, the filtered first feature map The second feature map after dimension alignment Global average pooling is performed separately to compress the spatial dimension into a vector form. Global average pooling transforms the high-dimensional spatial feature map into a compact one-dimensional feature vector by calculating the mean of the feature map at all spatial coordinate positions, thereby aggregating global feature distribution information, reducing computational cost, and mitigating the risk of overfitting.

[0034] Finally, using the overlap rate index The pooled feature vectors are augmented with cross-level residuals to synthesize target discriminative feature vectors. The specific logic is as follows: based on the filtered detailed feature vector, add the overlap rate index... The dynamically scaled detail residual vector (i.e., the difference between the filtered detail feature vector and the high-level contour feature vector), where a preset detail gain coefficient is used. Control the overall magnitude of residual enhancement. Let This indicates a global average pooling operation. If the preset detail gain coefficient is used, then the target discrimination feature vector for: ; The causal logic of this formula has important physical significance: the difference term within the parentheses It characterizes the excess response of mid-level detail features relative to high-level contour features, i.e., fine-grained discriminative components unique to the subclass but not possessed by the parent class; overlap rate index As a dynamic amplification factor for the difference term, it is used when there is a high degree of overlap between the detail feature and the contour feature space (i.e., When the amplitude is relatively large, increasing the enhancement amplitude means that the detailed features have reliable spatial assignments, and their subclass residual information can be safely used as a discrimination criterion for enhancement; conversely, when the amplitude is relatively small, the enhancement amplitude is increased. When the value is small, it indicates that the relationship between the detailed features and the spatial classification of the contour region is weak. Therefore, the enhancement weight should be reduced to prevent noisy pseudo-details from interfering with the final classification.

[0035] Through this dynamic residual enhancement mechanism, the target discrimination feature vector effectively highlights the fine-grained discrimination components unique to the subclass while incorporating the parent class contour information constraints. This fundamentally improves the ability to distinguish across subclasses and completely changes the technical limitation of traditional equal-weight fusion schemes in being unable to establish cross-level causal constraints.

[0036] To illustrate with a specific numerical example: Based on ResNet-50 feature extraction, the filtered first feature map is subjected to global average pooling to obtain a 1024-dimensional feature vector. Similarly, the dimension-aligned second feature map is subjected to global average pooling to obtain a 1024-dimensional feature vector. Assuming the overlap rate index of the current batch is... Preset detail gain coefficient The residual enhancement is then 1.5 × 0.78 = 1.17 times; target discrimination feature vector The synthesis of the basic feature vector plus a 1.17 times subclass residual feature vector significantly amplifies the fine-grained differences unique to subclasses, effectively improving the discrimination resolution for highly similar subclasses.

[0037] The present invention further proposes that the process of obtaining a classification result of the clothing image based on the target discriminative feature vector, obtaining a feedback optimization parameter characterizing the confidence level of the classification result, and updating the adaptive threshold based on the parameter includes: The synthesized target discriminant feature vector The input is fed into a preset classifier. The preset classifier typically consists of a fully connected layer and a Softmax activation function, mapping the input feature vector to a probability distribution of the clothing image belonging to various preset fine-grained categories. Each value in the probability distribution is between 0 and 1, and the sum of the probabilities of all categories equals 1. The category index corresponding to the highest probability value in the probability distribution is taken as the classification result, outputting a fine-grained classification prediction for the clothing.

[0038] To achieve closed-loop feedback updates for adaptive thresholds, this invention designs feedback optimization parameters composed of a first feedback factor and a second feedback factor, which evaluate the current spatial filtering quality from two dimensions: classification confidence and mask coverage.

[0039] First feedback factor The confidence level of the classification result is represented by the difference between the highest and second-highest probability values ​​in the probability distribution. The first feedback factor reflects the certainty of the classification result: if... A large difference indicates a significant score gap between the best and second-best categories, high classification confidence, and good performance of the current spatial filtering and feature enhancement. A smaller threshold indicates that the scores of the two candidate categories are close, and the classification is in a state of ambiguity and uncertainty. This means that there may be a problem with the quality of the current features, and it is necessary to diagnose and adjust the adaptive threshold.

[0040] Second feedback factor The first factor characterizes the main coverage area of ​​the spatial mask matrix, and its value is the ratio of the number of pixels in the spatial mask matrix whose mask value is greater than a preset mask threshold to the total number of pixels in the spatial mask matrix. The second feedback factor reflects the proportion of the entire image that the spatial filter allows to pass through under the current adaptive threshold: if If the threshold is too small, it indicates that the current threshold is too high and the spatial mask is too strict, which may have incorrectly filtered out key details; if... An excessively large value indicates that the current threshold is too low, the spatial mask is too loose, and background noise may have seeped into the filtered details. The first feedback factor... With the second feedback factor The set of parameters is determined as the feedback optimization parameters.

[0041] This invention is based on the first feedback factor With the second feedback factor A three-stage dynamic update decision logic was designed, wherein the lower limit of the preset coverage range is denoted as... The upper limit is denoted as : Phase 1: Determining the First Feedback Factor Is it below the preset threshold? .like Not less than This indicates that the classification confidence is sufficient and the current adaptive threshold setting is reasonable. The classification results can be directly fed back to the relevant personnel without needing to go through the threshold adjustment process.

[0042] Phase Two: If Below the preset threshold This indicates that the classification is not sufficiently definitive and requires further assistance from a second feedback factor. Determine the adjustment direction. The lower limit of the preset coverage range and upper limit value The comparison is performed, and the following three-branch update logic is executed: If If the coverage is within the preset range, it means that the mask coverage is reasonable and the reason for the classification uncertainty is not due to the threshold setting. The current classification result should be directly fed back to the relevant personnel. like If the threshold is lower than the preset coverage range lower limit, it indicates that the current threshold is too high, the mask coverage area is too small, and key details may have been over-filtered. Based on the preset update step size, the current adaptive threshold is reduced to expand the allowed passage area for local details in the spatial mask matrix. If the threshold is higher than the upper limit of the preset coverage range, it indicates that the current threshold is too low, the mask coverage area is too large, and background noise has infiltrated the feature. Based on the preset update step size, the current adaptive threshold is increased to tighten the filtering boundary of the spatial mask matrix for background noise features.

[0043] The third stage: After adjusting the adaptive threshold, a new target discrimination feature vector is regenerated with the updated adaptive threshold, and the classification and confidence evaluation are performed again. This process is repeated until the confidence of the classification result meets the requirements, and then the final classification result is output.

[0044] Obtain the updated adaptive threshold The specific calculation method is as follows: based on the current adaptive threshold Based on this, plus a preset update step size factor Regulation, by the second feedback factor The midpoint of the preset coverage range The deviation drive, namely: ; The logical significance of this closed-loop update formula is that when the classification confidence is insufficient, the second feedback factor is used. The midpoint of the preset coverage range The direction of threshold adjustment is determined by the sign of the deviation. If... < (If the mask coverage area is too small), the update result is to lower the adaptive threshold to expand the mask coverage area; if > If the mask coverage area is too large, the update result is to increase the adaptive threshold to tighten the filtering boundary.

[0045] This two-factor, bidirectional closed-loop correction mechanism enables the classification system of this invention to automatically correct spatial filtering boundaries under different image input conditions, significantly improving the system's robustness in complex real-world scenarios. A specific numerical example illustrates this: assuming a pre-set confidence threshold... If the maximum probability value in the probability distribution of a certain classification output is 0.55 and the second maximum probability value is 0.42, then... This triggers the threshold update process; the spatial mask matrix is ​​statistically analyzed, and the percentage of pixels with a mask value greater than 0.5 is 0.15. The preset coverage range is [0.3, 0.7], and the median value is 0.5, confirming that the mask is too strict; [The text abruptly ends here, likely due to an incomplete sentence or missing information.] =0.5, then That is, the adaptive threshold is reduced by 0.01225, the mask boundary is widened, and then the spatial mask matrix is ​​regenerated based on the updated threshold and the subsequent classification process is executed. After iterating until the confidence level meets the requirements, the final classification result is output.

[0046] During the training phase, a fine-grained classification loss function is used to update the backpropagation parameters of the entire logical link end-to-end. By constraining the overlap rate index, the model is forced to prioritize the accuracy of the high-level class outlines when learning detailed features. This fundamentally solves the problem that minority class features are easily interfered with by the majority class under long-tailed distributions, effectively achieving substantial progress superior to existing technologies.

[0047] In another implementation, such as Figure 2 As shown, this application also provides a fine-grained classification system for unbalanced clothing that incorporates attention enhancement, including the following modules: The image acquisition module is used to acquire the image of the clothing to be identified and extract a first feature map representing its local detail features and a second feature map representing its global contour features. The first processing module is used to perform saliency compression processing on the first feature map and the second feature map respectively to obtain a first spatial response map and a second spatial response map, and to perform spatial scale alignment processing on the two. The image evaluation module is used to obtain an overlap rate index that characterizes the spatial attribution relationship between the local detail region and the global contour region of the clothing image based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map. The second processing module is used to generate an adaptive threshold for extracting the activation region of the clothing image based on the pixel activation statistics of the second spatial response map, and generate a spatial mask matrix of its activation region, and perform spatial filtering processing on the first feature map based on the spatial mask matrix to obtain the filtered first feature map. The image classification module is used to perform cross-level residual enhancement processing based on the second feature map on the filtered first feature map based on the overlap rate index, so as to output a target discrimination feature vector, obtain the classification result of the clothing image according to the target discrimination feature vector, obtain feedback optimization parameters characterizing the confidence level of the classification result, and update the adaptive threshold according to them.

[0048] The aforementioned modules work together to form a complete attention-enhanced imbalanced clothing fine-grained classification system, achieving fully automated processing from clothing image input to fine-grained classification result output. The core innovation of this application lies in organically embedding the legality verification capability of high-level parent class semantics into the activation process of mid-level detailed features through a cross-level semantic causal constraint link (first feature map → spatial mask filtering → cross-level residual enhancement → closed-loop threshold feedback). This breaks through the technical limitations of traditional equal-weighted fusion schemes, achieving the dual goals of accurate parent class recognition and reliable child class discrimination on imbalanced clothing datasets, demonstrating significant theoretical innovation and engineering application value.

[0049] In another embodiment, this application also provides a computer storage medium storing computer-executable instructions that, when executed, implement the aforementioned attention-enhanced fine-grained classification method for unbalanced clothing.

[0050] The computer storage medium can be a read-only memory (ROM), random access memory (RAM), disk, or optical disk, etc. When the computer-executable instructions stored thereon are called and executed by the processor, the processing logic of each of the above steps can be fully implemented, including feature extraction, channel compression, scale alignment, overlap rate index calculation, adaptive threshold determination, spatial mask matrix generation, spatial filtering processing, cross-level residual enhancement, target discrimination feature vector output, classification result output, and adaptive threshold closed-loop feedback update. This effectively ensures the portability and deployability of the method of the present invention on various computing platforms.

[0051] The above embodiments are only used to illustrate the technical methods of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of this application without departing from the spirit and scope of the technical methods of this application.

Claims

1. A fusion attention enhanced unbalanced garment fine-grained classification method, characterized in that, Includes the following steps: Acquire the image of the clothing to be identified, and extract the first feature map representing its local detail features and the second feature map representing its global contour features; The first feature map and the second feature map are subjected to saliency compression processing of the channel dimension to obtain the first spatial response map and the second spatial response map, and spatial scale alignment processing is performed on the two. Based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map, the overlap rate index, which characterizes the spatial attribution relationship between the local detail region and the global contour region of the clothing image, is obtained. An adaptive threshold for extracting the activation region of the clothing image is generated based on the pixel activation statistics of the second spatial response map, and a spatial mask matrix of the activation region is generated. Spatial filtering is performed on the first feature map based on the spatial mask matrix to obtain the filtered first feature map. Based on the overlap rate index, cross-level residual enhancement processing guided by the second feature map is performed on the filtered first feature map to output the target discrimination feature vector; The classification result of the clothing image is obtained based on the target discrimination feature vector, a feedback optimization parameter characterizing the confidence level of the classification result is obtained, and the adaptive threshold is updated based on it; The process of extracting the first feature map and the second feature map includes: The clothing image is input into a preset feature extraction network. Based on the feature extraction network, feature maps with a first spatial resolution and a first number of channels are extracted and used as first feature maps, and feature maps with a second spatial resolution and a second number of channels are extracted and used as second feature maps. The first spatial resolution is higher than the second spatial resolution, and the first number of channels is less than the second number of channels. The process of performing spatial scale alignment on the first and second spatial response maps includes: After performing saliency compression, the activation intensity sets of all channels corresponding to each pixel point of the first feature map and the second feature map are obtained respectively. The maximum activation intensity in each activation intensity set is recorded as the activation intensity value of the corresponding pixel point in the corresponding spatial response map. The second spatial response map is upsampled using a bilinear interpolation algorithm to make its spatial dimension consistent with that of the first spatial response map. The pixel coordinates of the upsampled second spatial response map are then calibrated according to the downsampling step size of the preset feature extraction network to obtain the aligned second spatial response map. The process of obtaining the overlap ratio index includes: pixel-wise multiplying the aligned second spatial response map with the first spatial response map to obtain an interaction matrix representing the hierarchical commonality region , the overlap rate index wherein, represents the aligned second spatial response map, represents the first spatial response map, represents summing operation on the activation intensity values of all the pixel points in the matrix .

2. The fusion attention enhanced unbalanced garment fine-grained classification method according to claim 1, characterized in that, The process of generating a spatial mask matrix includes: The average of the activation intensity values of all pixel points in the aligned second spatial response map is taken as an adaptive threshold value thereof In the aligned second spatial response map, a coordinate set with an activation intensity value greater than the adaptive threshold value is taken as an activation region, a coordinate set with an activation intensity value less than or equal to the adaptive threshold value is taken as a background region, and a pixel point in the activation region is given a weight value higher than that of a pixel point in the background region. obtaining the spatial mask matrix a mask value corresponding to a pixel point at a coordinate (i, j) wherein, denotes an activation intensity value of the aligned second spatial response map at the coordinate (i, j), is a preset smoothing control coefficient.

3. The fusion attention enhanced unbalanced garment fine-grained classification method according to claim 2, characterized in that, The process of obtaining the filtered first feature map includes: The spatial mask matrix is expanded along the channel dimension to make its dimension consistent with the first feature map before performing the spatial filtering process , obtaining the filtered first feature map wherein, denotes an element-wise multiplication operation.

4. The fusion attention enhanced unbalanced garment fine-grained classification method according to claim 2, characterized in that, The process of outputting the target discriminant feature vector and obtaining the classification result based on it includes: performing a dimension transformation process on the second feature map to make the number of channels thereof consistent with the number of channels of the filtered first feature map , to obtain a second feature map after dimension alignment , the target discrimination feature vector , represents performing a global average pooling operation on a matrix , is a preset detail gain coefficient; The target discrimination feature vector is input into a preset classifier to output the probability distribution of the clothing image belonging to each preset fine-grained category, and the category index corresponding to the largest probability value is taken as the classification result.

5. The fine-grained classification method for unbalanced clothing with attention enhancement as described in claim 4, characterized in that, The process of updating the adaptive threshold includes: The difference between the probability value with the largest value and the probability value with the second largest value in the probability distribution is obtained and used as the first feedback factor. The number of pixels in the spatial mask matrix whose mask value is greater than a preset mask threshold is counted, and the ratio of this number to the total number of pixels in the spatial mask matrix is ​​used as a second feedback factor. ; The feedback optimization parameters include a first feedback factor and a second feedback factor. The first feedback factor is determined. Is it below the preset threshold? If not, the classification result will be fed back to the relevant personnel; if yes, it will be based on the second feedback factor. With preset coverage range For deviations between the states, execute the following update logic: like The classification results will then be fed back to the relevant personnel. or Then based on the preset update step size Update the current adaptive threshold to obtain the updated adaptive threshold. ; in, For the current adaptive threshold, The median value of the preset coverage range. The new target discrimination feature vector is repeatedly obtained based on the updated adaptive threshold until the corresponding classification result is fed back to the relevant personnel.

6. A fine-grained classification system for unbalanced clothing that incorporates attention enhancement, characterized in that: Includes the following modules: The image acquisition module is used to acquire the image of the clothing to be identified and extract a first feature map representing its local detail features and a second feature map representing its global contour features. The first processing module is used to perform saliency compression processing on the first feature map and the second feature map respectively to obtain a first spatial response map and a second spatial response map, and to perform spatial scale alignment processing on the two. The image evaluation module is used to obtain an overlap rate index that characterizes the spatial attribution relationship between the local detail region and the global contour region of the clothing image based on the pixel activation intensity distribution of the aligned second spatial response map and the first spatial response map. The second processing module is used to generate an adaptive threshold for extracting the activation region of the clothing image based on the pixel activation statistics of the second spatial response map, and generate a spatial mask matrix of its activation region, and perform spatial filtering processing on the first feature map based on the spatial mask matrix to obtain the filtered first feature map. The image classification module is used to perform cross-level residual enhancement processing based on the second feature map on the filtered first feature map based on the overlap rate index, so as to output a target discrimination feature vector, obtain the classification result of the clothing image according to the target discrimination feature vector, obtain feedback optimization parameters characterizing the confidence level of the classification result, and update the adaptive threshold according to them. The process of extracting the first feature map and the second feature map includes: The clothing image is input into a preset feature extraction network. Based on the feature extraction network, feature maps with a first spatial resolution and a first number of channels are extracted and used as first feature maps, and feature maps with a second spatial resolution and a second number of channels are extracted and used as second feature maps. The first spatial resolution is higher than the second spatial resolution, and the first number of channels is less than the second number of channels. The process of performing spatial scale alignment on the first and second spatial response maps includes: After performing saliency compression, the activation intensity sets of all channels corresponding to each pixel point of the first feature map and the second feature map are obtained respectively. The maximum activation intensity in each activation intensity set is recorded as the activation intensity value of the corresponding pixel point in the corresponding spatial response map. The second spatial response map is upsampled using a bilinear interpolation algorithm to make its spatial dimension consistent with that of the first spatial response map. The pixel coordinates of the upsampled second spatial response map are then calibrated according to the downsampling step size of the preset feature extraction network to obtain the aligned second spatial response map. The process of obtaining the overlap ratio index includes: The aligned second spatial response map is multiplied pixel-by-pixel with the first spatial response map to obtain the interaction matrix representing the common regions of the hierarchy. The overlap rate index ,in, This represents the aligned second spatial response map. This represents the first spatial response diagram. Represents a matrix The activation intensity values ​​of all pixels within the range are summed.

7. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement the fine-grained classification method for unbalanced clothing with fusion attention enhancement as described in any one of claims 1-5.

Images