A classification method and system

Abstract

The application relates to a classification method and system, which comprises generating positive sample pairs according to image information of an article; extracting enhanced features of the positive sample pairs and storing the enhanced features into a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score; screening negative sample pairs from the feature queue according to the enhanced features and the corresponding time decay weights and confidence scores; constructing a multiple contrast loss based on the enhanced features and the negative sample pairs, training a model based on the multiple contrast loss, and obtaining a classification model to classify the article. The system comprises a data acquisition module, an edge computing module and a training server, which are respectively used for executing steps of the corresponding method. Thus, under the premise of greatly reducing or even not needing manual fine-grained labeling, the discrimination ability for local subtle differences of the article is improved, the instability caused by negative sample noise and old features is inhibited, the model training period is shortened, and the classification precision is improved.

Description

Technical Field

[0001] This application relates primarily to the field of computer vision, and in particular to a classification method and system. Background Technology

[0002] In service robot visual recognition scenarios, the demand for refined classification of items in everyday life scenarios such as homes and healthcare is becoming increasingly urgent. Accurate fine-grained classification is the foundation for realizing core functions such as autonomous item retrieval and environmental adaptation services.

[0003] However, current fine-grained classification technology for service robots still faces numerous bottlenecks. First, the annotation cost is extremely high. Fine-grained classification requires manual annotation of millimeter-level differences (such as label details and defects) pixel by pixel. A dataset of 100,000 images requires multiple annotators working for weeks, with labor costs increasing exponentially with the category. Second, classification accuracy is insufficient. Traditional CNNs rely on global pooling, which easily overlooks key local regions, making them insensitive to brand logos, minor scratches, and other key local areas, failing to capture fine-grained differences and resulting in blurred discrimination boundaries. Third, training convergence is slow. Randomly selected negative samples contain a large number of easily confused samples, misleading the gradient direction. Furthermore, outdated features in the FIFO queue are not weighted, leading to training oscillations and distribution drift. Training a dataset of 1000 categories often takes more than 5 days. These problems are interconnected, hindering the practical application of fine-grained classification in service robots. Summary of the Invention

[0004] One objective of this application is to provide a classification method and system that addresses the problems of high annotation costs, low classification accuracy, and slow training convergence in the prior art.

[0005] According to one aspect of this application, a classification method is provided, the method comprising: generating positive sample pairs based on image information of an item; extracting enhancement features of the positive sample pairs and storing them in a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score; selecting negative sample pairs from the feature queue based on the enhancement features and the corresponding time decay weight and confidence score; constructing a multiple contrast loss based on the enhancement features and the negative sample pairs; training a model based on the multiple contrast loss to obtain a classification model, the classification model being used to classify the item.

[0006] Optionally, the step of extracting the enhanced features of the positive sample pairs includes: inputting the positive sample pairs into a query encoder and a momentum encoder respectively; extracting the query global features and query local features of the positive sample pairs through the query encoder, and extracting the momentum global features and momentum local features of the positive sample pairs through the momentum encoder; determining the local attention weights of the query local features and the momentum local features respectively, performing weighted fusion with the query global features and the momentum global features respectively, and superimposing a preset perturbation to obtain query enhanced features and momentum enhanced features; and obtaining enhanced features based on the query enhanced features and the momentum enhanced features.

[0007] Optionally, the step of selecting negative sample pairs from the feature queue based on the enhanced features and the corresponding time decay weights and confidence scores includes: obtaining the current class label of the model; selecting candidate negative samples from the feature queue based on the current class label; obtaining a query sample based on the enhanced features; determining the cosine similarity between the candidate negative sample and the query sample based on the query sample, the candidate negative sample, the time decay weights and confidence scores corresponding to the candidate negative sample; and determining negative sample pairs from the candidate negative samples based on the current class label, the cosine similarity, and the confidence scores.

[0008] Optionally, the enhanced features include local features of different visual or local regions of the same item, and the construction of multiple contrast loss based on the enhanced features and the negative sample pairs includes: determining a global loss based on the enhanced features and the negative sample pairs; determining a local alignment loss based on different local features in the enhanced features; determining a label consistency loss based on the current category label of the model, the enhanced features, and the negative sample pairs; and determining the multiple contrast loss based on the global loss, the local alignment loss, and the label consistency loss.

[0009] Optionally, determining the multiple contrast loss based on the global loss, the local alignment loss, and the label consistency loss includes: dynamically adjusting the weights corresponding to each loss based on the time decay weight and the confidence score; and determining the multiple contrast loss based on the global loss, the local alignment loss, the label consistency loss, and the adjusted weights.

[0010] Optionally, the step of training the model based on the multiple contrastive loss to obtain a classification model includes: updating the time decay weights, the confidence score, and the parameters of the enhanced momentum encoder through backpropagation based on the multiple contrastive loss to complete one round of model training, wherein the enhanced momentum encoder is used to extract the enhanced features of the positive sample pairs; repeating the iterative training process until the model converges to obtain a classification model.

[0011] Optionally, the method further includes: feeding back the classification result of the item output by the classification model to the robot, so that the robot plans a robotic arm path for the target item according to the classification result, and grasps the target item according to the robotic arm path.

[0012] Optionally, the method further includes: mixing a preset proportion of coarse-grained label images into the image information and sampling them to obtain a data subset; inputting the data subset into the enhanced momentum encoder for processing and outputting a prediction probability; determining a cross-entropy loss based on the coarse-grained labels and the prediction probability, and determining a semi-supervised loss based on the cross-entropy loss and the multiple contrast loss; training the model based on the semi-supervised loss to obtain an optimized classification model, wherein the optimized classification model is used to classify the items.

[0013] According to another aspect of this application, a classification system is also provided, comprising: a data acquisition module, an edge computing module, and a training server; the data acquisition module is used to generate positive sample pairs based on image information of the items; the edge computing module is used to extract enhancement features of the positive sample pairs and store them in a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score; the training server is used to filter negative sample pairs from the feature queue based on the enhancement features and the corresponding time decay weights and confidence scores; the training server is also used to construct a multiple contrast loss based on the enhancement features and the negative sample pairs, and to train a model based on the multiple contrast loss to obtain a classification model, the classification model being used to classify the items.

[0014] According to another aspect of this application, a computer-readable medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the above methods.

[0015] Compared with existing technologies, this application generates positive sample pairs based on the image information of the items; extracts the enhancement features of the positive sample pairs and stores them in a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score; filters negative sample pairs from the feature queue based on the enhancement features and the corresponding time decay weights and confidence scores; constructs a multiple contrast loss based on the enhancement features and the negative sample pairs; trains the model based on the multiple contrast loss to obtain a classification model, which is used to classify the items. Thus, while significantly reducing or even eliminating the need for manual fine-grained annotation, it improves the model's ability to discriminate subtle local differences in items, suppresses instability caused by negative sample noise and outdated features, shortens the model training cycle, and improves classification accuracy. Attached Figure Description

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings, wherein:

[0017] Figure 1 A flowchart illustrating a classification method provided according to one aspect of this application is shown;

[0018] Figure 2 This diagram illustrates the implementation steps of the enhanced momentum contrast learning method in one embodiment of this application.

[0019] Figure 3 This diagram illustrates the architecture of enhanced momentum contrastive learning in one embodiment of this application.

[0020] Figure 4 A schematic diagram of the framework of a classification system provided according to another aspect of this application is shown.

[0021] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein, and therefore this application is not limited to the specific embodiments disclosed below.

[0024] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0025] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0026] (1) Adaptive Local-attention Momentum Encoder (ALME): A component integrated into the backbone network of an object image recognition system. It embeds a learnable local attention subnetwork in the momentum encoder to dynamically assign differentiated weights to local salient regions in the input image, such as brand logos, texture details, and label information, thereby enhancing the capture and representation of fine-grained features without additional manual annotation.

[0027] (2) Dynamic Hard Negative Mining (DHNM): An online negative sample screening mechanism that calculates the cosine similarity between each negative sample in the queue and the current query sample in real time; verifies the class consistency of negative samples based on the class pseudo-labels or periodic clustering results output by the current model; and dynamically extracts the most easily confused negative samples from the FIFO feature queue according to the preset similarity threshold and class inconsistency conditions for comparative learning, so as to strengthen the inter-class boundaries and improve the discrimination accuracy.

[0028] (3) Triple Contrastive Loss (TCL): The objective function of multi-task contrastive learning is composed of a weighted combination of global InfoNCE loss, local patch alignment loss, and cluster pseudo-label consistency loss. Among them, the global InfoNCE loss represents maximizing the mutual information of positive sample pairs and minimizing the mutual information of negative sample pairs along the batch or queue dimension; the local patch alignment loss represents constraining the feature consistency of the same item between different perspectives or different local regions of the same perspective based on the local features extracted by the adaptive local attention module; the cluster pseudo-label consistency loss represents using the category pseudo-labels or periodic cluster centers generated by DHNM to force samples with the same pseudo-label to cluster in the feature space and separate them from samples with different pseudo-labels, so as to maintain the separability of the category structure under the condition of no fine manual annotation.

[0029] (4) Enhanced FIFO Feature Queue (EFFQ): A data structure used to store historical sample features. It introduces a time decay weight factor on the basis of the traditional FIFO queue, so that the weight of newly enqueued features is higher, and the weight of earlier enqueued features decays exponentially or linearly over time. The decayed weights directly participate in the subsequent comparison loss calculation, suppressing the negative impact of old information on model convergence and improving training stability and convergence speed.

[0030] Existing technologies for service robots to perform fine-grained classification of everyday items in scenarios such as homes, living spaces, and healthcare typically rely on the following technical details:

[0031] (1) Regarding data collection and manual annotation, existing technologies are typically deployed on home robots or smart cameras to collect images of items in the home environment (such as medicines, food, and daily necessities). After collection, each image needs to be manually annotated at the pixel level or instance level. The annotation dimensions should include at least the item category (such as medicine name, food brand, and daily necessities model), production batch or expiration date, appearance defects (such as damage, stains, and blurred labels), and other attributes (such as applicable population and storage conditions). In a typical home service scenario, a dataset containing 1,000 categories and 100,000 images requires multiple annotators to work for several weeks, resulting in high labor costs.

[0032] (2) In terms of network structure, mainstream solutions use ImageNet pre-trained ResNet50, EfficientNet-B3 / B4, or RegNet as the backbone network. After global average pooling, a fully connected layer is connected to output logits, or directly connected to additive angular margin loss layers such as ArcFace, AM-Softmax, and CosFace; the input resolution is 224×224 or 384×384, and the number of channels is 3; the number of parameters in the fully connected layer is ≈2×C×d, where C is the number of classes and d is the feature dimension (usually 512 or 1024). The model as a whole adopts an end-to-end supervised learning paradigm and does not distinguish between global features and local features.

[0033] (3) In the training strategy, the loss function adopts cross-entropy loss, additive angular margin loss (ArcFace, m=0.5, s=30), or focal loss (γ=2) to alleviate class imbalance. Regarding the dataset size, the training set has ≥100k images, the validation set has ≥10k images, and the number of classes is ≥1000. Regarding the training cycle, 50~100 epochs are used, with a batch size of 256, requiring 3~5 days using 4×V100 GPUs. For class balancing, oversampling, cost-sensitive re-weighting, or logit adjustment are used for long-tail classes.

[0034] (4) Some solutions employ contrastive learning reinforcement, while some advanced solutions perform unsupervised or self-supervised contrastive pre-training before supervised fine-tuning: the framework uses MoCo-v2 or SimCLR; the feature queue uses a FIFO structure with a length K=65536; the loss uses InfoNCE with a temperature coefficient τ=0.07; pre-training for 200 epochs, followed by fine-tuning for 30 epochs. However, this step is only used to improve the initialization, and subsequent supervised fine-tuning still relies on manual labeling.

[0035] (5) In the contrastive learning stage, negative samples are constructed by randomly drawing samples from batches or FIFO queues without introducing any “difficulty” or “class consistency” filtering conditions. This results in a large number of easily confused negative samples (with cosine similarity > 0.7 to the query sample but different classes) participating in gradient calculation.

[0036] (6) In the feature queue maintenance mechanism, the existing FIFO queue maintains historical features in a strict first-in-first-out manner. The enqueued features are accompanied by a timestamp, but the weight is constant at 1; there is no decay or elimination strategy; old features (from early weights or low-quality images) have the same weight as the latest features in InfoNCE, which can easily introduce distribution drift noise.

[0037] Based on the details of the existing solutions described above, the problems existing in the current technology are as follows:

[0038] (1) Data annotation is expensive. Fine-grained differences (such as millimeter-level labels, textures, and defects) require high-resolution images and manual annotation of each pixel / instance. The human resource input increases exponentially with the category and defect dimension.

[0039] (2) The model training converges slowly, the differences between fine-grained categories are weak, and the network needs a lot of iterations to converge; in addition, the data scale is huge, and the overall training cycle is more than 5 days, which seriously slows down the iteration speed.

[0040] (3) Insufficient feature representation capability: Traditional CNN global pooling layers are not sensitive to local key regions (such as a 10×10 pixel brand logo or a 1mm scratch), resulting in limited ability to represent subtle differences.

[0041] (4) Negative samples are sensitive to noise. Randomly selected negative samples contain a large number of “confusing” samples (high similarity but different categories), which mislead the gradient direction, reduce the clarity of the discrimination boundary, and cause training oscillation.

[0042] (5) The feature queue is unstable. The FIFO queue does not weight or eliminate old features. Outdated or low-quality features continue to participate in the comparison loss calculation, resulting in training drift and unstable convergence.

[0043] To address the aforementioned issues, the enhanced momentum contrastive learning method proposed in this application can significantly reduce or even completely eliminate manual fine-grained annotation while simultaneously improving the model's ability to discriminate subtle local differences in items, accelerating training convergence, and suppressing instability caused by negative sample noise and outdated features.

[0044] Figure 1 A flowchart illustrating a classification method according to one aspect of this application is shown, the method comprising steps S11 to S14.

[0045] Step S11: Generate positive sample pairs based on the image information of the items.

[0046] Image information of items is acquired through image acquisition. In home robot service scenarios, the original images of the items to be classified can be acquired by the vision module or independent camera mounted on the home robot, ensuring that the images clearly present the appearance features of the items (such as shape, texture, and markings). For the acquired original images, positive sample pairs are generated through data augmentation operations. A positive sample pair is a different augmented view of the same item, such as two views (global and local).

[0047] Data augmentation operations can include random cropping, color jittering, and light geometric perturbation. Random cropping involves cropping the image at random size and location within the effective area of ​​the original image, preserving the core features of the object while introducing local viewpoint differences. Color jittering involves making small random adjustments to color parameters such as brightness, contrast, and saturation of the image to simulate the appearance of the object under different lighting conditions. Light geometric perturbation involves implementing slight geometric transformations such as rotation, translation, or scaling to avoid the model relying too much on the fixed posture of the object.

[0048] This allows two images obtained by performing different enhancement processes on the same original object image to be combined into a pair of positive samples. Although there are subtle visual differences between the two, they are essentially of the same type of object. This can effectively help the model learn the feature invariance of similar objects and improve the generalization ability of subsequent fine-grained classification.

[0049] Step S12: Extract the enhanced features of the positive sample pairs and store them in the feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score.

[0050] For each pair of positive samples (two images of the same item after enhancement) generated in step S11, feature extraction is performed on the two sample images to obtain a high-dimensional vector that can represent the core features of the item, i.e., the enhanced features.

[0051] Each extracted enhancement feature is labeled with a time decay weight and then weighted by confidence. The time decay weight measures the timeliness of the feature; newly extracted features are given higher weights, and as new features are continuously added to the queue, the weights of older features gradually decay exponentially or linearly over time, thereby reducing the interference of stale features on training. Confidence weighting involves attaching a confidence score based on prediction probability or feature stability to each feature, filtering out invalid features with too low confidence, thus suppressing stale noise.

[0052] The enhanced features, carrying time decay weights and confidence scores, are stored in a pre-defined feature queue in an orderly manner to form a structured feature dataset. Compared with traditional feature storage methods without attribute labels, the dual attributes carried by this feature queue can help the subsequent training process to accurately select effective features and optimize the gradient descent direction, thereby alleviating the problems of slow training convergence and distribution drift.

[0053] In practical applications, the feature queue can be an enhanced FIFO feature queue (EFFQ), which applies exponential time decay to the queue features. And attached based on The confidence score was improved, with high-confidence samples receiving a 1.5-fold increase in weight. Compared to the traditional FIFO approach, the effective weight of outdated features decreased by more than 50%, and the training curve volatility decreased by 40%. The high-confidence priority strategy increased the proportion of effective samples from 68% to 91%, and reduced the overall convergence epoch from 80 to 25.

[0054] Step S13: Select negative sample pairs from the feature queue based on the enhanced features, the corresponding time decay weights, and confidence scores.

[0055] Negative sample pairs must belong to a different category of items than positive sample pairs, and include different views of the same dissimilar item. The selection criteria for negative sample pairs can be the degree of difference in enhanced features; for example, the cosine similarity of their enhanced feature vectors must be below a preset threshold to ensure significant feature differences. The time decay weight and confidence score marked in step S12 can be used as auxiliary selection conditions. For example, enhanced features with high time decay weights and high confidence scores can be prioritized for negative sample matching to avoid outdated and invalid features interfering with the selection results.

[0056] For each positive sample pair, the enhanced features are retrieved and matched in the feature queue, meeting the above criteria and matching conditions. For example, if a feature belongs to a different category than the current positive sample feature, its feature similarity is below a preset threshold, and its time decay weight and confidence score meet the set requirements, then this feature can be paired with the current positive sample feature to form a negative sample pair. This eliminates "easily confused negative samples" (such as similar features of similar items, invalid features with low confidence, and outdated features) commonly found in traditional random screening, ensuring that the final selected negative sample pairs have clear category discrimination.

[0057] Step S14: Construct a multiple contrast loss based on the enhanced features and the negative sample pairs, train the model based on the multiple contrast loss to obtain a classification model, which is used to classify the items.

[0058] The role of contrastive loss is to constrain the distribution of sample features, making features of similar samples more similar (intra-class aggregation) and features of dissimilar samples more distant (inter-class separation). Multiple contrastive losses can form synergistic constraints from multiple dimensions, rather than a single-dimensional loss function. Based on the acquired enhanced features and selected negative sample pairs, the aggregation and separation relationships of similar and dissimilar features can be constrained by combining global sample features and queue feature distribution. For the differences and consistency requirements of local features of items, corresponding loss constraints can be added, enabling the model to capture fine-grained discriminative information. To adapt to the training requirements of low labeling costs, losses can be constructed by combining pseudo-labels or clustering-related information to ensure the separability of class structures. By combining multi-dimensional losses, the loss function can be made more suitable for the needs of fine-grained classification, while also adapting to different data labeling scenarios.

[0059] During model training, enhanced object images are used as input, and object category information is used as supervision. Backpropagation with multiple contrastive loss optimizes network parameters. The training process fully utilizes selected high-quality negative sample pairs to avoid interference from invalid samples and accelerate model convergence. The final trained classification model can be deployed on service robots, directly receiving object images captured by the robot or its camera. After preprocessing, the model performs classification, providing visual recognition support for autonomous services in scenarios such as homes and elderly care facilities.

[0060] exist Figure 2The illustrated embodiment demonstrates the complete implementation steps of the enhanced momentum contrastive learning method. Before starting model training, dataset preparation is required to provide high-quality image data that meets the task requirements for subsequent model training. For the training needs of unsupervised models, the dataset is collected and constructed through a training server; for example, 300,000 unlabeled object images are collected and standardized to form an unlabeled dataset. "Unlabeled" is used to eliminate the interference of manual annotation on unsupervised training, allowing the model to autonomously learn and discover feature relationships between images.

[0061] After the dataset is prepared, the basic model architecture needs to be built for subsequent training, and the parameters of key components need to be initialized. The structure and parameters of the self-supervised training query encoder, momentum encoder, and enhanced FIFO queue are initialized respectively.

[0062] The query encoder employs a combination of ResNet50 and a lightweight local attention module. ResNet50 is a classic feature extraction backbone network in computer vision, possessing hierarchical feature capture capabilities and efficiently extracting general image features. The lightweight local attention module adapts to the local feature focusing requirements of subsequent SALAME processing; by embedding this module in advance within the encoder, the structure can be avoided during training. The parameters of the query encoder are randomly initialized to ensure no prior bias, allowing the model to autonomously learn features based on the dataset.

[0063] The momentum encoder can employ an Adaptive Local-attention Momentum Encoder (ALME). ALME is a key component integrated into the backbone network of an object image recognition system. By embedding a learnable local attention subnetwork into the momentum encoder, it dynamically assigns differentiated weights to local salient regions of the input image (such as brand logos, texture details, and label information), thereby enhancing the model's ability to capture and represent fine-grained features without additional manual annotation.

[0064] In self-supervised contrastive learning, the momentum encoder parameters serve as a stable source of feature reference. Instead of performing independent initialization, it is used in conjunction with the query encoder parameters. Binding is achieved through a momentum update mechanism, as shown in the following formula:

[0065]

[0066] in, To query the encoder's real-time parameters; These are the parameters of the momentum encoder; The momentum coefficient can take values ​​of... =0.996, thus keeping the momentum encoder parameters relatively stable, while slowly absorbing the updated information from the query encoder to avoid excessive feature fluctuations.

[0067] Enhanced FIFO Feature Queues (EFFQ) are data structures used to store historical sample features, and can be used to store negative sample features in self-supervised training. Building upon the traditional "first-in, first-out" (FIFO) principle, a time decay weight factor is introduced, giving higher weights to recently enqueued features and exponentially or linearly decaying the weights of earlier enqueued features over time. These decayed weights directly participate in subsequent contrastive loss calculations, suppressing the negative impact of stale information on model convergence and improving training stability and convergence speed.

[0068] The parameters of the enhanced FIFO queue can be set as follows: queue length K = 65536, ensuring that the queue can store a sufficient number of negative samples; the more negative samples, the stronger the feature discrimination; time decay coefficient. =0.999, to decay old features stored in the queue in the early stage, reduce the interference of outdated features on training; the confidence threshold is set to 0.7 to filter low confidence features, allowing only high-quality features to enter the queue, thereby improving the quality of negative samples.

[0069] After completing the structural construction and parameter configuration of the query encoder, ALME momentum encoder and enhanced FIFO queue, the model architecture is fully adapted to the subsequent training requirements. The queue has the ability to store and filter features, and the whole model has entered the "trainable state", which can be directly connected to subsequent training processes such as image sampling and feature extraction.

[0070] After the core components of the model have completed their structure building and parameter initialization, the training iteration is triggered. The total number of epochs can be set to 100. An epoch represents the training cycle that traverses the entire dataset once. Taking a dataset of 300,000 images as an example, it is necessary to traverse all 300,000 images 100 times. In actual training, this can be fine-tuned according to the model's convergence.

[0071] In each round of training, a batch of image data is first obtained from the prepared initial dataset D0. A batch size of 256 can be randomly selected from D0. Random selection breaks the original order of the dataset, preventing the model from learning irrelevant patterns in the data arrangement; batch size is the batch size, and a batch size of 256 is suitable for the memory configuration of mainstream training servers. This yields the original image tensor { Tensors are the standard input type for deep learning frameworks. } represents 256 images in the batch (i ranges from 1 to 256), providing high-quality raw data input for each round of training.

[0072] For each original item image Two independent strong enhancement strategies were applied respectively. , Finally, positive sample pairs are generated. The specific operations of the strong enhancement strategy can include random cropping, color dithering, Gaussian blurring, and random erasing. For example, cropping to a size of 224×224 pixels ensures that the image specifications of the input model are uniform; and setting the dithering intensity. =0.4, by adjusting parameters such as image brightness, contrast, and saturation, the model's adaptability to changes in lighting is enhanced; a Gaussian kernel of kernel=3 (3×3 pixels) is used for blurring to simulate the blurring of objects in real-world scenes; the erasure probability p=0.25 randomly occludes local areas of the image, forcing the model to focus on the overall features of the object rather than local irrelevant areas.

[0073] In one embodiment of this application, in step S12, the positive sample pairs are input into the query encoder and the momentum encoder respectively; the query encoder extracts the global query features and local query features of the positive sample pairs, and the momentum encoder extracts the global momentum features and local momentum features of the positive sample pairs; the local attention weights of the local query features and the local momentum features are determined respectively, and the local attention weights are weighted and fused with the global query features and the global momentum features respectively, and a preset perturbation is superimposed to obtain query enhancement features and momentum enhancement features; enhancement features are obtained based on the query enhancement features and the momentum enhancement features.

[0074] An enhanced momentum encoder can consist of a query encoder and a momentum encoder. By working together, the two encoders can extract features from positive sample pairs and perform attention-weighted fusion to obtain enhanced features.

[0075] Specifically, the query encoder and momentum encoder are parallel inputs. The generated positive sample pairs (two samples of the same item obtained through data augmentation) are input into the query encoder and momentum encoder respectively. Parallel processing allows for the simultaneous acquisition of two sets of features. The query encoder and momentum encoder simultaneously capture global and local features, respectively. Global features characterize the overall appearance and outline of the item, while local features capture detailed information about the item's regions (such as millimeter-level key difference regions required for fine-grained classification). In other words, the query encoder obtains global and local query features, while the momentum encoder obtains global and local momentum features.

[0076] Next, the local attention weights for query local features and momentum local features are calculated separately. These weights adaptively distinguish the importance of local features, assigning higher weights to key detail regions and lower weights to irrelevant regions, thereby highlighting the role of key local features in fine-grained classification.

[0077] Subsequently, the obtained local attention weights are weighted and fused with the corresponding global query features and global momentum features, respectively, so that the fused features retain both global macroscopic information and enhance local key details. Next, a pre-defined controllable perturbation is superimposed to further improve the robustness of the features, ultimately yielding query-enhanced features and momentum-enhanced features. Through collaborative processing of the query-enhanced features and momentum-enhanced features (such as feature alignment and fusion verification), enhanced features are obtained and used in subsequent feature queue storage, negative sample screening, and loss calculation processes.

[0078] Continue to refer to Figure 2 The illustrated embodiment can employ an adaptive local attention momentum encoder (SALAME) as an enhanced momentum encoder. For positive sample pairs... Input them into the query encoder and momentum encoder respectively, that is... Encoder inquiry Process and output global query features and query local features ;Will Momentum encoder Process and output global momentum features and momentum local features Thus, query features are obtained. and momentum characteristics .

[0079] Next, SALAME performs the specific enhancement processing. First, it calculates the local attention weights. The calculation method is as follows: CAM (Class Activation Mapping) generates feature activation maps based on the Adaptive Local Attention Module (ALME), specifically by analyzing the query's local features. Global average pooling or maximum projection is performed along the channel dimension to obtain a weight matrix reflecting the importance of image spatial regions, thereby locating fine-grained, high-discrimination regions such as brand logos and textures. Gaussian noise is then generated. Through formula Complete feature fusion and enhancement to obtain query enhancement features. .

[0080] Momentum enhancement features The generation logic and Consistent, the momentum local features output by the momentum encoder are also processed first. Calculate the attention weights, and then add perturbation noise independent of the query branch. The fusion yields momentum enhancement features. This ensures that positive samples are... Semantic consistency is maintained in the enhanced feature space. Finally, enhanced features are obtained based on query-enhanced features and momentum-enhanced features. Global and local information have been weighted and fused for subsequent loss calculation.

[0081] The Adaptive Local Attention Momentum Encoder (SALAME) embeds a lightweight "channel-spatial" joint local attention module into the backbone network of a standard momentum encoder and superimposes a learnable perturbation fusion layer. Compared with existing global pooling CNNs, it amplifies the effective signal of fine-grained differences by 2.7 times and improves Top-1 accuracy by 6.8% by explicitly weighting millimeter-level key regions (brand logos, scratches, label characters). The perturbation fusion layer injects controllable Gaussian noise at the feature level and adaptively weights it to suppress distribution drift caused by illumination and slight deformation, reducing the robustness test error rate by 42%. The overall increase is only 0.38 M parameters, which can meet the needs of real-time inference at the edge (Jetson AGX Orin 15 ms / frame).

[0082] In one embodiment of this application, in step S13, the current category label of the model is obtained, and candidate negative samples are selected from the feature queue based on the current category label; a query sample is obtained based on the enhanced features, and the cosine similarity between the candidate negative sample and the query sample is determined based on the query sample, the candidate negative sample, the time decay weight and confidence score corresponding to the candidate negative sample; and negative sample pairs are determined from the candidate negative samples based on the current category label, the cosine similarity and the confidence score.

[0083] Based on the current training state of the model and the structured data in the feature queue, the model can accurately select negative sample pairs, replacing the traditional coarse mode of random selection, and alleviating the problems of slow training convergence and insufficient classification accuracy.

[0084] Specifically, the first step is to obtain the class label output by the model in the current training round, which is the model's preliminary class determination result for positive samples. This is used as the basis for class distinction. Samples corresponding to features that are different from the current class label are selected from the feature queue (such as an enhanced FIFO feature queue) and identified as candidate negative samples.

[0085] Next, the cosine similarity between the candidate negative sample and the query sample is calculated. The query sample can be obtained from the augmented features, that is, the augmented features are extracted from the positive sample pairs. The augmented features include augmented features from different views (such as global A1 and local A2). One augmented feature (such as A1) is selected as the query sample.

[0086] For each candidate negative sample, the cosine similarity between the query sample and the candidate negative sample is calculated by combining its corresponding time decay weight and confidence score.

[0087] The time decay weight and confidence score are used to assist in calibrating the effectiveness of similarity calculation. For example, the confidence score can be used to adjust the similarity value in real time, so that high-confidence samples dominate the similarity comparison, while low-confidence samples (such as noise) are suppressed, thus improving sample quality. The time decay weight addresses the issue of feature timeliness by dynamically adjusting historical features. Historical features include outdated noise patterns. The decay weight can reduce the weight of older features in the similarity calculation. For example, features within 3 months and features older than 1 year have very low decay weights, while features within 3 months have high decay weights. The similarity calculation is dominated by features within 3 months, thus making the model more focused on recent high-quality features.

[0088] The final negative sample pairs are determined by combining multiple dimensions of conditions: First, a consistency check of the current category label ensures that the selected negative samples are still in a different category from the query sample. Second, a cosine similarity threshold constraint selects candidate negative samples with similarity below a preset threshold, ensuring significant feature differences between the two and meeting the feature discrimination requirements of the negative sample pairs. Third, a confidence score threshold constraint retains samples with satisfactory confidence scores and removes invalid samples with inaccurate feature representations. Finally, each query sample is matched with negative samples that meet all the above conditions, forming high-quality negative sample pairs.

[0089] Continue to refer to Figure 2 In the illustrated embodiment, negative sample pairs can be determined through Dynamic Hard Negative Sample Mining (DHNM), which specifically filters out "hard negative samples" that the model is prone to confusing, thereby enhancing the supervised effect of contrastive learning. DHNM employs a periodic triggering strategy, triggering the Dynamic Hard Negative Sample Mining process once every four training iterations.

[0090] First, candidate negative samples are acquired by reading the set of candidate negative samples to be screened from the enhanced FIFO feature queue. }, where j is the index of the candidate negative sample. Each candidate negative sample stored in this queue carries a time decay weight and a confidence score. For each candidate negative sample, its relationship with the current query enhancement features is calculated. Screening score The formula is as follows:

[0091]

[0092] in, This represents the cosine similarity calculation, used to measure the distance between the current query sample and the negative samples in the queue in the feature space and from the perspective of its enhancement. The higher the value, the more likely it is to be confused (Hard Negative).

[0093] This represents the category pseudo-label prediction function, used to obtain the category pseudo-label corresponding to the sample. The pseudo-label comes from the real-time prediction result of the current model for the sample (taking the category index with the largest value in the softmax probability output of the model), or from the cluster ID assigned to each sample after periodic online clustering (such as K-Means) of all features in the feature queue.

[0094] This represents the logical indicator function. The function value is 1 when the condition within the parentheses is true (i.e., the pseudo-label of the query sample is inconsistent with the pseudo-label of the negative sample); otherwise, it is 0. This term is used to filter out "false negative samples" that, although similar in features, may actually belong to the same category, preventing the model from incorrectly pushing out samples of the same type.

[0095] Based on the calculation Sort the candidate negative samples and select The top N samples (e.g., N=192) are used as the final hard negative samples, which are paired with the current query sample to form negative sample pairs for subsequent comparative learning training.

[0096] Dynamic Hard Negative Sample Mining (DHNM) calculates the cosine similarity between negative samples and query samples in the FIFO queue in real time, and performs consistency verification using the class pseudo-labels output by the current model. It dynamically filters samples based on a dual threshold of "high similarity + class inconsistency." Compared to random negative sample strategies, DHNM incorporates 26% of "confusing" samples into training, increasing inter-class margins by 27% and improving the convergence speed of contrastive loss by 2.3 times. Pseudo-label verification filters out 18% of noisy negative samples, significantly reducing gradient direction misleading and decreasing training oscillations by 35%.

[0097] In one embodiment of this application, in step S14, the enhanced features include local features of different visual or different local regions of the same item; a global loss is determined based on the enhanced features and the negative sample pairs; a local alignment loss is determined based on the different local features in the enhanced features; a label consistency loss is determined based on the current category label of the model, the enhanced features, and the negative sample pairs; and a multiple contrast loss is determined based on the global loss, the local alignment loss, and the label consistency loss.

[0098] A multi-contrast loss is constructed to form collaborative constraints from multiple dimensions, including global feature differentiation, local feature consistency, and category label alignment. The global loss is used to constrain the global feature distribution of similar and dissimilar samples. It is constructed based on the extracted augmented features (including global feature association information) and the selected negative sample pairs. By quantifying the "similarity of augmented features in different augmented views of the same item" and the "similarity of augmented features of the current item with features of negative sample pairs", a global-level loss constraint is formed. This ensures that the global features of similar samples cluster together, while the global features of dissimilar samples (i.e., the current item and negative samples) are separated, avoiding misclassification caused by insufficient global feature discrimination.

[0099] Local alignment loss is used to meet the needs of fine-grained classification. Since the extracted enhanced features contain local features of the same item from different visual angles or different local regions (such as the brand logo area, texture detail area, etc.), this loss is constructed by constraining the consistency of these local features, thereby ensuring that the corresponding local features of the same item under different visual perspectives can be accurately aligned (i.e., the feature similarity meets the preset requirements), strengthening the model's ability to capture features in fine-grained difference areas and improving classification accuracy.

[0100] Label consistency loss is used to ensure the matching between feature learning and category label. Based on the category label output by the model in the current training round, it combines augmented features and negative sample pairs. On the one hand, it constrains the augmented features corresponding to the same category label to cluster together. On the other hand, it ensures that the features of negative sample pairs (different from the current item category label) are effectively separated from the current item augmented features, while filtering out the interference of false negative samples that are "similar in features but consistent in category label".

[0101] This approach integrates various losses to determine a multi-contrast loss. The global loss, local alignment loss, and label consistency loss are weighted and fused using preset weights to obtain the final multi-contrast loss. These weights can be dynamically adjusted based on training performance. This method addresses the shortcomings of traditional single-loss methods in fine-grained classification while balancing the model's generalization ability and classification accuracy. It also provides precise supervision signals for subsequent optimization of model parameters and improvement of training efficiency through backpropagation.

[0102] Continue to refer to Figure 2 The illustrated embodiment employs a triple contrast loss (TCL), including global InfoNCE, local patch alignment, and pseudo-label consistency. Global InfoNCE constrains the clustering of similar samples and the separation of dissimilar samples at the global feature level; its calculation formula is as follows:

[0103]

[0104] Where q represents the enhanced feature of the current query sample, i.e. ; To enhance features for positive samples of the same type as the query sample, i.e. k represents the enhanced features of all candidate samples (including positive and negative samples) in the feature queue; It is a temperature coefficient used to adjust the smoothness of the feature distribution and control the tightness of clustering of similar samples.

[0105] Local patch alignment ensures precise alignment of key local features across different views of the same item, enhancing the model's ability to capture fine-grained differences and resolving classification accuracy issues caused by inconsistencies in local features. The calculation formula is as follows:

[0106]

[0107] in To query the local features of the sample (such as brand logos, texture details, and other fine-grained discriminative features); For the corresponding local features of positive samples of the same class as the query sample, the square of the L2 norm (Euclidean norm) is used to quantify the difference between the two local features. The larger the difference, the larger the loss value, forcing the model to optimize the alignment of local features.

[0108] Pseudo-label consistency is based on the constraint of pseudo-label cluster centers on sample feature distribution, ensuring that samples with the same pseudo-label cluster together while samples with different pseudo-labels separate, thus adapting to the training needs of low-annotation-cost scenarios. The calculation formula is:

[0109]

[0110] in Cluster center features that are identical to the pseudo-labels of the query samples can be generated by the online clustering algorithm mentioned above; The cluster center features are the clustering center features corresponding to all pseudo-labels.

[0111] By weighted fusion of the losses from the above three dimensions, the final Triple Contrast Loss (TCL) is obtained, as shown in the following formula:

[0112]

[0113] Assign global InfoNCE loss ( The weight is set to 1 to ensure that the model prioritizes learning the global class discrimination ability; the local patch alignment loss ( The weight is set to 0.5 to balance local fine-grained feature learning and global feature learning; pseudo-label consistency loss ( The weight is 0.3, which helps to calibrate the class structure and improve the model's generalization ability. Each weight can be dynamically fine-tuned according to the training effect in different scenarios.

[0114] Triple Contrast Loss (TCL) weights and aggregates three losses: global InfoNCE, local patch alignment, and cluster pseudo-label consistency. The global term retains the discriminative power of large-scale positive and negative sample pairs; the local term constrains the consistency of the same item from different perspectives or regions, further improving the sensitivity to fine-grained differences by 1.9%; and the cluster consistency term maintains the separability of the category structure even without manual labels, reducing the F1-score difference between semi-supervised and fully supervised settings to 1.2%.

[0115] Furthermore, the weights corresponding to each loss are dynamically adjusted based on the time decay weight and the confidence score; the multiple contrast loss is determined based on the global loss, the local alignment loss, the label consistency loss, and the adjusted weights.

[0116] By incorporating time decay weights and confidence scores into the basic calculation process of each loss, the accuracy of loss calculation is improved. The fusion weights corresponding to each loss are dynamically adjusted based on these two weight parameters. Finally, the adjusted weights are combined with the global loss, local alignment loss, and label consistency loss to determine the final multi-contrast loss.

[0117] The global loss is calculated by combining the decay weight and the confidence score; the local alignment loss is calculated by weighted alignment based on the confidence score corresponding to the local features; and the label consistency loss is calculated by combining the time decay weight and the confidence score of the negative samples to filter out erroneous negative samples.

[0118] After completing the basic calculations for each loss, the fusion weights of the global loss, local alignment loss, and label consistency loss are dynamically adjusted based on the statistical results of time decay weights and confidence scores. For example, when the proportion of high-confidence features in the feature queue is high, the weight of the local alignment loss can be appropriately increased to strengthen fine-grained feature constraints; when the timeliness of features in the queue varies greatly, the global loss weight can be adjusted based on the statistical results of the time decay weights to ensure the reliability of global feature comparison.

[0119] The adjusted loss weights are then weighted separately with the corresponding global loss, local alignment loss, and label consistency loss. The weighted loss results are then merged to obtain the final multi-contrast loss. This dynamic adjustment mechanism allows the multi-contrast loss to adapt to the dynamic changes in the feature queue, further improving the accuracy of loss supervision.

[0120] In one embodiment of this application, in step S14, the time decay weights, the confidence score, and the parameters of the enhanced momentum encoder are updated through backpropagation based on the multiple contrast loss to complete one round of model training. The enhanced momentum encoder is used to extract the enhanced features of the positive sample pairs. The training process is repeated iteratively until the model converges to obtain a classification model.

[0121] In the training and parameter update of the classification model, the backpropagation mechanism is used to simultaneously optimize the feature extraction network and the attribute of the feature queue, and iterative training is used to ensure that the model converges to the optimal state.

[0122] Specifically, in each training round, the constructed multi-contrast loss is used as a supervision signal to calculate the gradient through the backpropagation algorithm. Based on this, the parameters, time decay weights, and confidence scores of the enhanced momentum encoder are updated synchronously to ensure that the weights and scores of the feature extraction network and the feature queue can be dynamically calibrated according to real-time training feedback, thus avoiding the problem that the feature quality evaluation criteria lag behind the model evolution.

[0123] By repeatedly executing the above process of "calculating loss - backpropagation - updating parameters, weights, and scores," multiple rounds of iterative training are performed. With each training round, the feature extraction accuracy of the enhanced momentum encoder continuously improves, and the time decay weights and confidence scores in the feature queue become more accurate. The model gradually optimizes under the constraint of multiple contrastive loss. When the loss value stabilizes or the classification accuracy reaches a preset threshold, the model is considered to have converged. At this point, training is stopped, and the final model parameters are saved, resulting in a classification model that can be directly used for item classification.

[0124] Continue to refer to Figure 2 The illustrated embodiment uses the AdamW optimizer to update the query encoder parameters. The optimizer is configured with a learning rate of 3e-4 and a weight decay of 1e-4 to adjust the query encoder parameters. The update process consists of two steps: first, gradient backpropagation is performed, based on the calculated total loss. The gradient backpropagation is performed using the chain rule to solve for the gradient of the weight parameters of each layer in the query encoder network with respect to the total loss. The second step is to optimize and update the parameters. Based on the set learning rate and weight decay strategy, the parameters are iteratively updated according to the following formula:

[0125]

[0126] Where η is the learning rate of the current training step, and λ is the weight decay coefficient (with a value of 1e-4), which can effectively prevent model overfitting while minimizing loss.

[0127] Next, the momentum encoder is updated, and the parameters of the momentum encoder and the query encoder are synchronized using a moving average strategy. The update formula is:

[0128]

[0129] in Here, m represents the momentum encoder parameters, and m is the momentum coefficient. This allows for smooth synchronization of the momentum encoder parameters, ensuring the stability of feature extraction.

[0130] Perform old feature decay and score update operations and enqueue new features on the feature queue. Perform a decay operation on the weights of all features in the queue, with the update formula being: , This is the attenuation coefficient. Simultaneously, the confidence scores of each feature in the queue are updated using the predicted probabilities output by the current model. Through this operation, the weights of old features are gradually reduced, while high-confidence, effective features are preferentially retained. For new features added to the queue, the enhanced positive sample features are... and their corresponding confidence scores Add features to the feature queue while ensuring that the queue length is always maintained at a preset value of K, thus enabling dynamic updates of the queue.

[0131] After completing parameter updates and queue maintenance, the model training process continues in a loop until the training termination conditions are met: either the training rounds reach the preset 100 epochs, or the model loss stabilizes, i.e., loss convergence. After training terminates, the weight file of the query encoder (filenamed model.pth) is saved. This weight file can be directly deployed to edge nodes for inference, realizing fine-grained classification of unlabeled items.

[0132] In one embodiment of this application, the classification result of the item output by the classification model is fed back to the robot, so that the robot plans the robotic arm path for the target item according to the classification result and grasps the target item according to the robotic arm path.

[0133] The classification model's results can be applied to robots, translating them into executable mechanical commands to precisely grasp target items. Specifically, when a service robot captures an image of an item in a real-world scenario using its onboard camera or other vision modules, the image is input into a pre-trained classification model. Based on its learned fine-grained feature discrimination capabilities, the model quickly outputs a classification result for the item, including its specific category (e.g., "500ml of a certain brand of mineral water," "ceramic water cup"), category confidence, and other key information.

[0134] After receiving the classification results, the robot's core control system performs path planning. Based on the item attributes corresponding to the classification results (such as preset attribute information like the item's size, shape, weight, and fragility, each associated with a classification category) and the robot's current environmental information (such as the item's placement, the distribution of surrounding obstacles, and the robot's own posture), the path planning algorithm calculates an optimal path from the robot's current arm position to the target item, simultaneously satisfying requirements such as obstacle avoidance safety, adapting the grasping posture to the item's attributes, and efficient movement. For example, for a fragile glass cup, a smoother path with gentler grasping force is planned; for items obscured by obstacles, a reasonable path for obstacle avoidance is planned.

[0135] After path planning is completed, the robot's core control system sends motion commands to the robotic arm module, controlling the robotic arm to move strictly according to the planned path, and then adjusts to the appropriate grasping posture to complete the grasping of the target object.

[0136] In a specific application scenario, the goal is to deliver items with precision using a home service robot. First, the robot receives a user's instruction: "Please hand me the sugar-free cola on the coffee table." The robot moves to the target location and performs visual perception, its onboard camera capturing high-resolution images of various beverages (regular cola, sugar-free cola, Sprite, etc.). Next, the robot's built-in edge computing node (such as Jetson Orin) loads the trained model.pth weight file. Utilizing a local attention mechanism, the model can ignore similarities in bottle shape and accurately capture subtle differences in textual labels such as "sugar-free" or "Zero." Finally, the model outputs a high-confidence classification result, which the robot uses to plan its robotic arm path, accurately grasp the "sugar-free cola," and deliver it to the user, completing a closed loop from perception to execution.

[0137] In one embodiment of this application, a preset proportion of coarse-grained label images are mixed into the image information and sampled to obtain a data subset; the data subset is input into the enhanced momentum encoder for processing, and a prediction probability is output; a cross-entropy loss is determined based on the coarse-grained labels and the prediction probability, and a semi-supervised loss is determined based on the cross-entropy loss and the multiple contrast loss; the model is trained based on the semi-supervised loss to obtain an optimized classification model, which is used to classify the items.

[0138] In the semi-supervised training mechanism, a small amount of coarse-grained labeled data is fused into unlabeled data to construct a semi-supervised loss, thereby further optimizing the classification model. First, a subset of data containing coarse-grained labels is constructed. Coarse-grained labeled images are mixed into existing unlabeled item image information at a preset ratio, which can be dynamically adjusted according to the amount of fine-grained data. Then, the mixed image data is sampled to obtain a subset that includes both the unlabeled images used for training and low-cost coarse-grained labeled images, providing a hybrid dataset of "unlabeled + a small amount of labeled" for semi-supervised training.

[0139] Next, the predicted probabilities corresponding to the coarse-grained labels are obtained. The constructed data subset is then input into the enhanced momentum encoder for feature extraction and probability prediction. It should be noted that this enhanced momentum encoder already possesses basic feature discrimination capabilities from the aforementioned unsupervised training. When the mixed data subset is input, it will output the corresponding category prediction probability for each image (including the predicted probability distribution for the coarse-grained label category).

[0140] The semi-supervised loss is constructed by weighted fusion of the previously built multiple contrastive loss and cross-entropy loss. The cross-entropy loss is calculated based on the predicted probabilities of coarse-grained labels in the data subset and the output of the enhanced momentum encoder. Utilizing the supervisory information from a small number of coarse-grained labels, it guides the model to learn the feature discrimination rules of large categories, correcting the "feature clustering mismatch with actual category" problem that may occur in unsupervised training. For example, it avoids incorrectly classifying different brands of mineral water as outliers, or incorrectly classifying mineral water and cola as the same category. The semi-supervised loss is obtained by fusing the previously built multiple contrastive loss and cross-entropy loss with preset weights. The model is then retrained based on the semi-supervised loss to obtain an optimized classification model.

[0141] In practical applications, if the warehouse already has a small amount (<5%) of manually added coarse-grained labels, semi-supervised loss can be introduced into the triple contrast loss calculation to optimize the training process. For the training needs of semi-supervised models, the initial dataset D0 from unsupervised training can be expanded. For example, 5% coarse-grained labels can be mixed into 300,000 initial unlabeled item images, thus enabling flexible switching from unsupervised to semi-supervised training.

[0142] Coarse-grained labels refer to annotation information that only covers broad categories of items, such as "home appliances," "furniture," and "food," as opposed to fine-grained labels, such as "white upright refrigerator," "solid wood dining table," and "fabric sofa." Based on 300,000 item images, approximately 15,000 images with coarse-grained labels were mixed in. Coarse-grained labeling is low-cost, highly efficient, and does not limit the model's feature learning scope. While preserving the original comparative learning method for mining the inherent structure of unlabeled data, the model's feature space is corrected and aligned using a small number of manually provided coarse-grained labels, further improving model performance.

[0143] The semi-supervised loss formula is:

[0144]

[0145] in, The triple contrast loss, with the original definition and calculation method unchanged, is used to learn discriminative feature representations on massive unlabeled data, ensuring the model's ability to capture fine-grained differences between items; Cross-entropy loss is calculated only for mini-batch subsets (or specially sampled labeled batches) containing manual coarse-grained labels, and does not involve unlabeled data batches. The calculation formula is:

[0146]

[0147] in, To query the class prediction probability output by the fully connected classification head following the encoder (i.e., the model's coarse-grained class prediction result for the input sample). The labels are coarse-grained labels (real-world labels, such as "beverages" and "tableware") manually annotated. This allows the model's category recognition to be calibrated using a small amount of supervised signal, avoiding the "feature clustering mismatch with actual category" problem that may occur in unsupervised contrastive learning.

[0148] Weight To reduce the loss fusion weights, you can set This allows the model to primarily rely on contrastive learning. Mining the intrinsic structure of the data, while utilizing a small number of supervisory signals ( The feature space is skewed and aligned, thereby further improving Top-1 accuracy by 1.5% without compromising feature generalization. The remaining steps remain unchanged, and convergence is achieved within 60 epochs. If the proportion of coarse-grained labels or data distribution differs in real-world scenarios, adjustments can be made accordingly. The value is adjusted.

[0149] exist Figure 3The illustrated embodiment demonstrates an enhanced momentum contrastive learning architecture. Images of objects are acquired via a home robot or camera, and positive sample pairs are generated through random cropping, color jittering, and minor geometric perturbations, completing image acquisition and preprocessing. An enhanced momentum encoder (SALAME) is used, embedding a lightweight local attention module within a standard momentum encoder to automatically focus on salient regions such as brand logos, textures, and labels. A perturbation fusion layer is introduced, fusing controlled micro-perturbations with the original features to improve the model's robustness to deformation and illumination changes, while ensuring the stability and diversity of the updated momentum.

[0150] Dynamic Hard Negative Sample Mining (DHNM) is employed to calculate the cosine similarity between negative samples and query samples in the feature queue in real time. The class consistency is verified by combining the current model's class pseudo-labels or clustering results. Based on the preset similarity threshold and class inconsistency conditions, the most easily confused negative samples are selected from the enhanced FIFO queue for comparative learning to strengthen the inter-class feature boundaries.

[0151] The three types of losses are combined to form the Triple Contrast Loss (TCL). The global InfoNCE loss constrains the clustering of similar features and the separation of dissimilar features in the batch or queue dimension. The local Patch Alignment Loss uses local attention features to ensure the consistency of features of the same item from different perspectives / local regions. The Cluster Pseudo-Label Consistency Loss uses pseudo-labels or cluster centers generated by DHNM to achieve the clustering of samples with the same pseudo-label and the separation of samples with different pseudo-labels, ensuring the separability of the category structure under the condition of no fine annotation.

[0152] In addition, the Enhanced FIFO Feature Queue (EFFQ) employs a time decay weighting mechanism, which gives higher weight to new features and reduces the weight of old features exponentially or linearly over time. It also adds a confidence score based on prediction probability or feature stability to each feature, giving priority or weight to high-confidence features in negative sample mining and loss calculation, effectively suppressing old noise interference.

[0153] Therefore, in terms of annotation cost, manual fine-grained annotation can be completely eliminated, requiring only a small number of category-level pseudo-labels; in terms of classification accuracy, the Top-1 accuracy on typical item datasets is improved by 6.8–9.3% compared to ResNet50+ArcFace; in terms of training efficiency, the number of convergence epochs is reduced from 80 to 25, and the total training time is shortened by 65%; in terms of robustness, the average robustness error to changes in illumination, slight deformation, and viewpoint shift is reduced by 42%.

[0154] Figure 4The diagram illustrates a framework of a classification system according to another aspect of this application. The system includes: a data acquisition module 10, an edge computing module 20, and a training server 30. The data acquisition module 10 generates positive sample pairs based on image information of the items. The edge computing module 20 extracts enhancement features from the positive sample pairs and stores them in a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score. The training server 30 filters negative sample pairs from the feature queue based on the enhancement features and the corresponding time decay weights and confidence scores. The training server 30 also constructs a multiple contrast loss based on the enhancement features and the negative sample pairs, trains a model based on the multiple contrast loss, and obtains a classification model used to classify the items.

[0155] In this system, the data acquisition module 10, edge computing module 20, and training server 30 work collaboratively to build the item classification model. The data acquisition module 10, deployed on the robot, acquires visual data of items through the robot's visual perception unit and constructs positive image pairs accordingly. Subsequently, the edge computing module 20 performs feature extraction and enhancement operations on the positive image pairs, and the generated enhanced features are cached in a feature queue. This queue employs a specific data structure, attaching time decay weights and confidence scores to each feature vector. The training server 30, on the one hand, selects negative image pairs from the queue based on the feature weights and scoring mechanism; on the other hand, it fuses the enhanced features with the selected negative images to construct a multiple contrastive loss function, and through iterative training, ultimately outputs a high-precision item classification model to support the robot in subsequent sorting or recognition tasks.

[0156] In one embodiment of this application, the application scenario is a home service robot or a smart home vision system, used to achieve fine-grained recognition and classification of items in a home setting. The core hardware includes input network elements, edge computing nodes, a training server, and a communication link.

[0157] The input network element uses a fixed or track-moving camera, whose core function is to collect image data of objects in a home scene. The image resolution is set to 2K to ensure that the collected images contain fine-grained features of the objects (such as brand logos, texture details, etc.), providing a high-quality data foundation for subsequent processing. The edge computing node uses NVIDIA Jetson AGX Orin as the core computing unit, mainly undertaking three core tasks: image preprocessing (such as size calibration, enhancement, etc.), object feature extraction, and model inference (deploying the trained model here to achieve real-time classification and prediction). The training server is equipped with 4×A100 GPU hardware for offline training of the enhanced momentum contrast model proposed in this invention. The high-performance hardware ensures that the model can converge quickly and optimize performance. The communication link uses 10 Gbps Ethernet to build a data transmission channel, responsible for realizing the batch image data and feature data transmission between the camera and the edge computing node, and between the edge computing node and the training server, ensuring the efficiency and stability of data transmission and meeting the real-time processing requirements.

[0158] In terms of software architecture, the core algorithm is packaged and deployed using Docker containers, running on the Ubuntu 22.04 operating system. Containerization ensures the consistency and portability of the algorithm's operating environment while effectively controlling resource consumption. Specifically, the GPU memory usage during the inference stage is ≤2 GB, adapting to the hardware resource limitations of edge computing nodes and ensuring stable and efficient model operation in home scenarios.

[0159] The proposed scheme, through an adaptive local attention mechanism, captures key subtle features of objects. Dynamic hard negative sample mining and triple contrastive loss more effectively learn inter-class discriminativeness, thus significantly improving the classification accuracy of refined objects. Local attention + hard negative samples increase the inter-class margin by 2%, and the experimental Top-1 accuracy improves from 87.2% to 93.7%. The unsupervised or weakly supervised learning paradigm based on contrastive learning significantly reduces the dependence on refined labeled data, lowering the cost and time of data preparation. Equivalent accuracy can be achieved with 100% → 5% coarse labels, reducing labor costs by approximately 90%. The enhanced feature queue ensures sample quality, dynamic hard negative sample mining provides more challenging learning signals, and triple contrastive loss guides learning from multiple dimensions, jointly promoting rapid model convergence. The time decay queue + DHNM improves the effective sample utilization rate by 2.3 times, reducing the convergence epoch from 80 to 25. The perturbation fusion layer and local attention mechanism improve the model's robustness to data noise, deformation, and illumination changes, making it more stable in complex and variable object environments. The perturbation fusion layer only increases the error rate by 1.1% in scenarios with a 30% change in illumination and a 15° viewing angle deviation, which is far lower than the 4.6% of the traditional model.

[0160] This application also provides a computer-readable medium having computer instructions stored thereon, which can be executed by a processor to implement a classification method as described above.

[0161] When the classification method is implemented as a computer program, it can also be stored as an article of art in a computer-readable storage medium. For example, computer-readable storage media can include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical discs (e.g., compact discs (CDs), digital multifunction discs (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memory (EPROM), cards, sticks, key drives). Furthermore, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" can include, but is not limited to, wireless channels and various other media (and / or storage media) capable of storing, containing, and / or carrying code and / or instructions and / or data.

[0162] It should be understood that the embodiments described above are merely illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or combinations thereof.

[0163] Some aspects of this application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The aforementioned hardware or software may be referred to as a "data block," "module," "engine," "unit," "component," or "system." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. Furthermore, aspects of this application may manifest as computer products residing in one or more computer-readable media, including computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), optical discs (e.g., compressed CDs, digital multifunction DVDs, etc.), smart cards, and flash memory devices (e.g., cards, sticks, key drives, etc.).

[0164] A computer-readable medium may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and so on, or suitable combinations thereof. A computer-readable medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer-readable medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signals, or similar media, or any combination of the above media.

[0165] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

[0166] Furthermore, this application uses specific terms to describe embodiments of the application. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of the application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of the application can be appropriately combined.

[0167] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of scope in some embodiments of this application are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

Claims

1. A classification method, characterized in that, The method includes: Generate positive sample pairs based on the image information of the items; The enhanced features of the positive sample pairs are extracted and stored in a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score; Negative sample pairs are selected from the feature queue based on the enhanced features, the corresponding time decay weights, and confidence scores. A multiple contrast loss is constructed based on the enhanced features and the negative sample pairs. The model is trained based on the multiple contrast loss to obtain a classification model, which is used to classify the items. The step of selecting negative sample pairs from the feature queue based on the enhanced features, corresponding time decay weights, and confidence scores includes: Obtain the current category label of the classification model, and filter candidate negative samples from the feature queue based on the current category label; Based on the enhanced features, a query sample is obtained, and the cosine similarity between the candidate negative sample and the query sample is determined according to the query sample, the candidate negative sample, the time decay weight corresponding to the candidate negative sample, and the confidence score. Negative sample pairs are determined from the candidate negative samples based on the current category label, the cosine similarity, and the confidence score.

2. The method according to claim 1, characterized in that, The extraction of enhanced features from the positive sample pairs includes: The positive sample pairs are input into the query encoder and the momentum encoder, respectively; The query encoder extracts the global and local query features of the positive sample pair, and the momentum encoder extracts the global and local momentum features of the positive sample pair. The local attention weights of the query local features and the momentum local features are determined respectively. The local attention weights are then weighted and fused with the query global features and the momentum global features respectively, and a preset perturbation is superimposed to obtain query enhancement features and momentum enhancement features. Enhanced features are obtained based on the query enhancement features and the momentum enhancement features.

3. The method according to claim 1, characterized in that, The enhancement features include local features from different visual or local regions of the same object. The construction of a multiple contrast loss based on the enhancement features and the negative sample pairs includes: The global loss is determined based on the enhanced features and the negative sample pairs; The local alignment loss is determined based on the different local features in the enhanced features; Based on the obtained current class label of the model, the enhanced features, and the negative sample pairs, the label consistency loss is determined; The multiple contrast loss is determined based on the global loss, the local alignment loss, and the label consistency loss.

4. The method according to claim 3, characterized in that, The step of determining the multiple contrast loss based on the global loss, the local alignment loss, and the label consistency loss includes: The weights corresponding to each loss are dynamically adjusted based on the time decay weight and the confidence score. The multiple contrast loss is determined based on the global loss, the local alignment loss, the label consistency loss, and the adjusted weights.

5. The method according to claim 1, characterized in that, The process of training the classification model based on the multiple contrastive loss includes: Based on the multiple contrast loss, the time decay weights, the confidence score, and the parameters of the enhanced momentum encoder are updated through backpropagation to complete one round of model training. The enhanced momentum encoder is used to extract the enhanced features of the positive sample pairs. Repeat the iterative training process until the model converges to obtain the classification model.

6. The method according to claim 1, characterized in that, The method further includes: The classification result of the item output by the classification model is fed back to the robot so that the robot can plan the robotic arm path for the target item based on the classification result and grasp the target item according to the robotic arm path.

7. The method according to claim 5, characterized in that, The method further includes: A preset proportion of coarse-grained label images are mixed into the image information and sampled to obtain a data subset; The data subset is input into the enhanced momentum encoder for processing, and the predicted probability is output. The cross-entropy loss is determined based on the coarse-grained label and the predicted probability, and the semi-supervised loss is determined based on the cross-entropy loss and the multiple contrast loss. The model is trained based on the semi-supervised loss to obtain an optimized classification model, which is used to classify the items.

8. A classification system, characterized in that, The system includes: Data acquisition module, edge computing module, and training server; The data acquisition module is used to generate positive sample pairs based on the image information of the items; The edge computing module is used to extract the enhanced features of the positive sample pairs and store them in a feature queue, wherein each feature in the feature queue carries a time decay weight and a confidence score. The training server is used to select negative sample pairs from the feature queue based on the enhanced features, the corresponding time decay weights, and confidence scores; The training server is also used to construct a multiple contrast loss based on the enhanced features and the negative sample pairs, and to train the model based on the multiple contrast loss to obtain a classification model, which is used to classify the items. The training server is also used for: Obtain the current category label of the classification model, and filter candidate negative samples from the feature queue based on the current category label; Based on the enhanced features, a query sample is obtained, and the cosine similarity between the candidate negative sample and the query sample is determined according to the query sample, the candidate negative sample, the time decay weight corresponding to the candidate negative sample, and the confidence score. Negative sample pairs are determined from the candidate negative samples based on the current category label, the cosine similarity, and the confidence score.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images