A Visual Recognition Method and System Based on COT Text Retrieval Enhancement and Offline Distillation

Abstract

This invention relates to the field of visual recognition technology, and discloses a visual recognition method and system based on COT text retrieval enhancement and offline distillation. The method includes: S1, acquiring an implicit text prior database, a set of labeled images, and a set of unlabeled images; constructing a multi-stage COT text memory; S2, constructing a teacher model; S3, obtaining enhanced visual features, and fusing visual features with a multi-layer retrieval fusion module through the multi-stage COT text memory; S4, supervising the teacher model and outputting offline pseudo-labels for the unlabeled images; S5, constructing a pure visual student model and training the student model to obtain a visual recognition student model; S6, performing visual recognition based on the trained student model. This invention can provide more structured and fine-grained semantic information, improving the ability to recognize complex samples.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology, specifically a visual recognition method and system based on COT text retrieval enhancement and offline distillation. Background Technology

[0002] Semi-supervised learning aims to improve model performance using a small amount of labeled data and a large amount of unlabeled data, and is an important technical approach to reduce the annotation cost of visual recognition tasks. Current mainstream semi-supervised learning methods typically rely on confidence-based pseudo-label strategies. That is, the model first predicts unlabeled samples, and then selects high-confidence predictions as pseudo-labels for training. These methods perform well in scenarios with independent and identically distributed data. However, in real-world open scenarios, unlabeled data often exhibits a significant distributional shift compared to labeled data, due to changes in background environment, imaging conditions, visual style, or category combinations. In such cases, model confidence is prone to calibration bias; high-confidence predictions do not necessarily correspond to correct results, leading to the continuous reinforcement of incorrect pseudo-labels and further exacerbating confirmation bias.

[0003] To address the instability of semi-supervised learning under distribution shift, existing methods typically reduce the impact of low-quality pseudo-labels through dynamic confidence thresholds, sample filtering, or open set noise detection. However, these methods primarily rely on the model's own predictions for filtering, lacking external semantic knowledge to correct visual representations. When visual features are affected by background changes, domain differences, or local interference, relying solely on internal confidence is insufficient to accurately determine the reliability of pseudo-labels. In recent years, retrieval-enhanced visual learning methods have begun to incorporate external retrieval evidence, extending the model learning process from single confidence judgments to external knowledge-assisted judgments, thereby mitigating representation drift and pseudo-label noise problems to some extent. However, existing retrieval enhancement methods often rely on visual neighbors or shallow text descriptions, resulting in retrieval results lacking a clear reasoning structure and making it difficult to perform stepwise semantic analysis on complex samples. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a visual recognition method and system based on COT (Chain-of-Thought) retrieval enhancement and offline distillation, which can provide more structured and fine-grained semantic information and improve the recognition ability of complex samples.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A visual recognition method based on COT text retrieval enhancement and offline distillation, characterized by comprising: S1. Obtain the implicit text prior database, the set of labeled images, and the set of unlabeled images; the set of labeled images includes labeled images and their real labels, and the set of unlabeled images includes unlabeled images and their corresponding COT texts; perform multi-stage semantic decomposition on the COT texts and construct a multi-stage COT text memory; S2. Construct a teacher model, including a visual encoder, a multi-layer retrieval fusion module, and a visual recognizer; the number of layers in the multi-layer retrieval fusion module is the same as the number of stages in the multi-stage COT text memory. S3. The visual encoder extracts the initial visual features of the input image, and the external implicit text prior database obtains the implicit text prior features of the input image. The initial visual features and the implicit text prior features are superimposed to obtain the enhanced visual features. The enhanced visual features are used as the first layer input visual features of the multi-layer retrieval fusion module. The multi-stage COT text memory is used to perform visual feature fusion on the multi-layer retrieval fusion module. The visual recognizer outputs the recognition result based on the last layer output visual features. S4. Supervised training of the teacher model is performed based on the labeled image set to obtain the trained teacher model; unlabeled images are input into the trained teacher model to output offline pseudo-labels for the unlabeled images; S5. Construct a pure vision student model. Train the student model based on a set of labeled images, unlabeled images, and offline pseudo-labels to obtain a visual recognition student model. S6. Perform visual recognition based on the trained student model.

[0006] Preferably, the construction of the multi-stage COT text memory specifically includes: S101. Perform multi-stage decomposition of COT text, and for unlabeled images... The corresponding COT text is decomposed into multiple stages according to semantic hierarchy, denoted as: ; in, Indicates the number of semantic stages obtained by decomposing the COT text; Represents text; Indicates the first Zhang unlabeled image in the first COT text fragments corresponding to each semantic stage; S102. Encode the COT text for each stage using a text encoder to obtain the corresponding COT text features: ; in, Indicates a text encoder. Indicates the first Zhang unlabeled image in the first COT text features at each semantic stage; S103. Construct a phased COT text memory and establish a retrieval index for each semantic phase. The COT text features of all unlabeled images at this stage are stored in the corresponding text memory, resulting in: ; in, Indicates the first COT text memory corresponding to each semantic stage; This represents the number of unlabeled images. S104, will The individual COT text memories were merged into a multi-stage COT text memory.

[0007] Preferably, the calculation of the enhanced visual features specifically includes: S301. Obtain the implicit text prior database. The image index and its corresponding implicit text features are represented as follows: ; in, Represents the image index. Represents the implicit text features corresponding to the image; S302. Based on the image index or implicit text features, retrieve the implicit text prior features corresponding to image x from the implicit text prior database, denoted as... ; S303. Input image x into the visual encoder of the teacher model to extract initial visual features: ; in, Represents a visual encoder. Represents the initial visual features of image x; S304. The retrieved implicit text prior features Mapped to the same dimensional space as the initial visual features: ; in, Represents a mapping network. Represents the implicit textual prior features after mapping; S305, Map the implicit text prior features Perform average pooling and combine with the initial visual features Adding them together yields the enhanced visual features: ; in, Indicates average pooling. This indicates an enhancement of visual features.

[0008] Preferably, the step of performing visual feature fusion on the multi-layer retrieval fusion module through a multi-stage COT text memory specifically includes: The teacher model is based on the visual features input to the current layer. Retrieve the multi-stage COT text memory to obtain a set of candidate COT text features. And for the candidate COT text feature set Perform a rating and screening process to obtain the filtered results. Each COT text feature constitutes a filtering set. ; Subscript This indicates the current layer of the multi-layer retrieval fusion module, i.e., the [layer number]. layer; For the filter set In The average aggregation of the COT text features yields the COT semantic guidance features of the current layer in the multi-layer retrieval fusion module. : ; in, This indicates that the multi-layer retrieval fusion module is in the first... The first layer of the filter set Candidate COT text features The range of values ​​is ; Input visual features into the current layer As a query, COT semantic guidance features As keys and values, they are fused through cross-modal attention to obtain the output visual features of the current layer, which then serve as the input visual features for the next layer. : ; in, This is a cross-modal attention computation operation.

[0009] Preferably, the candidate COT text feature set The acquisition specifically includes: In the multi-layer retrieval fusion module In each fusion layer, the current stage COT text memory is determined based on the retrieval index: ; in, This indicates the semantic stage corresponding to the current fusion layer. Indicates the number of COT text stages; Input visual features into the current layer As a query, in the COT Text Memory at Each Stage Similarity retrieval is performed to obtain a set of candidate COT text features. : ; in, For similarity retrieval.

[0010] Preferably, the set of candidate COT text features Perform a rating and screening process to obtain the filtered results. Each COT text feature constitutes a filtering set. Specifically, it includes: A lightweight scoring network is used to score the text features of candidate COT texts. The scoring formula can be expressed as: ; in, Represents the text features of candidate COTs. This represents a multilayer perceptron. This represents the Sigmoid function. This represents the relevance score of candidate COT text features, based on which the candidate COT text feature set is selected. Select the top scores in descending order. Each COT text feature is used to obtain the filtered COT text feature set.

[0011] Preferably, in S4: Based on labeled image sets The teacher model is trained under supervision to obtain a fully trained teacher model. ,in: ; in, Indicates the first Zhang has tagged images, This indicates the true category label corresponding to the image. This indicates the number of labeled images; For image detection tasks, the teacher model outputs the object detection category logit and the predicted object bounding box: ; in, Indicates the detection category logit, Indicates the predicted target box. Let x represent the teacher model and x represent the input image; The teacher model is trained using the loss functions corresponding to the detection task, including class loss, bounding box regression loss, and GIoU loss: ; in, Indicates the target category loss. This represents the bounding box regression loss. Indicates GIoU loss, The total loss of the teacher model; Unlabeled images Input the trained teacher model The offline pseudo-labels corresponding to the unlabeled image are obtained, including pseudo-boundary boxes and pseudo-detection category logit. .

[0012] Preferably, in S4: Based on labeled image sets The teacher model is trained under supervision to obtain a fully trained teacher model. ,in: ; in, Indicates the first Zhang has tagged images, This indicates the true category label corresponding to the image. This indicates the number of labeled images; For classification tasks, the teacher model outputs labeled images. The category logit, and the real category label Calculate cross-entropy loss : ; in, Represents the cross-entropy loss function; Indicates a labeled image The detection category is logit; Unlabeled images Input the trained teacher model The offline pseudo-labels corresponding to the unlabeled image are obtained, including the pseudo-detection category logit. .

[0013] Preferably, S5 specifically includes: S501, Constructing a Pure Visual Student Model ; S502, The pseudo-detection category logit in the offline pseudo-label is... Converted into teacher pseudo-labels and confidence weights; For unlabeled images Get the corresponding pseudo-detection category logit It is then converted into a class probability distribution using the Softmax function. : ; The category with the highest probability is selected as the pseudo-label for teachers. : ; Take the highest class probability as the confidence level. : ; in, This is the category probability distribution vector; This represents the category confidence level; the subscript 'c' indicates the category. Based on confidence level Calculate sample weights ; S503, Unlabeled images Input student model, get student model output Using pseudo-labels of teachers and confidence weight Calculate the weighted unlabeled loss: ; Where B is the batch sample size; Labeled images Input student model, get student model output : ; and with real labels Calculate the labeled supervised loss : ; S504, Based on labeled image loss and unlabeled image loss Establish a loss function and train the student model: ; in, The weights represent the unlabeled loss. This represents the total loss of the student model.

[0014] This invention also proposes a visual recognition system based on COT text retrieval enhancement and offline distillation, comprising: The acquisition module is used to acquire the implicit text prior database, the labeled image set, and the unlabeled image set; the labeled image set includes labeled images and their real labels, and the unlabeled image set includes unlabeled images and their corresponding COT texts; the COT texts are subjected to multi-stage semantic decomposition to construct a multi-stage COT text memory; The building module is used to construct the teacher model, including a visual encoder, a multi-layer retrieval fusion module, and a visual recognizer; the number of layers in the multi-layer retrieval fusion module is the same as the number of stages in the multi-stage COT text memory. The visual fusion module extracts initial visual features from the input image based on the visual encoder, obtains implicit text prior features from the input image using an external implicit text prior database, and superimposes the initial visual features with the implicit text prior features to obtain enhanced visual features. These enhanced visual features serve as the first layer of input visual features for the multi-layer retrieval fusion module. The multi-stage COT text memory database is used to perform visual feature fusion on the multi-layer retrieval fusion module. The visual recognizer outputs the recognition result based on the final layer of output visual features. Teacher model training module; used to supervise the training of the teacher model based on a set of labeled images to obtain a trained teacher model; input unlabeled images into the trained teacher model and output offline pseudo-labels for the unlabeled images; Student model training module; used to build a pure vision student model. The student model is trained based on a set of labeled images, offline pseudo-labels and unlabeled images to obtain a visual recognition student model. The recognition module is used for visual recognition based on the trained student model.

[0015] Compared with the prior art, the beneficial effects of the present invention are: (1) By introducing COT text in this embodiment, the model can logically find image details. The model no longer relies solely on surface visual similarity, but can more systematically focus on detailed information related to category discrimination during the training process.

[0016] (2) This embodiment proposes a hierarchical retrieval fusion teacher model. The visual features are updated at each level according to the COT semantic information retrieved at the current stage, thereby forming a progressive visual semantic reasoning process and improving the model's discrimination ability under distribution shift and complex scenarios.

[0017] (3) This embodiment proposes an offline distillation-type student model training mechanism, which can transfer the COT retrieval-enhanced reasoning ability in the teacher model to the student model, while avoiding the repeated execution of complex text retrieval and cross-modal fusion operations during the student training and reasoning stages.

[0018] (4) In the training phase, this embodiment makes full use of COT text, implicit text prior and retrieval enhancement teacher model to improve the quality of unlabeled sample utilization; in the inference phase, only pure visual student model is retained, and prediction can be completed by input image. This can improve the robustness of semi-supervised visual recognition in complex distribution scenarios and meet the efficiency requirements in actual deployment. Attached Figure Description

[0019] Figure 1 This is a flowchart of the visual recognition method based on COT text retrieval enhancement and offline distillation proposed in Embodiment 1 of the present invention.

[0020] Figure 2 This is a block diagram of the visual recognition system based on COT text retrieval enhancement and offline distillation proposed in Embodiment 1 of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 The visual recognition method based on COT text retrieval enhancement and offline distillation proposed in this invention is mainly used in the field of visual recognition and can be applied to multiple fields such as image classification, object detection, image segmentation, and 3D reconstruction according to actual needs. In this embodiment, image classification is used as an example to further illustrate the solution of this invention.

[0023] Please see Figure 1 This embodiment provides a visual recognition method based on COT text retrieval enhancement and offline distillation, including the following steps; A visual recognition method based on COT text retrieval enhancement and offline distillation includes: S1. Obtain the implicit text prior database, the set of labeled images, and the set of unlabeled images; the set of labeled images includes labeled images and their real labels, and the set of unlabeled images includes unlabeled images and their corresponding COT texts; perform multi-stage semantic decomposition on the COT texts and construct a multi-stage COT text memory.

[0024] Specifically, implicit text prior database The system can store image indices and their corresponding implicit text features, which are used in this application to provide additional global semantic references for the teacher model.

[0025] Each unlabeled image has a corresponding COT (Content on the Object) text, which is textual information describing the image content according to a certain logical sequence. This text may include global scene, key targets, local details, attribute features, and category-related semantics. The construction of a multi-stage COT text memory specifically includes: S101. Perform multi-stage decomposition of COT text, and for unlabeled images... The corresponding COT text is decomposed into multiple stages according to semantic hierarchy, denoted as: ; in, Indicates the number of semantic stages obtained by decomposing the COT text; Represents text; Indicates the first Zhang unlabeled image in the first COT text fragments corresponding to each semantic stage; This is the complete COT mind chain text accompanying the j-th unlabeled image.

[0026] In one embodiment of the present invention, The approach involves breaking down COT text into four semantic stages: global scene understanding, key target localization, fine-grained attribute analysis, and category-related semantic judgment. This breakdown allows the model to progressively focus on image content from the overall picture to the details, rather than using a single, mixed text description.

[0027] S102. Encode the COT text for each stage using a text encoder to obtain the corresponding COT text features: ; in, Indicates a text encoder. Indicates the first Zhang unlabeled image in the first COT text features at each semantic stage.

[0028] A text encoder is a technique that converts text data into numerical representations and is widely used in Natural Language Processing (NLP) tasks. By converting text into vectors, a text encoder can capture the semantic information of the text, providing a foundation for subsequent model processing. In this application, the text encoder can be a bidirectional mask encoder.

[0029] S103. Construct a phased COT text memory and establish a retrieval index for each semantic phase. The COT text features of all unlabeled images at this stage are stored in the corresponding text memory, resulting in: ; in, Indicates the first COT text memory corresponding to each semantic stage; This represents the number of unlabeled images. S104, will The individual COT text memories were merged into a multi-stage COT text memory.

[0030] In this invention, COT text features from different semantic stages are stored in separate text memories, rather than being mixed in the same memory. This avoids interference between semantic information at different levels, allowing the subsequent teacher model to retrieve COT semantic information that better matches the current visual representation in different network layers. Subsequently, Faiss (Facebook AI Similarity Search) is used to build a vector retrieval index for the COT text memory at each stage, enabling rapid retrieval of relevant COT text features based on visual characteristics. Faiss is an efficient similarity search and dense vector clustering library capable of handling large-scale vector sets, thus achieving rapid text retrieval.

[0031] Through the above steps, this invention obtains a multi-stage COT text memory, with each memory corresponding to a specific level of semantic information, providing a structured semantic source for the subsequent hierarchical retrieval and fusion of the teacher model.

[0032] S2. Construct a teacher model, including a visual encoder, a multi-layer retrieval fusion module, and a visual recognizer; the number of layers in the multi-layer retrieval fusion module is the same as the number of stages in the multi-stage COT text memory.

[0033] In this application, the visual recognizer can be a classifier or a detection head, depending on the actual application field. The number of layers l in the multi-layer retrieval fusion module can be the same as the number of stages s in the multi-stage COT text memory, which facilitates subsequent retrieval and matching.

[0034] S3. The visual encoder extracts the initial visual features of the input image, and the external implicit text prior database obtains the implicit text prior features of the input image. The initial visual features and the implicit text prior features are superimposed to obtain the enhanced visual features. The enhanced visual features are used as the first layer input visual features of the multi-layer retrieval fusion module. The multi-stage COT text memory is used to perform visual feature fusion on the multi-layer retrieval fusion module. The visual recognizer outputs the recognition result based on the last layer output visual features.

[0035] The purpose of this step is to perform hierarchical COT retrieval fusion on the multi-layer retrieval fusion module, so that visual features can gradually focus on key content and detailed information in the image according to the semantic hierarchy provided by the COT text.

[0036] S301. Obtain the implicit text prior database. The image index and its corresponding implicit text features are represented as follows: ; in, Represents the image index. This represents the implicit text features corresponding to the image.

[0037] S302. Based on the image index or implicit text features, retrieve the implicit text prior features corresponding to image x from the implicit text prior database, denoted as... ; S303. Input image x into the visual encoder of the teacher model to extract initial visual features: ; in, Represents a visual encoder. Represents the initial visual features of image x; In this embodiment, the teacher model can use ALBEF as the basic visual language model. (Image...) The visual tags can be extracted by inputting them into ALBEF's visual encoder. .

[0038] S304. The retrieved implicit text prior features Mapped to the same dimensional space as the initial visual features: ; in, Represents a mapping network. This represents the implicit textual prior features after mapping.

[0039] Specifically, in this embodiment, a lightweight mapping network can be used to process the retrieved implicit text prior features. It maps to the same dimensional space as visual features. Lightweight mapping networks are a type of small neural network with low parameter count and low computational overhead. Their core function is mapping transformation: establishing an efficient conversion relationship between two data spaces.

[0040] For example, implicit textual prior features The following information was obtained: By pre-calculating the text vectors for each of the j images... Store the implicit text prior database. For the input image x, find the best matching images in the implicit text prior database and extract their corresponding implicit text features. By combining these, we can obtain the semantic reference vector of the input image x. This refers to implicit textual prior features.

[0041] In this embodiment, due to the retrieved implicit text prior features Integrating implicit text features from multiple images results in different feature dimensions. Therefore, a lightweight mapping network can be used to integrate the retrieved implicit text prior features. Mapping to the same dimensional space as the initial visual features facilitates subsequent computation and recognition. The lightweight mapping network can select a cross-modal lightweight mapper.

[0042] S305. Map the implicit text prior features Perform average pooling and combine with the initial visual features Adding them together yields enhanced visual features: ; in, Indicates average pooling. This indicates an enhancement of visual features.

[0043] Mean pooling (also known as average pooling) is a common pooling technique in convolutional neural networks (CNNs). It can be used to reduce the spatial dimensions of feature maps, such as height and width, while preserving the overall feature trend. Unlike max pooling, mean pooling calculates the average of all elements within the pooling window as the output, resulting in a smoother feature representation. This embodiment utilizes implicit textual prior features... Performing average pooling can reduce the impact of local noise, preserve the overall trend, and reduce feature dimensionality and computational cost. This step allows the teacher model to obtain some supplementary global semantic information before entering hierarchical retrieval fusion.

[0044] In this embodiment, the teacher model can also be efficiently fine-tuned by introducing LoRA parameters. To adapt to the current task with lower training costs, this embodiment introduces LoRA low-rank adaptation parameters into the teacher model. Specifically, trainable low-rank parameters are added to some attention projection matrices in the ALBEF visual encoder and cross-modal fusion layer, while keeping most of the pre-trained parameters frozen. The parameter update form of LoRA can be expressed as: ; in, This represents the original weight matrix in the pre-trained model; and Represents a low-rank trainable matrix; This represents the equivalent weight matrix after incorporating LoRA. In this way, the teacher model can adapt to current semi-supervised visual recognition tasks with lower training costs, while retaining the representational capabilities of the ALBEF pre-trained visual language model.

[0045] Visual feature fusion is performed on a multi-layered retrieval and fusion module using a multi-stage COT text memory. The purpose of this step is to train the teacher model using the COT text memory, enabling the teacher model to alternately perform COT text retrieval and cross-modal fusion in different network layers. This allows visual features to progressively focus on key content and detailed information in the image according to the semantic hierarchy provided by the COT text. Specifically, the following steps can be used: The teacher model is based on the visual features input to the current layer. Retrieve the multi-stage COT text memory to obtain a set of candidate COT text features. And for the candidate COT text feature set Perform a rating and screening process to obtain the filtered results. Each COT text feature constitutes a filtering set. ; Subscript This indicates the current layer of the multi-layer retrieval fusion module, i.e., the [layer number]. layer.

[0046] In this embodiment, the teacher model includes a multi-layered retrieval fusion module. The number of modules is consistent with the number of COT inference steps. Each layer corresponds to an inference stage. Specifically, it may include four layers of retrieval fusion modules, which correspond to the global scene understanding, key target localization, fine-grained attribute analysis, and category-related semantic judgment stages of COT text, respectively.

[0047] First, determine the COT text memory corresponding to the current fusion module.

[0048] In the multi-layer retrieval fusion module In each fusion layer, the multi-stage COT text memory currently in use is determined based on the retrieval index: ; in, This indicates the semantic stage corresponding to the current fusion layer. This indicates the number of COT text stages.

[0049] In the initial stage, the visual features are the enhanced initial features obtained in step S305. .

[0050] The current COT text memory is determined by layer indexing, and the current visual features can be determined based on the layer number corresponding to the current COT text memory. .

[0051] Then, COT text feature retrieval is performed based on the current visual features.

[0052] Specifically, Input visual features into the current layer As a query, in the COT Text Memory at Each Stage Similarity retrieval is performed to obtain a set of candidate COT text features. : ; in, This indicates a similarity search.

[0053] In this embodiment, Faiss can be used to retrieve a large number of candidate COT text features, for example... A number of candidate features are selected to ensure that the candidate set contains sufficient relevant semantic information.

[0054] Next, the candidate COT text features are scored and filtered to obtain the filtered results. Each COT text feature constitutes a filtering set. A lightweight scoring network is used to score the text features of candidate COT texts.

[0055] The scoring formula can be expressed as: ; in, Represents the text features of candidate COTs. This represents a multilayer perceptron. This represents the Sigmoid function. The relevance score of candidate COT text features is used to select candidate COT text features from the set of features. Select the top scores in descending order. Each COT text feature is used to obtain the filtered COT text feature set.

[0056] In this embodiment, the lightweight scoring network is a lightweight multilayer perceptron (MLP) for text feature ranking. By scoring the text features of candidate COTs through the lightweight scoring network, it is easier to perform subsequent screening tasks.

[0057] Then, the top candidates are selected from the candidate set based on their scores. Each COT text feature is used to obtain the filtered COT text feature set. .

[0058] For the filter set In The average aggregation of the COT text features yields the COT semantic guidance features of the current layer in the multi-layer retrieval fusion module. : ; in, This indicates that the multi-layer retrieval fusion module is in the first... The first layer of the filter set One candidate COT text feature, The range of values ​​is ; In detail, by encoding the textual features of multiple filtered COTs in the candidate set into vectors, and averaging all vectors dimension by dimension, the overall inference semantic features of multiple COTs can be fused. This embodiment adopts the common global mean aggregation method, summing and averaging all COT vectors with equal weights, without additional parameters, and is suitable for the low-latency requirements of lightweight scoring networks.

[0059] Input visual features into the current layer As a query, COT semantic guidance features As keys and values, they are fused through cross-modal attention to obtain the output visual features of the current layer, which then serve as the input visual features for the next layer. : ; in, This is a cross-modal attention computation operation.

[0060] The teacher model alternately performs COT text retrieval and cross-modal fusion in different network layers, enabling visual features to gradually focus on key content and detailed information in the image according to the semantic hierarchy provided by the COT text.

[0061] Specifically, the current visual features As a query, COT semantic guidance features By using the key and value pairs as keys and calculating their similarity weights, bidirectional modality alignment can be achieved.

[0062] S4. Supervised training of the teacher model is performed based on the labeled image set to obtain the trained teacher model; unlabeled images are input into the trained teacher model to output offline pseudo-labels for the unlabeled images.

[0063] For different visual recognition application areas, the loss training function of the teacher model can be determined according to actual needs.

[0064] Taking image detection as an example: Based on labeled image sets The teacher model is trained under supervision to obtain a fully trained teacher model. ,in: ; in, Indicates the first Zhang has tagged images, This indicates the true category label corresponding to the image. This indicates the number of labeled images; For image detection tasks, the teacher model outputs the target category logit and the predicted bounding box: ; in, Indicates the detection category logit, Indicates the predicted target box. Let x represent the teacher model and x represent the input image.

[0065] Labeled detection data provides the true target category and true bounding box, while unlabeled images can still be combined with their corresponding COT text to construct a multi-stage COT text memory. The teacher detection model absorbs COT text information from different semantic stages and external implicit text priors through a hierarchical COT retrieval fusion module, thereby obtaining more reliable detection prediction results.

[0066] The teacher model is trained using the loss functions corresponding to the detection task, including class loss, bounding box regression loss, and GIoU loss: ; in, Indicates the target category loss. This represents the bounding box regression loss. Indicates GIoU loss, This represents the total loss of the teacher model.

[0067] Unlabeled images Input the trained teacher model The offline pseudo-labels corresponding to the unlabeled image are obtained, including pseudo-boundary boxes and pseudo-detection category logit. .

[0068] Taking classification and recognition as an example: After multi-level COT retrieval and fusion, the final visual features are pooled and input into the classifier to obtain the class logit output by the teacher model: ; in, This indicates an enhanced teacher retrieval model. This represents the category logit output by the teacher model.

[0069] Based on labeled image sets The teacher model is trained under supervision to obtain a fully trained teacher model. ,in: ; in, Indicates the first Zhang has tagged images, This indicates the true category label corresponding to the image. This indicates the number of labeled images; For labeled images The teacher model outputs labeled images. The category logit, and the real category label Calculate cross-entropy loss : ; in, Represents the cross-entropy loss function; Indicates a labeled image The detection category is logit; Unlabeled images Input the trained teacher model The offline pseudo-labels corresponding to the unlabeled image are obtained, including the pseudo-detection category logit. .

[0070] S5. Construct a pure vision student model. Train the student model based on offline pseudo-labels, labeled image sets, and unlabeled images to obtain a visual recognition student model. This step is used to build a pure vision student model and perform offline distillation training.

[0071] S501, Constructing a Pure Visual Student Model ; In this embodiment, a purely visual student model can be constructed that takes only images as input, denoted as... Pure vision student models can use ViT-B / 16, semi-DETR, or other visual backbone networks. The student model does not receive COT text, access the COT text memory, or use external implicit text prior databases.

[0072] S502. Convert offline pseudo-label logit into teacher pseudo-labels and confidence weights; Taking classification and recognition as an example: For unlabeled images Get the corresponding pseudo-detection category logit It is then converted into a class probability distribution using the Softmax function. : ; The category with the highest probability is selected as the pseudo-label for teachers. : ; Take the highest class probability as the confidence level. : ; in, This is the category probability distribution vector; This represents the category confidence level; the subscript 'c' indicates the category. Based on confidence level Calculate sample weights .

[0073] S503, Unlabeled images Input student model, get student model output Using pseudo-labels of teachers and confidence weight Calculate the weighted unlabeled loss: ; Where B is the batch sample size; Labeled images Input student model, get student model output : ; and with real labels Calculate the labeled supervised loss : ; S504, Based on labeled image loss and unlabeled image loss Establish a loss function and train the student model: ; in, The weights represent the unlabeled loss. This represents the total loss of the student model.

[0074] Through step S5, the student model can be trained using real label supervision and offline pseudo-label logit generated by the teacher model, thereby transferring the reliable supervision signal obtained by the teacher model through COT text retrieval enhancement to the pure vision student model.

[0075] S6. Perform visual recognition based on the trained student model.

[0076] In this embodiment, only the pure visual student model is retained. After training is complete, only the pure visual student model is retained. The teacher model, COT text memory, and external implicit text prior database are no longer retained.

[0077] Taking classification as an example, the input is an image to be recognized, and the output is the category logit. For the image to be recognized... Inputting it into the student model yields the category logit: ; The final image recognition result is obtained. The final predicted category is: ; At this stage, the model no longer requires COT text, COT text memory, external implicit text prior database, or retrieval-enhanced teacher model; it can complete recognition solely based on the input image. Therefore, this invention can improve the quality of unlabeled sample utilization during the training phase by leveraging COT text and retrieval enhancement mechanisms, while maintaining the efficiency of a pure vision model during the deployment phase.

[0078] As can be seen, this embodiment, by introducing COT text, enables the model to logically search for image details: existing retrieval enhancement semi-supervised methods typically only utilize similar images, visual features, or ordinary text descriptions, resulting in fragmented external information obtained by the model and a lack of clear hierarchical structure. This invention introduces COT text, decomposing the semantic information of unlabeled images into multiple consecutive stages, enabling the model to progressively focus on image content according to the logic of "global scene understanding—key target localization—fine-grained attribute analysis—category-related semantic judgment." In this way, the model no longer relies solely on surface visual similarity but can more systematically focus on detailed information related to category discrimination during training.

[0079] This embodiment proposes a hierarchical retrieval fusion teacher model: Instead of simply concatenating visual and textual features, the teacher model in this invention alternately performs COT text retrieval and cross-modal fusion across multiple network layers. Visual features are updated at each layer based on the COT semantic information retrieved at the current stage, thus forming a progressively layered visual semantic reasoning process, improving the model's discriminative ability under distributional biases and complex scenarios.

[0080] This embodiment proposes an offline distillation-based student model training mechanism: the invention offline saves the prediction results of unlabeled samples generated by the retrieval-enhanced teacher model as pseudo-labeled logits, and uses these pseudo-labeled logits to supervise the training of the pure vision student model. This mechanism can transfer the COT retrieval-enhanced reasoning ability in the teacher model to the student model, while avoiding the repeated execution of complex text retrieval and cross-modal fusion operations during the student model training and inference stages.

[0081] This embodiment fully utilizes COT text, external implicit text priors, and retrieval-enhanced teacher models during the training phase to improve the quality of unlabeled sample utilization; during the inference phase, only the pure visual student model is retained, and prediction can be completed simply by inputting an image. Therefore, this method can improve the robustness of semi-supervised visual recognition in complex distributed scenarios while meeting the efficiency requirements of practical deployment.

[0082] This embodiment can improve model accuracy and enhance the model's generalization ability to different data distributions and open scenarios when labeled samples are limited, unlabeled data distribution is complex, or there are cross-domain differences.

[0083] Example 2 like Figure 2 As shown, this embodiment of the invention also proposes a visual recognition system based on COT text retrieval enhancement and offline distillation, comprising: Module 1 is used to acquire an implicit text prior database, a set of labeled images, and a set of unlabeled images. The set of labeled images includes labeled images and their real labels, and the set of unlabeled images includes unlabeled images and their corresponding COT texts. The COT texts are subjected to multi-stage semantic decomposition to construct a multi-stage COT text memory. Module 2 is used to build the teacher model, which includes a visual encoder, a multi-layer retrieval fusion module, and a visual recognizer; the number of layers in the multi-layer retrieval fusion module is the same as the number of stages in the multi-stage COT text memory. The visual fusion module 3 is used to extract the initial visual features of the input image based on the visual encoder, obtain the implicit text prior features of the input image from the external implicit text prior database, and obtain the enhanced visual features by superimposing the initial visual features and the implicit text prior features. The enhanced visual features are used as the first layer input visual features of the multi-layer retrieval fusion module. The multi-stage COT text memory is used to perform visual feature fusion on the multi-layer retrieval fusion module. The visual recognizer outputs the recognition result based on the last layer output visual features. Teacher model training module 4; used to supervise the training of the teacher model based on a set of labeled images to obtain a trained teacher model; input unlabeled images into the trained teacher model and output offline pseudo-labels for the unlabeled images; Student Model Training Module 5; used to build a pure vision student model. The student model is trained based on offline pseudo-labels, labeled image sets, and unlabeled images to obtain a visual recognition student model. Recognition module 6 is used for visual recognition based on the trained student model.

[0084] The visual recognition method based on COT text retrieval enhancement and offline distillation proposed in this embodiment of the invention can be executed by a computer program, which is stored in a computer-readable storage medium.

[0085] To verify the performance of this invention in conventional semi-supervised environments, transductive environments, and cross-domain unlabeled data environments, the following experiments were conducted: The classification task experiments were conducted on the CIFAR-10 and CIFAR-100 datasets, with four different semi-supervised learning scenarios. The settings are as follows: Setting-I: This is a traditional semi-supervised learning environment, with only 40 labeled images and 50,000 unlabeled images in CIFAR-10; the corresponding few-label semi-supervised setting is used in CIFAR-100.

[0086] Setting-II: This is a direct-push scenario. In addition to Setting-I, 10,000 test images are added to the unlabeled dataset, allowing the model to access more unlabeled samples during the training phase.

[0087] Setting-III: There are still only 40 labeled images, but all the training set images of the STL-10 dataset are used as unlabeled data to simulate cross-domain scenarios where there are significant differences in the distribution between unlabeled and labeled data.

[0088] Setting-IV: We still use only 40 labeled images from CIFAR-10, while the unlabeled data comes from five datasets: Caltech-101, ImageNet, iNaturalist, STL-10, and CIFAR-100. We randomly select 20,000 images from each dataset to simulate open unlabeled scenes with more complex sources and more diverse distributions.

[0089] The results of the classification task experiment are shown in Table 1 below.

[0090] Table 1. Experimental results of the classification task (accuracy, %)

[0091] As shown in Table 1, on the CIFAR-10 dataset, the method of this invention achieved accuracies of 95.35%, 95.47%, 89.66%, and 88.91% in four settings, respectively. In particular, in settings like Setting-III and Setting-IV where the unlabeled data distribution is more complex, this approach significantly outperforms existing methods. For example, RAVT achieved 87.71% and 83.63% accuracy on CIFAR-10's Setting-III and Setting-IV, respectively, while the method of this invention improved these figures to 89.66% and 88.91%, respectively. This demonstrates that by introducing COT text, this invention enables the teacher model to obtain more hierarchical semantic guidance, thereby generating more reliable unlabeled supervision signals.

[0092] On the CIFAR-100 dataset, the method of this invention achieved accuracies of 63.26%, 62.69%, 61.53%, and 59.44% in four settings, respectively. Compared to RAVT's 61.79%, 61.36%, 58.72%, and 57.47%, this invention achieved higher accuracy in all settings, especially in Settings-III and Settings-IV, where the improvement was more significant. This indicates that even on the CIFAR-100 dataset, which has a larger number of categories and is more difficult to classify, this invention can still maintain strong discriminative ability and stability.

[0093] 1. Improved classification accuracy. This invention achieves superior results compared to existing methods in multiple settings of CIFAR-10 and CIFAR-100, demonstrating that COT text retrieval enhancement and offline distillation mechanisms can effectively improve semi-supervised classification performance.

[0094] 2. More stable performance in cross-domain and complex unlabeled scenarios. In Setting-III and Setting-IV, there are more obvious distribution differences between unlabeled and labeled data. Traditional semi-supervised methods such as FixMatch, FlexMatch, FreeMatch, and SoftMatch show a significant drop in accuracy, while this invention still maintains high accuracy, indicating that it has stronger generalization ability.

[0095] 3. COT text improves the quality of unlabeled data utilization. This invention decomposes COT text into multiple semantic stages and retrieves them through a staged text memory, enabling the teacher model to focus on image information according to a logic from the whole to the details. Compared to methods that only use similar images or ordinary text features, this invention can obtain more fine-grained and structured semantic information.

[0096] 4. Offline distillation ensures deployment efficiency. Although the teacher model uses a COT text retrieval enhancement mechanism during the training phase, only the pure visual student model is retained during final inference, without requiring COT text, text memory, or external retrieval databases. Therefore, this invention improves accuracy without increasing the inference complexity of the final deployment phase.

[0097] Image detection task technical performance In addition to classification tasks, this invention can also be extended to semi-supervised image detection tasks. Detection task experiments were conducted under COCO-Partial 10% settings, and four experimental scenarios, from Setting-I to Setting-IV, were further constructed to verify the effectiveness of this invention in conventional semi-supervised detection scenarios and cross-domain unlabeled detection scenarios. The settings are as follows: Setting-I: This is a traditional semi-supervised image detection environment. In COCO-Partial 10%, only 10% of the labeled images in the COCO training set are used as labeled data, and the remaining unlabeled images are used as unlabeled data.

[0098] Setting-II: This is a direct detection scenario. Building upon Setting-I, it adds test set images to the unlabeled dataset to verify the detection performance of the method after introducing more unlabeled target images.

[0099] Setting-III: Labeled data still comes from the COCO-Partial 10% setting, while unlabeled data uses the OpenImages dataset to simulate a scenario where there is a significant domain difference between unlabeled detection data and labeled COCO data.

[0100] Setting-IV: Labeled data still comes from the COCO-Partial 10% setting, while unlabeled data is composed of multiple datasets such as VOC, Objects365, and Open Images, used to simulate an open unlabeled detection environment with more complex sources and more diverse categories and scene distributions.

[0101] The experimental results of the detection task are shown in Table 2 below.

[0102] Table 2. Experimental results of the image detection task (mAP, %)

[0103] In the detection task, the method of this invention achieved detection performance of 44.7, 45.3, 35.9, and 32.5 under Settings-I, Settings-II, Settings-III, and Settings-IV, respectively. Specifically, under Settings-I, the CoRT result was 44.7, slightly lower than STEP-DETR's 45.4, but still superior to methods such as DINO, Omni-DETR, Semi-DETR, Sparse Semi-DETR, and MixPL. Under Settings-II, CoRT achieved 45.3, superior to STEP-DETR's 45.1, achieving a better result among the currently compared methods.

[0104] More importantly, the advantages of this invention are even more pronounced in cross-domain or more complex unlabeled data settings. In Setting-III, CoRT reaches 35.9, significantly higher than STEP-DETR's 32.8, MixPL's 32.2, and Sparse Semi-DETR's 32.1. In Setting-IV, CoRT reaches 32.5, also significantly higher than STEP-DETR's 27.7, MixPL's 27.1, and Sparse Semi-DETR's 26.5. This demonstrates that in detection scenarios with more complex distributions of unlabeled images, this invention can generate more reliable pseudo-detection targets, thereby effectively improving the training performance of the student detector.

[0105] 1. Improved detection performance. This invention achieves high detection performance under multiple settings of COCO-Partial 10%, especially in Settings-II, Settings-III, and Settings-IV, indicating that the method is not only suitable for classification tasks but can also be effectively transferred to detection tasks.

[0106] 2. Significant advantages in complex unlabeled data scenarios. In Setting-III and Setting-IV, the sources of unlabeled data are more complex, and the performance of traditional semi-supervised detection methods deteriorates significantly. This invention enhances the teacher detection model through COT text retrieval, enabling the generation of higher-quality pseudo-boundaries, category scores, and confidence weights, thereby improving detection results.

[0107] 3. Enhancing the reliability of pseudo-labels in detection tasks. In detection tasks, pseudo-labels contain not only category information but also the location of the bounding box. Compared to classification tasks, detection pseudo-labels are more susceptible to localization errors and category confusion. This invention utilizes COT text and external implicit text priors to enhance the teacher detection model, making the pseudo-detection targets generated by the teacher model more stable and reliable.

[0108] 4. Maintaining the inference efficiency of pure visual detectors. During the training phase, this invention utilizes a retrieval-enhanced teacher detection model to generate offline pseudo-detection targets; during the inference phase, only pure visual student detectors, such as semi-DETR, are retained, eliminating reliance on COT text retrieval and external databases. Therefore, this invention can improve detection performance while maintaining high deployment efficiency.

[0109] In a preferred embodiment of the present invention, a frequency-constrained dual-dictionary competitive sparse representation model for hydrogen refueling station scenarios is established. Through the constructed frequency-constrained dual-dictionary denoising module, deep separation of leakage features and complex background noise within the same frequency band is achieved, improving the feature preservation and noise suppression capabilities of weak leakage signals.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memory, special components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0111] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0112] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A visual recognition method based on COT text retrieval enhancement and offline distillation, characterized in that, include: S1. Obtain the implicit text prior database, the set of labeled images, and the set of unlabeled images; the set of labeled images includes labeled images and their real labels, and the set of unlabeled images includes unlabeled images and their corresponding COT text; Multi-stage semantic decomposition of COT texts is performed to construct a multi-stage COT text memory. S2. Construct a teacher model, including a visual encoder, a multi-layer retrieval fusion module, and a visual recognizer; the number of layers in the multi-layer retrieval fusion module is the same as the number of stages in the multi-stage COT text memory. S3. The visual encoder extracts the initial visual features of the input image, and the external implicit text prior database obtains the implicit text prior features of the input image. The initial visual features and the implicit text prior features are superimposed to obtain the enhanced visual features. The enhanced visual features are used as the first layer of input visual features of the multi-layer retrieval fusion module. The multi-stage COT text memory is used to perform visual feature fusion on the multi-layer retrieval fusion module. The visual recognizer outputs the recognition result based on the visual features of the last layer. S4. Supervised training of the teacher model is performed based on the labeled image set to obtain the trained teacher model; unlabeled images are input into the trained teacher model to output offline pseudo-labels for the unlabeled images; S5. Construct a pure vision student model. Train the student model based on a set of labeled images, unlabeled images, and offline pseudo-labels to obtain a visual recognition student model. S6. Perform visual recognition based on the trained student model.

2. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 1, characterized in that, The construction of the multi-stage COT text memory specifically includes: S101. Perform multi-stage decomposition of COT text, and for unlabeled images... The corresponding COT text is decomposed into multiple stages according to semantic hierarchy, denoted as: in, Indicates the number of semantic stages obtained by decomposing the COT text; Represents text; Indicates the first Zhang unlabeled image in the first COT text fragments corresponding to each semantic stage; S102. Encode the COT text for each stage using a text encoder to obtain the corresponding COT text features: in, Indicates a text encoder. Indicates the first Zhang unlabeled image in the first COT text features at each semantic stage; S103. Construct a phased COT text memory and establish a retrieval index for each semantic phase. The COT text features of all unlabeled images at this stage are stored in the corresponding text memory, resulting in: in, Indicates the first COT text memory corresponding to each semantic stage; This represents the number of unlabeled images. S104, will The individual COT text memories were merged into a multi-stage COT text memory.

3. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 1, characterized in that, The calculation of the enhanced visual features specifically includes: S301. Obtain the implicit text prior database. The image index and its corresponding implicit text features are represented as follows: in, Represents the image index. Represents the implicit text features corresponding to the image; S302. Based on the image index or implicit text features, retrieve the implicit text prior features corresponding to image x from the implicit text prior database, denoted as... ; S303. Input image x into the visual encoder of the teacher model to extract initial visual features: in, Represents a visual encoder. Represents the initial visual features of image x; S304. The retrieved implicit text prior features Mapped to the same dimensional space as the initial visual features: in, Represents a mapping network. Represents the implicit textual prior features after mapping; S305. Map the implicit text prior features Perform average pooling and combine with the initial visual features Adding them together yields the enhanced visual features: in, Indicates average pooling. This indicates an enhancement of visual features.

4. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 2, characterized in that, The step of performing visual feature fusion on the multi-layer retrieval fusion module through a multi-stage COT text memory specifically includes: The teacher model is based on the visual features input to the current layer. Retrieve the multi-stage COT text memory to obtain a set of candidate COT text features. And for the candidate COT text feature set Perform a rating and screening process to obtain the filtered results. Each COT text feature constitutes a filtering set. ; Subscript This indicates the current layer of the multi-layer retrieval fusion module, i.e., the [layer number]. layer; For the filter set In The average aggregation of the COT text features yields the COT semantic guidance features of the current layer in the multi-layer retrieval fusion module. : in, This indicates that the multi-layer retrieval fusion module is in the first... The first layer of the filter set Candidate COT text features The range of values ​​is ; Input visual features into the current layer As a query, COT semantic guidance features As keys and values, they are fused through cross-modal attention to obtain the output visual features of the current layer, which then serve as the input visual features for the next layer. : in, This is a cross-modal attention computation operation.

5. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 4, characterized in that, The candidate COT text feature set The acquisition specifically includes: In the multi-layer retrieval fusion module In each fusion layer, the current stage COT text memory is determined based on the retrieval index: in, This indicates the semantic stage corresponding to the current fusion layer. Indicates the number of COT text stages; Input visual features into the current layer As a query, in the COT Text Memory at Each Stage Similarity retrieval is performed to obtain a set of candidate COT text features. : in, For similarity retrieval.

6. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 4, characterized in that, The set of candidate COT text features Perform a rating and screening process to obtain the filtered results. Each COT text feature constitutes a filtering set. Specifically, it includes: A lightweight scoring network is used to score the text features of candidate COT texts. The scoring formula can be expressed as: in, Represents the text features of candidate COTs. This represents a multilayer perceptron. This represents the Sigmoid function. This represents the relevance score of candidate COT text features, based on which the candidate COT text feature set is selected. Select the top scores in descending order. Each COT text feature is used to obtain the filtered COT text feature set.

7. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 1, characterized in that, In S4: Based on labeled image sets The teacher model is trained under supervision to obtain a fully trained teacher model. ,in: in, Indicates the first Zhang has tagged images, This indicates the true category label corresponding to the image. This indicates the number of labeled images; For image detection tasks, the teacher model outputs the object detection category logit and the predicted object bounding box: in, Indicates the detection category logit, Indicates the predicted target box. Let x represent the teacher model and x represent the input image; The teacher model is trained using the loss functions corresponding to the detection task, including class loss, bounding box regression loss, and GIoU loss: in, Indicates the target category loss. This represents the bounding box regression loss. Indicates GIoU loss, The total loss of the teacher model; Unlabeled images Input the trained teacher model The offline pseudo-labels corresponding to the unlabeled image are obtained, including pseudo-boundary boxes and pseudo-detection category logit. .

8. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 1, characterized in that, In S4: Based on labeled image sets The teacher model is trained under supervision to obtain a fully trained teacher model. ,in: in, Indicates the first Zhang has tagged images, This indicates the true category label corresponding to the image. This indicates the number of labeled images; For classification tasks, the teacher model outputs labeled images. The category logit, and the real category label Calculate cross-entropy loss : in, Represents the cross-entropy loss function; Indicates a labeled image The detection category is logit; Unlabeled images Input the trained teacher model The offline pseudo-labels corresponding to the unlabeled image are obtained, including the pseudo-detection category logit. .

9. The visual recognition method based on COT text retrieval enhancement and offline distillation as described in claim 8, characterized in that, S5 specifically includes: S501, Constructing a Pure Visual Student Model ; S502, The pseudo-detection category logit in the offline pseudo-label is... Converted into teacher pseudo-labels and confidence weights; For unlabeled images Get the corresponding pseudo-detection category logit It is then converted into a class probability distribution using the Softmax function. : The category with the highest probability is selected as the pseudo-label for teachers. : Take the highest class probability as the confidence level. : in, This is the category probability distribution vector; This represents the category confidence level; the subscript 'c' indicates the category. Based on confidence level Calculate sample weights ; S503, Unlabeled images Input student model, get student model output Using pseudo-labels of teachers and confidence weight Calculate the weighted unlabeled loss: Where B is the batch sample size; Labeled images Input student model, get student model output : and with real labels Calculate the labeled supervised loss : S504, Based on labeled image loss and unlabeled image loss Establish a loss function and train the student model: in, The weights represent the unlabeled loss. This represents the total loss of the student model.

10. A visual recognition system based on COT text retrieval enhancement and offline distillation, characterized in that, include: The acquisition module is used to acquire the implicit text prior database, the labeled image set, and the unlabeled image set; the labeled image set includes labeled images and their real labels, and the unlabeled image set includes unlabeled images and their corresponding COT text; Multi-stage semantic decomposition of COT texts is performed to construct a multi-stage COT text memory. The building module is used to construct the teacher model, including a visual encoder, a multi-layer retrieval fusion module, and a visual recognizer; the number of layers in the multi-layer retrieval fusion module is the same as the number of stages in the multi-stage COT text memory. The visual fusion module is used to extract the initial visual features of the input image based on the visual encoder, obtain the implicit text prior features of the input image from the external implicit text prior database, and obtain the enhanced visual features by superimposing the initial visual features and the implicit text prior features. The enhanced visual features are used as the first layer of input visual features of the multi-layer retrieval fusion module, and the multi-stage COT text memory is used to perform visual feature fusion on the multi-layer retrieval fusion module. The visual recognizer outputs the recognition result based on the visual features of the last layer. Teacher model training module; used to supervise the training of the teacher model based on a set of labeled images to obtain a trained teacher model; input unlabeled images into the trained teacher model and output offline pseudo-labels for the unlabeled images; Student model training module; used to build a pure vision student model. The student model is trained based on a set of labeled images, unlabeled images, and offline pseudo-labels to obtain a visual recognition student model. The recognition module is used for visual recognition based on the trained student model.

Images