Convolutional neural network model training method based on contrast depth supervision

Abstract

The invention discloses a convolutional neural network model training method based on contrast depth supervision, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring image data, and performing random data enhancement on an image to obtain a positive sample and a negative sample; carrying out representation extraction; designing and calculating a loss function; model training; and training downstream tasks. According to the method, similarity differences between homologous views and similar views are observed, and the homologous views and the similar views are considered separately by using a comparison module, so that a model is helped to obtain more comprehensive knowledge; according to a proposed contrast depth supervision training framework, homologous and similar losses are used for optimizing an intermediate layer, so that the model is helped to extract better visual representation, and the performance of the model is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a convolutional neural network model training method based on contrastive depth supervision. Background Art

[0002] In the training of deep neural networks, effective supervision and representation learning of intermediate layer features have always been key challenges in improving model performance. Traditional end-to-end training often relies on a global loss signal in the last layer, making it difficult to fully utilize the rich information contained in shallow and intermediate layers. This makes it prone to vanishing or exploding gradients when the network propagates deep into the layers.

[0003] To address these bottlenecks, Deep Supervision inserts auxiliary classifiers in shallow or intermediate layers and applies additional losses, providing multi-level supervisory signals for the entire optimization process. This approach helps alleviate gradient vanishing and accelerates convergence, but it can also lead to a conflict between shallow layers' excessive focus on high-level semantics and low-level feature extraction. Shallow features often overly focus on high-level label information, neglecting low-level visual cues such as local texture and edges in the original image, resulting in the network's inability to capture fine-grained features. Furthermore, Contrastive Learning constructs positive and negative sample pairs, augments the data, and forces different views of the same image (positive samples) to be close to each other in the feature space, while different images (negative samples) remain distant. This approach, in turn, learns discriminative features that are invariant to data augmentation, resulting in excellent performance in unsupervised and semi-supervised scenarios. Combining these two approaches creates Contrastive Deep Supervision (CDS), which employs enhanced contrastive loss for supervision in the intermediate layers. This approach retains the guiding role of deep supervision in feature stratification while allowing shallow layers to focus on learning unbiased, transferable low-level features. This significantly improves the generalization and robustness of the network without increasing inference overhead. Therefore, contrastive deep supervision methods have broad application prospects. However, how to introduce contrastive learning supervision into deep supervision methods and how to capture better intermediate-layer representations have become pressing challenges in this field.

[0004] Existing contrastive deep supervision frameworks directly use contrastive learning losses from existing contrastive learning methods such as SupCon to supervise the network's intermediate layers. SupCon is a supervised contrastive learning method that has been particularly effective in improving classification performance. Unfortunately, existing contrastive deep supervision frameworks have not delved deeply into contrastive learning methods. From the perspective of contrastive learning theory, supervised contrastive learning expects augmented samples from the same category to exhibit high consistency in feature space. However, in reality, low-level semantic information (such as color and texture) within images of the same category can vary significantly. For example, although shelled and unshelled corn are classified as the same category, their visual features differ significantly. This difference can lead to different knowledge being acquired by the model during feature extraction. Specifically, maximizing the similarity between homologous augmented representations (i.e., augmented samples from the same original image) generally leads to a model that is more inclined to learn invariance to various data augmentations. In contrast, maximizing the similarity between homologous augmented representations (i.e., augmented samples from the same category) focuses more on learning common features among similar samples.

[0005] However, during the training process of existing contrastive deep supervision methods, SupCon pulls representations of the same category together and pushes representations of different categories apart, thereby considering these two similarity losses together. This implementation causes the model to focus more on learning similarities between similar samples while ignoring key knowledge between homologous samples. Furthermore, this design of SupCon causes each enhanced sample within the same category to ultimately present an overly consistent feature table, significantly reducing the diversity and quality of the representations. This negative impact is particularly evident in the shallow feature extraction stage. Therefore, when applying SupCon as a supervisory signal in the intermediate layer, a more sophisticated balance and design are required to fully utilize different types of similarity information. Summary of the Invention

[0006] In order to make up for the shortcomings of the existing technology, the purpose of the present invention is to provide a technical solution for a convolutional neural network model training method based on contrastive deep supervision. By designing a new contrastive deep supervision loss and contrast module, homologous representations and similar representations are considered separately, which reduces the interference of similar loss on homologous loss, so that the middle layer of the network learns more comprehensive knowledge, thereby significantly improving the representation quality of the middle layer; it can not only significantly improve the performance of the model, but also help the network extract more generalized representations, which can be widely used in downstream tasks such as fine-grained image classification and target detection.

[0007] The technical problem solved by the present invention can be achieved through the following specific technical solutions:

[0008] The convolutional neural network model training method based on contrastive deep supervision includes the following steps:

[0009] Step 1: Collect image data and perform random data augmentation on the image to obtain homologous sample views and similar sample views;

[0010] Step 2: Input the enhanced view into the deep supervision encoder to extract the representation of the intermediate layer and the final task representation;

[0011] Step 3: Use the intermediate layer representation to calculate the homology loss and the same type loss, and use the task representation to calculate the task loss, thus completing the design and calculation of the loss function;

[0012] Step 4: Calculate the gradient based on the loss obtained, and use the stochastic gradient descent (SGD) optimizer to perform gradient optimization to achieve model training;

[0013] Step 5: Based on the trained contrastive deep supervision encoder, replace the task module and perform fine-tuning to achieve model training for downstream tasks.

[0014] Furthermore, in step 1, in the image set consisting of N images, for any image x i , two enhanced views with differences but still retaining the original key information are generated through two data enhancement techniques, so that the image set is finally expanded to 2N views, recorded as {x1,x2,…,x N ,x N+1 ,…,x 2N}, provide samples for subsequent contrastive learning tasks; i and x N+i They are regarded as two enhanced views from the same image and are considered as a pair of homologous positive samples, while enhanced views from the same class are regarded as similar positive samples; in addition, enhanced views from different classes are regarded as negative samples.

[0015] Furthermore, in step 2, the specific steps of characterization extraction are as follows:

[0016] Step S21: Divide the convolutional neural network encoder into K consecutive layers, namely:

[0017]

[0018] Among them, the symbol Represents the coincidence operation of the function, F i represents the encoder of the i-th layer, i = 1, 2, 3... K-1, K; on this basis, a contrast module is added after each layer to enable the homology loss and contrast loss to independently supervise each level;

[0019] Step S22: In each comparison module, use Q k , G k and H kTo represent the different components in the k-th layer comparison module, where Q k represents the convolutional projection layer, which is used to reduce the dimension and map the output representation of the kth layer; G k It is a projection multi-layer perceptron MLP, which is used to further refine and map these features; H k Represents the prediction multi-layer perceptron MLP, which is used to generate the prediction vector for contrast loss calculation; the classifier C of the entire network is in the last layer, and the output features obtained by the convolutional neural network encoder are mapped to specific categories;

[0020] Step S23: For any data-enhanced view x i , which generates a series of normalized output representations when passing through the k-th layer comparison module of the convolutional neural network encoder. Specifically, this process outputs three feature representations, denoted as q k,i 、g k,i and h k,i , where q k,i Indicates that the output of the kth layer passes through the convolutional projection layer Q k The feature representation of g k,i It is the feature after further extraction and mapping by the projection multi-layer perceptron MLP, and h k,i is the prediction feature generated by the prediction multi-layer perceptron MLP; all these outputs can ensure the consistency of feature scale after normalization, which is conducive to the subsequent similarity calculation and optimization of contrast loss. The entire extraction process can be expressed by the following formula:

[0021]

[0022] Step S24, the classifier C in the last layer of the network is responsible for generating the final feature representation c i , and its extraction process is expressed as:

[0023]

[0024] Furthermore, in step 3, the deep supervision process is divided into two independent tasks, including learning for homologous samples and similar samples respectively. The similar loss function and the homologous loss function are as follows:

[0025]

[0026] Where τ represents the temperature hyperparameter; express The number of elements in the collection; is an index set containing the indices of all enhanced images except i and N+i; minimize the homology loss Use h directly N+i To predict gi , It can be expressed as:

[0027]

[0028] Among them, sg(·) means stopping gradient calculation and backpropagation, g i and g N+i is considered a constant.

[0029] Furthermore, in step 4, all losses are combined to obtain the total loss, and the intermediate layers and the overall objective are trained independently. The specific formula is as follows:

[0030]

[0031] Among them, λ1 and λ2 are hyperparameters; Represents the cross entropy loss function, which only supervises the model in the last layer of the model; It is the comparative supervision loss function of K intermediate layers, including homology loss and similarity loss.

[0032] Furthermore, in step 5, training the downstream task includes the following two complementary training strategies:

[0033] Strategy 1: End-to-end training of the backbone network with random initialization. The backbone network is randomly initialized from scratch. During training, a contrastive deep supervision framework is introduced. By constructing pairs of homologous and similar samples at different layers, the network parameters are jointly optimized by weighted summation of homologous and similar contrastive losses and the main task loss. Data augmentation and learning rate scheduling are also implemented.

[0034] Strategy 2: First, pre-train the network using a contrastive deep supervision framework on a large-scale general image dataset to obtain robust network weights. Then, the pre-trained weights are loaded into the target downstream task model, and the freezing and unfreezing strategies and layer-wise learning rates are set as needed to fine-tune the newly added task head and backbone network respectively.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] (1) The present invention observes the similarity differences between homologous views and similar views, and uses a comparison module to consider homologous views and similar views separately, helping the model to obtain more comprehensive knowledge; among them, the proposed contrastive deep supervision training framework uses homologous and similar losses to optimize the intermediate layers to help the model extract better visual representations, significantly improving model performance.

[0037] (2) The present invention uses a variety of convolutional neural network architectures and the experimental results on multiple public data prove that the models of this application exceed the current latest deep supervision models.

[0038] (3) By using a variety of model architectures, the paper demonstrates superior performance in comparative experiments on multiple public datasets and typical downstream tasks. This not only improves the quality of intermediate layer features, but also enhances the network's ability to capture detailed information, providing a more solid and transferable representation foundation for complex visual tasks, and has broad engineering applications and research prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural diagram of the model of the present invention;

[0040] Figure 2 This is a structural block diagram of the comparison module of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] This application is based on a convolutional neural network model training method using contrastive deep supervision, which includes the following steps:

[0043] (1) Collect image data and perform random data enhancement on the image to obtain homologous sample views and similar sample views.

[0044] In a small batch of N images, for any image x i By using multiple data enhancement techniques such as random cropping, rotation, and color jittering twice, two enhanced views with differences but still retaining the original key information are generated, so that the entire mini-batch data is finally expanded to 2N views, denoted as {x1,x2,…,x N ,x N+1 ,…,x 2N}, providing rich samples for subsequent contrastive learning tasks. i and x N+i They are considered as two augmented views from the same image and are treated as a pair of homologous positive samples, while augmented views from the same class are considered as homologous positive samples. In addition, augmented views from different classes are considered as negative samples.

[0045] (2) Representation extraction: The enhanced view input is compared with the deep supervision encoder to extract the representation of the intermediate layer and the final task representation.

[0046] The present invention is applicable to any convolutional neural network architecture and can improve the model's feature representation ability and final performance. For ease of explanation, the classic ResNet series network is used as an example for detailed description. Figure 1The following is the overall framework of the model. First, the convolutional neural network encoder is divided into K consecutive layers, namely:

[0047]

[0048] Among them, the symbol Represents the coincidence operation of the function, F i represents the encoder of the i-th layer, i = 1, 2, 3... K-1, K; on this basis, a contrast module is attached after each layer to enable the homology loss and contrast loss to independently supervise each level.

[0049] like Figure 2 The structure of the comparison module is shown as follows: In each comparison module, Q k , G k and H k To represent the different components in the k-th layer comparison module. k represents the convolutional projection layer, which is used to reduce the dimension and map the output representation of the kth layer; G k is a projected multi-layer perceptron (MLP) to further refine and map these features; H k It represents the prediction of the multi-layer perceptron (MLP), which is used to generate the prediction vector for contrast loss calculation. Finally, the classifier C of the entire network is at the last layer, which uses the output features obtained by the convolutional neural network encoder to map to specific categories.

[0050] For any data-augmented view x i , which generates a series of normalized output representations when passing through the k-th layer comparison module of the convolutional neural network encoder. Specifically, this process outputs three feature representations, denoted as q k,i 、g k,i and h k,i , where q k,i Indicates that the output of the kth layer passes through the convolutional projection layer Q k The feature representation of g k,i is the feature after further extraction and mapping by projection MLP, and h k,i is the prediction feature generated by the prediction MLP; all these outputs can ensure the consistency of feature scale after normalization, which is beneficial for subsequent similarity calculation and optimization of contrast loss. The entire extraction process can be expressed as follows:

[0051]

[0052] In addition, the classifier C in the last layer of the network is responsible for generating the final feature representation c i , and its extraction process can be expressed as:

[0053]

[0054] (3) Design and calculation of loss function: Use the intermediate layer representation to calculate the homology loss and the same type loss, and use the task representation to calculate the task loss, thereby completing the design and calculation of the loss function.

[0055] This paper deeply explores the problems of supervised contrast loss used as the supervision signal of the intermediate layer in existing methods, and conducts detailed derivation and analysis from the perspective of gradient contribution. It is found that the gradient contribution of homologous samples to the supervised contrast loss is very small. In order to reduce the influence of similar positive samples on the gradient of homologous positive samples, this paper divides the deep supervision process into two independent tasks, including learning for homologous samples and similar samples respectively. Such a separation strategy not only helps the intermediate layer to extract features more finely, but also improves the generalization ability and overall performance of the model in various visual tasks. Next, the similar loss function and the homologous loss function will be introduced in detail:

[0056]

[0057] Where τ represents the temperature hyperparameter; express The number of elements in the collection; Is an index set that contains the indexes of all enhanced images except i and N+i. In fact, the same loss is a supervised contrast loss The supervised contrast loss is a variant of the supervised contrast loss, but there is a key difference between the two. Specifically, the supervised contrast loss considers homologous representations as part of the same-category representation during calculation. This causes the model to prefer learning common features between similar samples in intermediate layers, thereby ignoring the invariance of learning data augmentation contained within homologous samples. In contrast, the same-category loss function only considers representations belonging to the same category across different image augmentations, intentionally excluding the participation of homologous representations.

[0058] In order to more fully learn the invariance features brought by data enhancement, this application separates the learning of homologous representations and minimizes the homologous loss. To learn such low-level features that are task-independent and transferable to various visual tasks. More specifically, Use h directly N+i To predict g i To avoid using other similar samples or negative samples to represent, It can be expressed as:

[0059]

[0060] Among them, sg(·) stops gradient calculation and backpropagation, g i and gN+i is considered a constant.

[0061] (4) Model training: Gradient calculation is performed based on the obtained loss, and the stochastic gradient descent (SGD) optimizer is used to perform gradient optimization to achieve model training.

[0062] The present invention combines all losses to obtain the total loss, and independently trains the intermediate layers and the overall objective. The specific formula is as follows:

[0063]

[0064] Among them, λ1 and λ2 are hyperparameters; Represents the cross entropy loss function, which only supervises the model in the last layer of the model; the second term It is the comparative supervision loss function of K intermediate layers, including homology loss and similarity loss.

[0065] (5) Training downstream tasks: Based on the trained contrastive deep supervision encoder, the task module is replaced and fine-tuned to achieve model training for downstream tasks.

[0066] The training method proposed in this paper can be flexibly applied to various downstream visual task models based on convolutional neural networks, such as fine-grained image classification and object detection, significantly improving the model's feature expression capabilities and overall performance. In specific implementation, the following two complementary training strategies can be selected based on actual needs.

[0067] Strategy 1: End-to-end training of the backbone network is performed using random initialization. The backbone network is randomly initialized from scratch. During training, a contrastive deep supervision framework is introduced. By constructing pairs of homologous and similar samples at different layers, network parameters are jointly optimized using a weighted summation of homologous and similar contrastive losses with the primary task loss (such as cross-entropy loss, focal loss, and smooth L1 loss). Data augmentation methods such as multi-scale cropping, color jittering, MixUp, and CutMix, as well as learning rate scheduling, are implemented to improve the network's ability to discern fine-grained features and generalize performance.

[0068] Strategy 2: First, pre-train the network using a contrastive deep supervision framework on a large-scale general-purpose image dataset (e.g., ImageNet) to obtain robust network weights. These pre-trained weights are then loaded into the target downstream task model. The freezing and unfreezing strategies and layer-wise learning rates are then configured as needed to fine-tune the newly added task heads (classification, detection, or segmentation) and the backbone network, significantly shortening training time and reducing the reliance on labeled data. This approach enables efficient, robust, and high-performance model training for a variety of downstream vision tasks, including fine-grained image classification, object detection, semantic segmentation, and instance segmentation.

[0069] Verification Example

[0070] (1) Dataset

[0071] The invention uses the following data sets to verify its effectiveness:

[0072] The CIFAR-10 dataset is a classic benchmark in image classification. It contains 60,000 32×32 color images classified into 10 categories: airplane, car, bird, cat, deer, dog, frog, horse, boat, and truck, with 6,000 images per category. Of these, 50,000 images are used for training and 10,000 for testing. The dataset is divided into five training batches and one test batch, with 10,000 images per batch. Due to its low-resolution and well-balanced image quality, CIFAR-10 is widely used to evaluate and compare the performance of various machine learning and computer vision algorithms.

[0073] The CIFAR-100 dataset is a classic benchmark for image classification. It contains 60,000 32×32 color images, covering 100 subcategories and 20 supercategories, with 600 images per category. Each image is assigned both a fine label and a coarse label, with 50,000 images in the training set and 10,000 in the test set. This hierarchical structure provides a benchmark for fine-grained and supercategorical classification in machine learning algorithms.

[0074] ImageNet-1K (a subset of ILSVRC 2012) is the most widely used large-scale visual recognition benchmark. Based on the WordNet hierarchy, it covers 1,000 object categories. The dataset contains 1,281,167 training images, 50,000 validation images, and 100,000 test images, all of which have been manually quality-controlled and annotated. This dataset facilitates the performance evaluation of deep learning and computer vision algorithms and has become a standard for model pre-training and performance comparison.

[0075] The MS COCO 2017 dataset, jointly released by Microsoft and the University of Berkeley, is a public benchmark for object detection, instance segmentation, keypoint detection, and image captioning. The dataset contains 118,000 training images, 5,000 validation images, and 20,000 test images, covering 80 object categories and over 2,099,063 annotated instances, providing bounding boxes, segmentation masks, and keypoint annotations. The rich collection of everyday scene images and diverse annotations facilitates the evaluation and comparison of computer vision models.

[0076] The liquid level detection dataset is a custom-built dataset for supernatant detection. It contains 2,000 images of liquid surfaces after centrifugation, including 1,000 images without supernatant and 1,000 images with supernatant. All images are manually annotated with supernatant bounding boxes and support COCO / YOLO formats. The images have a resolution of 1920×1080 and cover a variety of samples, including blood and cell culture supernatants. The dataset is divided into a training set of 1,600 images, a validation set of 200 images, and a test set of 200 images, providing high-quality training and evaluation material for liquid level detection algorithms.

[0077] (2) Evaluation indicators

[0078] In order to comprehensively evaluate the performance of the model, this paper adopts the following commonly used evaluation indicators:

[0079] Accuracy is the ratio of the number of samples correctly predicted by the model to the total number of samples, which can be expressed as (TP+TN) / (TP+TN+FP+FN); Top-1 accuracy refers to the ratio of the highest-probability predicted labels by the model to the true labels; Top-5 accuracy is used to evaluate the performance of multi-category tasks when the true label appears in the top five predicted probabilities by the model.

[0080] In object detection, the core indicator for evaluating model performance is Average Precision (AP) and its various variants, such as AP at different IoU thresholds and AP based on target size (APS, APM, APL). AP comprehensively measures the detection quality of the model at different confidence thresholds by calculating the area under the Precision-Recall curve. The COCO challenge further averages the results over multiple IoU thresholds (0.5 to 0.95, with a step size of 0.05) to obtain a more robust AP indicator. In addition, AP is calculated separately for small, medium, and large targets. S (Small), AP M (Medium) and AP L (Large), which can evaluate the model's ability to detect objects of different scales.

[0081] (3) Benchmark Model

[0082] The model of the present invention is compared with the following deep supervision models: DSN not only implements supervision at the output layer, but also introduces adjoint loss to supervise the hidden layer, which makes the learning process of the hidden layer more transparent. DKS introduces collaborative loss, which integrates pairwise knowledge matching between all supervision branches. DHM proposes a dynamic collaborative strategy to optimize the interaction between the backbone network and supervision branches at different layers. CDS simply applies contrastive learning loss to the intermediate layer, alleviating the impact of high-level semantic information on the shallow network. In the experiment, a variety of mainstream convolutional neural network architectures were selected, including ResNet (RST), ResNeXt (RXT), Wide ResNet (WRN), SENet (SET) and PreAct ResNet (PRN) to ensure that the results are widely applicable and reliable.

[0083] (IV) Experimental results and analysis

[0084] (1) Experimental results

[0085] Table 1 Comparison of image classification performance on the CIFAR-100 dataset and other deep supervision methods (Top-1 accuracy / %)

[0086]

[0087] Table 2 Comparison of image classification performance on CIFAR-10 dataset and other deep supervision methods (Top-1 accuracy / %)

[0088]

[0089] Table 3 Comparison of image classification performance on the ImageNet dataset with other deep supervision methods (accuracy / %)

[0090]

[0091] Experimental results on the CIFAR100, CIFAR10, and ImageNet datasets are shown in Tables 1, 2, and 3, respectively. The image classification performance of our method is compared with other deep-supervised methods using various neural network architectures. It can be seen that our method significantly outperforms previous methods, achieving average improvements of 5.68% and 2.30% on CIFAR100 and CIFAR10, respectively. Compared to the second-best deep-supervised method, the top-1 classification accuracy improves by an average of 2.3% and 0.60%. Furthermore, the evaluation results for CIFAR10 are presented in Table 2. These results show that our method can achieve an average classification accuracy improvement of 0.6% over CDS. On ImageNet, our method improves the top-1 accuracy of ResNet-18, ResNet-34, and ResNet-50 by 4.37%, 4.14%, and 4.22%, respectively. On average, it outperforms the baseline method and the second-best method by 4.24% and 1.04% in top-1 accuracy.

[0092] Table 4 Comparison of ResNet50 as the backbone network with other comparative learning methods on ImageNet

[0093]

[0094] The detailed experimental results on the ImageNet dataset using ResNet50 as the backbone network are shown in Table 4. In this experiment, the proposed method is comprehensively compared with the current state-of-the-art contrastive learning and deep supervision methods. These methods include MOCO, SimCLR, Asym-Siam, CaCo, BYOL, and SupCon. Among them, the method marked as “ 1 "The method means that the backbone network is pre-trained using the corresponding contrastive learning method, and then its parameters are frozen, and only the final classification head is fine-tuned; the method marked as " 2 " is to pre-train the backbone network using the corresponding contrastive learning method, and then fine-tune the entire network; while the method marked as " 3 The method adopts a one-stage deep supervision strategy, directly combining contrastive learning with supervision signals, rather than the traditional two-stage pre-training + fine-tuning process.

[0095] The experimental data show that our method achieves the highest classification accuracy using the smallest batch size and the fewest training rounds, significantly outperforming other state-of-the-art contrastive learning and deep supervision methods. This result demonstrates the superior performance of the proposed strategy on large-scale datasets.

[0096] Table 5: Fine-grained classification experiments comparing models trained from scratch using ResNet50 and deep supervision methods

[0097]

[0098] Table 6 Comparison of deep supervision methods for fine-grained classification with models fine-tuned based on ImageNet pre-trained weights

[0099]

[0100] Experimental results for fine-grained image classification are shown in Table 6. It can be seen that compared to deep supervision, a consistent and significant improvement in accuracy is achieved across the five datasets. On average, it improves accuracy by 4.96%, 3.56%, 1.89%, 6.76%, and 4.53% on the five datasets, respectively. Furthermore, our method achieves similar performance in both fine-tuning from ImageNet and training from scratch, demonstrating the consistent effectiveness of our approach across different training environments.

[0101] Table 7 Experimental results of object detection using ResNet50 pre-trained on ImageNet and then fine-tuned on COCO2017

[0102]

[0103] Table 7 demonstrates the performance of our method for object detection. In these experiments, ResNet50 was first pre-trained on ImageNet using standard training (Baseline), four deep supervision methods, and our method, and then fine-tuned as the backbone network for object detection models. Results for Faster RCNN and RetinaNet on the COCO2017 dataset show that using our method to pre-train the backbone network improves the AP of Faster RCNN and RetinaNet by 1.8 and 1.65, respectively, outperforming the next-best method by 0.9 and 0.84, respectively. This demonstrates that the representations learned by our method are more beneficial for downstream tasks.

[0104] Table 8 Experimental results of target detection using ResNet50 as the backbone network on the liquid level detection dataset

[0105]

[0106]

[0107] Table 8 demonstrates the performance of the proposed method in real-life applications. Using the Faster RCNN and YOLO models for target detection on a liquid level detection dataset, the proposed method achieves improvements of 2.1 and 2.6 mAP, respectively, compared to the baseline model, and 1.2 and 1.4 mAP, respectively, compared to other deep supervision methods. This demonstrates that the proposed method can be applied in real-life applications and significantly improves performance.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A convolutional neural network model training method based on contrastive deep supervision, characterized in that: The following steps are involved: Step 1: Collect image data and perform random data augmentation on the image to obtain homologous sample views and similar sample views; Step 2: Input the enhanced view into the deep supervision encoder to extract the representation of the intermediate layer and the final task representation; Step 3: Use the intermediate layer representation to calculate the homology loss and the same type loss, and use the task representation to calculate the task loss, thus completing the design and calculation of the loss function; Step 4: Calculate the gradient based on the loss obtained, and use the stochastic gradient descent (SGD) optimizer to perform gradient optimization to achieve model training; Step 5: Based on the trained contrastive deep supervision encoder, replace the task module and perform fine-tuning to achieve model training for downstream tasks.

2. The convolutional neural network model training method based on contrastive deep supervision according to claim 1, characterized in that In step 1, in the image set consisting of N images, for any image x i , two enhanced views with differences but still retaining the original key information are generated through two data enhancement techniques, so that the image set is finally expanded to 2N views, recorded as {x1,x2,…,x N ,x N+1 ,…,x 2N }, provide samples for subsequent contrastive learning tasks; i and x N+i They are regarded as two enhanced views from the same image and are considered as a pair of homologous positive samples, while enhanced views from the same class are regarded as similar positive samples; Furthermore, augmented views from different categories are considered as negative samples.

3. The convolutional neural network model training method based on contrastive deep supervision according to claim 1, characterized in that In step 2, the specific steps of characterization extraction are as follows: Step S21: Divide the convolutional neural network encoder F into K consecutive layers, namely: Among them, the symbol Represents the coincidence operation of the function, F i represents the encoder of the i-th layer, i = 1, 2, 3... K-1, K; on this basis, a contrast module is added after each layer to enable the homology loss and contrast loss to independently supervise each level; Step S22: In each comparison module, use Q k , G k and H k To represent the different components in the k-th layer comparison module, where Q k represents the convolutional projection layer, which is used to reduce the dimension and map the output representation of the kth layer; G k It is a projection multi-layer perceptron MLP, which is used to further refine and map these features; H k Represents the prediction multi-layer perceptron MLP, which is used to generate the prediction vector for contrast loss calculation; the classifier C of the entire network is in the last layer, and the output features obtained by the convolutional neural network encoder are mapped to specific categories; Step S23: For any data-enhanced view x i , which generates a series of normalized output representations when passing through the k-th layer comparison module of the convolutional neural network encoder. Specifically, this process outputs three feature representations, denoted as q k,i 、g k,i and h k,i , where q k,i Indicates that the output of the kth layer passes through the convolutional projection layer Q k The feature representation of g k,i It is the feature after further extraction and mapping by the projection multi-layer perceptron MLP, and h k,i is the prediction feature generated by the prediction multi-layer perceptron MLP; all these outputs can ensure the consistency of feature scale after normalization, which is conducive to the subsequent similarity calculation and optimization of contrast loss. The entire extraction process can be expressed by the following formula: Step S24, the classifier C in the last layer of the network is responsible for generating the final feature representation c i , and its extraction process is expressed as:

4. The convolutional neural network model training method based on contrastive deep supervision according to claim 1, characterized in that In step 3, the deep supervision process is divided into two independent tasks, including learning for homologous samples and similar samples respectively. The similar loss function and homologous loss function are as follows: Where τ represents the temperature hyperparameter; express The number of elements in the collection; is an index set containing the indices of all enhanced images except i and N+i; minimize the homology loss Use h directly N+i To predict g i , It can be expressed as: Among them, sg(·) means stopping gradient calculation and backpropagation, g i and g N+i is considered a constant.

5. The convolutional neural network model training method based on contrastive deep supervision according to claim 1, characterized in that In step 4, all losses are combined to get the total loss, and the intermediate layers and the overall target are trained independently. The specific formula is as follows: Among them, λ1 and λ2 are hyperparameters; Represents the cross entropy loss function, which only supervises the model in the last layer of the model; It is the comparative supervision loss function of K intermediate layers, including homology loss and similarity loss.

6. The convolutional neural network model training method based on contrastive deep supervision according to claim 1, characterized in that: In step 5, training the downstream task includes the following two complementary training strategies: Strategy 1: End-to-end training of the backbone network with random initialization. The backbone network is randomly initialized from scratch. During training, a contrastive deep supervision framework is introduced. By constructing pairs of homologous and similar samples at different layers, the network parameters are jointly optimized by weighted summation of homologous and similar contrastive losses and the main task loss. Data augmentation and learning rate scheduling are also implemented. Strategy 2: First, pre-train the network using a contrastive deep supervision framework on a large-scale general image dataset to obtain robust network weights. Then, the pre-trained weights are loaded into the target downstream task model, and the freezing and unfreezing strategies and layer-wise learning rates are set as needed to fine-tune the newly added task head and backbone network respectively.

Images