Training method and application of long-tail data semi-supervised segmentation model based on double teachers

Abstract

The invention discloses a training method and application of a long-tail data semi-supervised segmentation model based on double teachers. The training method comprises the following steps: enabling labeled data input to calculate supervised loss; inputting label-free data to obtain general identification student prediction features, professional student prediction features, general identification false labels and professional false labels; calculating a first cross entropy loss between the general recognition student prediction feature and the professional pseudo tag, and a second cross entropy loss between the professional student prediction feature and the general recognition pseudo tag; calculating the comparative learning loss between the general recognition teacher false label and the general recognition student prediction features; and iteratively updating the parameters. The method is composed of different learning modules and has different pseudo label screening strategies, so that model coupling can be avoided; only some additional parameters are introduced, so that the data occupation is small; and furthermore, a more discriminative feature space is formed through comparative learning, so that the utilization rate of the pseudo labels is remarkably improved while overfitting is avoided, and finally, the model training efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and machine learning, and particularly relates to a training method and application of a long-tail data semi-supervised segmentation model based on double teachers. BACKGROUND

[0002] The purpose of semantic segmentation is to assign a label to each pixel of a given image, which is a basic task in the field of computer vision. Recently, it has been widely used in many robot systems, such as autonomous vehicles, robots, etc. However, it is very time-consuming to collect a large number of pixel-level labels. In order to take advantage of the unlabeled data, semi-supervised learning framework has been studied, which can train a baseline model with only a small amount of labeled data and a large amount of unlabeled data.

[0003] Recent progress in semi-supervised semantic segmentation has mainly benefited from dual models with consistency loss. This loss function encourages sub-models to produce similar predictions when inputting different images. Perturbations include image augmentation, feature perturbation and network perturbation. However, dual models cause double costs in time and space during the learning process, which prevents them from being applied in robot systems with valuable computing and memory resources.

[0004] In addition, semi-supervised learning mainly trains an AI model with strong generalization ability by combining a small amount of labeled data and a large amount of unlabeled data. Generally, semi-supervised semantic segmentation methods generate pseudo labels for unlabeled data, and use entropy minimization of pseudo labels to further guide model training. However, since the generated pseudo labels often have noise, this will cause the model to overfit different noises in the later training period. In this regard, some methods use a higher threshold to filter the pseudo labels, and retain pseudo labels with higher confidence. Although this operation filters out most of the noise labels, it greatly reduces the utilization rate of unlabeled data, so these methods also have limitations. SUMMARY

[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a training method and application of a long-tail data semi-supervised segmentation model based on double teachers.

[0006] To achieve the foregoing application purpose, the technical scheme adopted by the present application comprises:

[0007] In a first aspect, the present application provides a semi-supervised semantic segmentation model training method based on double teachers, which comprises:

[0008] The initial model and training data are provided, the initial model includes a general student model, a general teacher model, a professional student model and a professional teacher model, and the training data includes labeled data and corresponding true value labels, and unlabeled data;

[0009] The labeled data is input into the initial model to obtain labeled prediction features, and a supervised loss is calculated based on the corresponding true value labels;

[0010] The unlabeled data is input into the initial model, the general student prediction features are obtained by using the general student model, the professional student prediction features are obtained by using the professional student model, the general pseudo-labels are predicted by using the general teacher model, and the professional pseudo-labels are predicted by using the professional teacher model;

[0011] The first cross-entropy loss between the general student prediction features and the professional pseudo-labels, and the second cross-entropy loss between the professional student prediction features and the general pseudo-labels are calculated;

[0012] The contrastive learning loss between the general teacher pseudo-labels and the general student prediction features is calculated;

[0013] Based on the supervised loss, the first cross-entropy loss, the second cross-entropy loss and the contrastive learning loss, the parameters of the initial model are iteratively updated.

[0014] In a second aspect, the present application further provides a semantic segmentation model trained by the above-mentioned semi-supervised semantic segmentation model training method, and the semantic segmentation model is used for semantic segmentation of an image.

[0015] In a third aspect, the present application further provides a semi-supervised semantic segmentation model training system based on double teachers, which comprises:

[0016] A supervised loss module, a general education module, a professional education module, a contrastive learning module and an iterative training module;

[0017] The general education module includes a general student model and a professional teacher model, which is used for supervised training of the general student model by taking the prediction provided by the professional teacher model as a pseudo-label, and calculating a first cross-entropy loss; the professional education module includes a general teacher model and a professional student model, which is used for supervised training of the professional student model by taking the prediction provided by the general teacher model as a pseudo-label, and calculating a second cross-entropy loss;

[0018] The contrastive learning module is used for contrastive learning of the prediction of the general teacher model and the output features of the general student model, and calculating a contrastive learning loss;

[0019] The supervised loss module is configured to input the labeled data into the initial model to obtain labeled predicted features, and calculate a supervised loss based on the corresponding true value labels.

[0020] The iterative training module is configured to iteratively update parameters of the initial model based on the supervised loss, the first cross-entropy loss, the second cross-entropy loss, and the contrastive learning loss.

[0021] In a fourth aspect, the present application further provides a readable storage medium, which stores a computer program, and the computer program is configured to execute the steps of the above-mentioned semi-supervised semantic segmentation model training method, or stores the above-mentioned semantic segmentation model.

[0022] Based on the above technical solutions, compared with the prior art, the present application has at least the following beneficial effects:

[0023] The training method provided by the present application is composed of different learning modules, has different pseudo-label screening strategies, and can avoid model coupling; compared with the baseline method in the prior art, the module only introduces some additional parameters, and the data occupation is small; and through contrastive learning, a more discriminative feature space is formed, which can avoid overfitting while significantly improving the pseudo-label utilization rate, and finally improves the model training efficiency and accuracy.

[0024] The above description is only a summary of the technical solutions of the present application, in order to enable those skilled in the art to more clearly understand the technical means of the present application, and can be implemented according to the content of the specification, as follows. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of the training method of the semantic segmentation model provided by a typical embodiment of the present application.

[0026] Figure 2 is a system structure and process schematic diagram of the training method of the semantic segmentation model provided by a typical embodiment of the present application. DETAILED DESCRIPTION

[0027] In view of the deficiencies in the prior art, the present inventors have long studied and practiced to propose the technical solutions of the present application. The technical solutions, implementation processes and principles will be further explained as follows.

[0028] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0029] Moreover, the terminology "first" and "second", and the like, are used merely to distinguish one from another of a similar name, and do not necessarily require or imply any such actual relationship or order between such referenced items.

[0030] Referring to Figure 1 and Figure 2 An embodiment of the present application provides a long-tail data semi-supervised semantic segmentation model training method based on double teachers, which comprises the following steps:

[0031] An initial model and training data are provided, the initial model comprising a general student model, a general teacher model, a professional student model, and a professional teacher model; the training data comprising labeled data and corresponding true value labels, and unlabeled data;

[0032] The labeled data is input into the initial model to obtain labeled prediction features, and a supervised loss is calculated based on the corresponding true value labels;

[0033] The unlabeled data is input into the initial model, general student prediction features are obtained using the general student model, professional student prediction features are obtained using the professional student model, general pseudo labels are predicted using the general teacher model, and professional pseudo labels are predicted using the professional teacher model;

[0034] A first cross-entropy loss between the general student prediction features and the professional pseudo labels, and a second cross-entropy loss between the professional student prediction features and the general pseudo labels are calculated;

[0035] A contrastive learning loss between the general teacher pseudo labels and the general student prediction features is calculated;

[0036] Based on the supervised loss, the first cross-entropy loss, the second cross-entropy loss, and the contrastive learning loss, the parameters of the initial model are iteratively updated.

[0037] As some typical application examples of the above technical solutions, the execution steps of the above method may comprise, for example:

[0038] 1) Obtain the base models and training sets of the general education module and the professional education module, the training sets comprising labeled data and corresponding true value labels, and unlabeled data.

[0039] 2) Input any of the unlabeled data into the general education module and the professional education module respectively, obtain two types of pseudo labels and perform pseudo label screening, and calculate the unsupervised loss values of the general student and the professional student based on the pseudo labels respectively.

[0040] 3) input any of the labeled data into the model to obtain the corresponding true value label, and calculate a supervised loss value.

[0041] 4) compare the output features of the general student with the predicted pseudo-labels of the general teacher to learn by comparison, and calculate a comparison learning loss value.

[0042] 5) iteratively update the parameters of the model based on the main supervised loss value and the main unsupervised loss value and based on the comparison learning loss value.

[0043] Preferably, the model can be pre-trained using the labeled data and the true value label, or a semantic segmentation model preliminarily trained in other methods or procedures can be directly obtained. The true value label is also referred to as the ground truth label. In order to facilitate display, the training method provided by the embodiment of the present application is used to train the semantic segmentation model, and the semantic segmentation model is used to perform semantic segmentation on the input image. Figure 2 A left rotation is performed.

[0044] The key technical means of the above technical solution is that the basic model and the training set of the general teacher module and the professional teacher module are used, the supervised loss value is calculated, the unlabeled data is input into the two modules to obtain the prediction results, the prediction results of the two modules are used as the supervision of the peer network of the two modules respectively, the knowledge is refined from the peer teacher model to improve the student model. The teacher model generates a pseudo-label, corrects the pseudo-label through a confidence weighted cross-entropy loss function, and calculates an unsupervised loss value. Based on the supervised and unsupervised loss values, the parameters are iteratively updated. The training method provided by the present application can make the semi-supervised semantic segmentation framework be used for a robot system with valuable computing and memory resources, effectively save resources, and reduce the coupling degree between the parameters of the two basic models. Through anchor-based comparison learning, the segmentation accuracy of the model is significantly improved.

[0045] In addition to the overall technical solution, regarding some specific details when the present application is implemented, in some embodiments, the calculation method of the first cross-entropy loss can be represented as:

[0046]

[0047] In some embodiments, the calculation method of the second cross-entropy loss can be represented as:

[0048]

[0049] In some embodiments, the first cross-entropy loss and the second cross-entropy loss are added together as a multi-classification cross-entropy loss, and the total loss of the model training is calculated, and the multi-classification cross-entropy loss can be represented as:

[0050]

[0051] wherein, represents the first cross-entropy loss; represents the second cross-entropy loss; unsup represents the multi-class cross-entropy loss; represents the cross-entropy-based calculation; represents the general student predicted feature; represents the professional student predicted feature; represents the professional pseudo label; represents the general pseudo label, and represents the label in which the professional student model and the general teacher model are consistent in prediction, represents the label in which the prediction of the professional student model and the general teacher model is inconsistent and highly mismatched.

[0052] In some embodiments, the training method can specifically include:

[0053] Using the general teacher model and the professional teacher model to obtain general teacher prediction results and professional teacher prediction results, and performing pixel screening on the general teacher prediction results and the professional teacher prediction results to obtain the general pseudo label and the professional pseudo label, respectively.

[0054] Based on the above technical solution, as some typical application examples, the training method can be implemented by using the following steps:

[0055] S1, creating and independently initializing a basic model containing a general teacher module, a professional teacher module and a contrast learning module.

[0056] S2, inputting the labeled data into the model to obtain the prediction results of the general student and the professional student, and calculating the cross-entropy between the model prediction and the corresponding label to obtain the respective supervised loss value.

[0057] S3, for unlabeled data, inputting the data into the model to obtain the general teacher prediction result and the general student prediction result, and performing feature screening as supervision of the professional student and the general student.

[0058] S4, performing contrast learning according to the prediction pseudo label of the general teacher and the output feature of the general student to obtain the contrast learning loss value.

[0059] S5, updating the model parameters using gradient descent, and repeating S2-S4 until convergence.

[0060] The process can be summarized as follows: training two education modules with labeled data; constructing different screening modules using the two education modules for knowledge distillation; based on the pseudo label, using the pseudo label for feature-based contrast learning; and updating the model parameters.

[0061] In some embodiments, the pixel screening can be represented as:

[0062]

[0063] where M represents the filtered pseudo-labels; denotes calculating the confusion matrix of both; denotes all pseudo-labels obtained by the generalist teacher without screening; argmax represents the conversion from the prediction probability vector to the class label;

[0064] The prediction of the generalist teacher is screened, and only consistent and high-divergent ones are selected as the subsequent pseudo-labels.

[0065] In some embodiments, the generation of the high-mismatched label is represented as:

[0066]

[0067] where I represents the mismatch score vector defined by the confusion matrix, and the element I q denotes the proportion of mismatch between the predictions of the specialist student model and the generalist teacher model when the prediction of any one model is q, denotes the pixel screening operation, which only selects pixels with high mismatch score vectors in the middle.

[0068] In some embodiments, the contrastive learning loss includes an ANL loss and a PCSL loss, and the calculation thereof can be represented as:

[0069]

[0070]

[0071] where C represents the number of classes; c represents class c; v i denotes the generalist student prediction feature, i is a natural number; P c denotes the feature of the pixels in each batch; F c denotes the features of C classes in the memory bank; σ c denotes the anchor index corresponding to class C; τ represents the temperature coefficient;

[0072] The ANL loss makes the features closer to the corresponding anchor points and farther away from other anchor points, forming a uniform feature distribution; the PCSL loss hopes that samples of the same class can be more compact.

[0073] In some embodiments, the distance between multiple anchor points is optimized by minimizing the loss, which is represented as:

[0074]

[0075] where j is a natural number;

[0076] In some embodiments, prototypes of each class are computed in the early stage of training and updated iteratively by EMA, and the prototype c i match with the anchor index with the following constraints and shortest optimization by the Hungarian algorithm:

[0077]

[0078] In some embodiments, the sampling manner of the training method can specifically include: first, a threshold φ is predefined to select features with higher confidence;

[0079] Then, features closer to the corresponding anchor points are preferred, and only the top-K closest features are used to update the sampling memory library.

[0080] As a typical application example of the above technical solution, define C as the number of classes, and define v i where i = 1, 2, 3,..., C as the anchor. The dimension of each anchor is equal to the dimension of the feature. These anchor points are randomly initialized, and in order to make each anchor point have the same distance from other anchor points, the following minimization method is performed:

[0081]

[0082] In the early stage of training (for example, the first 1 / 4 or 1 / 3 of the stage, the amount of calculation or the number of iterations, etc.), prototypes of each class are computed and updated iteratively by EMA. The prototype c i match with the anchor index with the following constraints and shortest optimization by the Hungarian algorithm:

[0083]

[0084] Further, in order to obtain a more compact memory library, a distance-based sampling strategy is proposed. First, a threshold φ is predefined to select features with higher confidence, and then features closer to the corresponding anchor points are preferred. Only the top-K closest features are used to update the memory library. It is worth noting that the preferred embodiment of the present application sets the size of the memory library to M.

[0085] In some embodiments, the calculation manner of the supervised loss includes:

[0086]

[0087]

[0088] L sup = H(P G , Y l ) + H(P P , Y l )

[0089] wherein H(P G , Y l ) represents the supervised sub-loss of the general student model; H(P p , Y l ) represents the supervised sub-loss of the professional student model; Y l represents the true value label; L sup represents the supervised loss; represents the labeled prediction feature corresponding to the model prediction; HxW represents the pixel size of the image, h and w represent the longitudinal and transverse coordinates of the pixel respectively; and c represents classification.

[0090] In some embodiments, the total loss of the semi-supervised semantic segmentation model training method is represented as:

[0091]

[0092] wherein L total represents the total loss, λ sup , λ unsup , and λ con represent the corresponding weight coefficients respectively.

[0093] Of course, linear superposition is the most easily implemented and most stable way, and other non-linear superposition ways can also be used for calculation, which can integrate three kinds of loss functions.

[0094] The selection of appropriate balancing coefficients can be adaptively adjusted based on model training, and the focus of the present application is still on the key model training method, and is not limited to specific parameter settings.

[0095] In addition, regarding other details in specific implementation, in some embodiments, the labeled data and unlabeled data can be input in batches; in some embodiments, the labeled data and / or unlabeled data are processed through data augmentation; in some embodiments, the parameters of the initial model can be updated by calculating the total loss value;

[0096] More specifically, the labeled data and unlabeled data are input in batches; the iterations in the training method can be performed in batches, that is, the training set is divided into multiple batches, and each batch corresponds to an iteration. The number of data in the batch can be greater than one, for example, 2-256, etc., or can be only one, which is equivalent to not being divided into batches. Further, the labeled data and / or unlabeled data are processed by data augmentation, such as inversion, horizontal shift, scaling, etc., without limitation.

[0097] Corresponding to the above training method, a second aspect of the embodiments of the present application also provides a semantic segmentation model trained by the above-mentioned semi-supervised semantic segmentation model training method, which is used for semantic segmentation of images.

[0098] In addition, also corresponding to the above training method, a third aspect of the embodiments of the present application also provides a semi-supervised semantic segmentation model training system based on double teachers, which includes the following component modules:

[0099] The supervised loss module, the general education module, the professional education module, the contrast learning module, and the iterative training module;

[0100] The general education module includes a general student model and a professional teacher model, which is used for supervised training of the general student model with the prediction provided by the professional teacher model as pseudo-label, and calculates a first cross-entropy loss; the professional education module includes a general teacher model and a professional student model, which is used for supervised training of the professional student model with the prediction provided by the general teacher model as pseudo-label, and calculates a second cross-entropy loss;

[0101] The contrast learning module is used for contrast learning of the prediction of the general teacher model and the output feature of the general student model, and calculates a contrast learning loss;

[0102] The supervised loss module is used for inputting the labeled data into the initial model to obtain a labeled prediction feature, and calculating a supervised loss based on the corresponding true value label;

[0103] The iterative training module is used for iteratively updating the parameters of the initial model based on the supervised loss, the first cross-entropy loss, the second cross-entropy loss, and the contrast learning loss.

[0104] Further, a fourth aspect of the embodiments of the present application also provides a readable storage medium, which stores a computer program, the computer program is executed to perform the steps of the above-mentioned semi-supervised semantic segmentation model training method, or stores the above-mentioned semantic segmentation model.

[0105] A fifth aspect of the embodiments of the present application also provides an electronic device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to perform the steps of the semi-supervised semantic segmentation model training method provided by any of the embodiments or the semantic segmentation model.

[0106] The technical solutions of the present application are further described in detail below through several embodiments in combination with the drawings. However, the selected embodiments are only used to illustrate the present application, and do not limit the scope of the present application.

[0107] Embodiment 1

[0108] The reliability of the double-teacher-based long-tail semi-supervised semantic segmentation method is verified by using known network data, and is compared with existing semi-supervised semantic segmentation methods. In specific applications, a complete model training can be based on the above technical solutions to perform actual complete steps:

[0109] (1) Create and independently initialize the base model. Then, the training set is composed of a small amount of labeled data and a large amount of unlabeled data. Divide the labeled data and unlabeled data into batches of a certain size, and input them into the model in turn for training.

[0110] (2) For small batches of labeled data, random flipping and horizontal translation operations are performed on each image. The corresponding formula can be as follows:

[0111] (3) After the above operation processing, the obtained labeled data batches are respectively input into the base model. Then, the supervised loss value is calculated based on the model prediction and the corresponding ground truth label. The cross-entropy function is used as the loss function for supervised training, and the formula is as follows:

[0112]

[0113]

[0114] where HxW represents the number of pixels, Y1 represents a one-hot vector converted from the ground truth, h, w and c represent the size of the vector.

[0115] Therefore, the definition of the supervised loss is as follows:

[0116] L sup =H(P G ,Y l )+H(P P ,Y l )

[0117] (4) For small batches of unlabeled data, random flipping and horizontal translation operations are performed on each image.

[0118] (5) Define C as the number of classes, and define v i where i = 1, 2, 3, …, C as anchors. The dimension of each anchor is equal to the dimension of the feature. These anchors are randomly initialized, and in order to make each anchor have the same distance from other anchors, the following minimization method is performed:

[0119]

[0120] The prototype of each class is calculated in the early stage of training, and they are updated iteratively by EMA. The prototype c i matches the anchor index with the following constraints, and the shortest optimization is performed by the Hungarian algorithm:

[0121]

[0122] (6) In order to obtain a more compact memory bank, the embodiment proposes a distance-based sampling strategy. First, a threshold φ is predefined to select features with higher confidence, and then features closer to the corresponding anchor point are preferred. Only the top-K closest features are used to update the memory bank. Note that the size of the memory bank is set to M.

[0123] The formula for calculating the contrastive learning loss value is:

[0124]

[0125]

[0126] The ANL loss makes the features closer to the corresponding anchor point and farther away from other anchor points, forming a uniform feature distribution; the PCSL loss hopes that samples of the same class can be more compact.

[0127] Finally, the total loss value can be as follows:

[0128]

[0129] where L total represents the total loss value, λ sup , λ unsup , λ con respectively represent the corresponding weight coefficients.

[0130] (7) Update the model parameters using gradient descent, repeat steps (2)-(6) until the model converges, thereby obtaining a model that can be used for semantic segmentation.

[0131] Specifically, the step (1) is as a general initialization step, having a general education module, a professional education module and a contrast learning module. In each iteration process, the network parameters are updated using gradient descent. In the steps (2) and (4), data processing operations are performed on each image in the small batch of labeled data and on the small batch of unlabeled data images. In the step (3), the supervised loss values of the two encoders are calculated using the cross-entropy function. More specifically, the contrast learning module in the step (6) specifically includes the following processes:

[0132] (i) initializing the prototype with labeled data.

[0133] (ii) using the prototype to find the corresponding anchor points by the Hungarian algorithm.

[0134] (iii) using the general teacher pseudo-label prediction and the output features of the general student to perform contrast learning.

[0135] Based on the above implementation process, the embodiment also illustrates the comparative experimental results of the provided training method and the existing semi-supervised semantic segmentation model training method:

[0136] The two modules in the embodiment share the same structure (DeepLabv3+), but are initialized differently. All networks are trained using a stochastic gradient descent (SGD) optimizer. When training using Cityscapes and PASCAL VOC 2012, the initial learning rates are 10 -2 and 5×10 -3 , respectively, and the momentum is 0.9. A polynomial decay strategy, i.e. , is used to adjust the learning rate. During training, for the Cityscapes dataset, each image is randomly cropped to 800×800, and the batch size is 3, and for the PASCAL VOC 2012 dataset, 512×512 is used, and the batch size is 8. The weights and hyperparameters are as follows: λ sup =1, λ unsup =1, λ con =0.1, τ=0.5, φ=0.95, and M=256 (in the present application, the parameters can be appropriately adjusted, for example, ±20% or even ±50%). In order to apply strong augmentation, three rectangular regions of random ratios (in the range of [0.25, 0.5]) are randomly positioned on the input, and the CutMix strategy is used for augmentation.

[0137] Table 1 Model prediction results of different training methods under the network structure of Cityscapes dataset and DeepLabv3+(ResNet50)

[0138]

[0139] Table 2. Model prediction results of different training methods under VOC 2012 dataset and deeplabv3+(ResNet50) network structure

[0140]

[0141] Table 3. Model prediction results of different training methods under VOC 2012 dataset and deeplabv3+(ResNet101) network structure

[0142]

[0143] The training sets of Cityscapes and VOC2012 dataset have 2975 and 10582 pictures respectively. The first column of the table represents the number of randomly selected labeled data, and the numerical value in the table is the recognition accuracy of the model in the test dataset (%); DCL is the training method provided in the embodiment. The dataset and network used in Table 1 are Cityscapes and deeplabv3+(ResNet50) respectively, the dataset and network used in Table 2 are VOC2012 and deeplabv3+(ResNet50) respectively, and the dataset and network used in Table 3 are VOC2012 and deeplabv3+(ResNet101) respectively. It can be seen from Table 1, Table 2 and Table 3 that the recognition method provided in the embodiment effectively improves the training efficiency of semi-supervised semantic segmentation and the accuracy of the semantic segmentation model.

[0144] Embodiment 2

[0145] This embodiment shows the training and prediction results of a single module in Table 4 to study how the pseudo each module helps the two decoders. In the technical scheme of the application, each module helps to improve the performance of semi-supervised semantic segmentation. The results show that the deep decoder increases the mIOU by 3.95%. This shows that the predictions of the shallow decoder and the deep decoder are complementary.

[0146] Table 4. Effectiveness of each module on the Cityscapes dataset

[0147] Supervised data Dual-module strategy Pseudo-label screening strategy Contrastive learning module mIou% √ 66.74 √ √ 73.37 √ √ √ 73.96 √ √ √ 73.56 √ √ √ √ 74.14

[0148] Embodiment 3

[0149] This embodiment illustrates the training method provided in the application. In order to verify the effectiveness of the label selection strategy, an ablation experiment is performed. As shown in Table 5 below, the selection of consistent pseudo labels and high-divergence pseudo labels is better than the selection of pseudo labels according to the confidence threshold.

[0150] Table 4 Effectiveness of pseudo-label correction module on VOC dataset

[0151] Consistent pseudo-label Low-divergence pseudo-label High-divergence pseudo-label mIou% √ 71.43 √ √ 72.61 √ √ 74.14 √ √ √ 73.56

[0152] It can be found that the effectiveness of the module based on the pseudo-label screening technology is significantly stronger than the implementation of the fixed confidence threshold.

[0153] Based on the above embodiments, it can be clear that the training method provided by the embodiments of the present application uses a general education module and a professional education module for cyclic iterative training, which can effectively reduce the noise pseudo-labels.

[0154] Therefore, the training method provided by the present application can avoid model coupling when generating pseudo-labels by two teachers located in different modules; compared with the baseline method in the prior art, the module only introduces some additional parameters, and the data occupancy is small; and by comparing the features output by the general student model and the pseudo-labels output by the general teacher model, contrastive learning is performed, which avoids overfitting while significantly improving the utilization rate of pseudo-labels, and finally improves the model training efficiency and accuracy.

[0155] It should be noted that the specific implementation means of the present application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium on which computer readable program instructions for causing a processor to implement various aspects of the present application are loaded. The computer readable storage medium can be a tangible device that maintains and stores instructions for use by an instruction execution device. The computer readable storage medium may, for example, include, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or a hole in the groove structure, and any suitable combination of the above.

[0156] Although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order, in fact, some of the steps can be performed concurrently, or even in reverse order, as long as the desired function can be achieved

[0157] It should be understood that the above-described embodiments are merely intended to illustrate the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and to implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made according to the spirit and principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method for training a dual-teacher based semi-supervised semantic segmentation model, the method comprising: The method comprises the following steps: providing an initial model and training data, wherein the initial model comprises a general student model, a general teacher model, a professional student model and a professional teacher model, and the training data comprises labeled data and corresponding true value labels, and unlabeled data; inputting the labeled data into the initial model to obtain labeled prediction features, and calculating a supervised loss based on the corresponding true value labels; inputting the unlabeled data into the initial model, using the general student model to obtain general student prediction features, using the professional student model to obtain professional student prediction features, using the general teacher model to predict general pseudo labels, and using the professional teacher model to predict professional pseudo labels; calculating a first cross-entropy loss between the general student prediction features and the professional pseudo labels, and a second cross-entropy loss between the professional student prediction features and the general pseudo labels; calculating a contrastive learning loss between the general teacher pseudo labels and the general student prediction features; updating the parameters of the initial model based on the supervised loss, the first cross-entropy loss, the second cross-entropy loss and the contrastive learning loss. 2.The method of claim 1, wherein, The calculation method of the first cross-entropy loss is represented as: and / or, the calculation method of the second cross-entropy loss is represented as: Preferably, the first cross-entropy loss and the second cross-entropy loss are added together as a multi-classification cross-entropy loss to further calculate the total loss of the model training, and the multi-classification cross-entropy loss is represented as: in, represents the first cross entropy loss; represents the second cross entropy loss; L unsup represents the multi-classification cross entropy loss; Indicates calculation based on cross entropy; represents the general student prediction characteristics; represents the predicted characteristics of students in the said major; represents the professional pseudo-label; represents the general pseudo-label, and Indicates that the professional student model and the general teacher model predict the same label, Labels indicating a high degree of mismatch between the professional student model and the general teacher model. 3.The method of claim 2, wherein, Specifically comprising: using the general teacher model and the professional teacher model to obtain general teacher prediction results and professional teacher prediction results, and performing pixel screening on the general teacher prediction results and the professional teacher prediction results to obtain the general pseudo labels and the professional pseudo labels respectively. 4.The method of claim 3, wherein, The pixel screening is represented as: Wherein M represents the general-purpose pseudo-label screened out; denotes the confusion matrix of the two; denotes all unfiltered pseudo-labels of professional teachers for general students; argmax represents the conversion from the predicted probability vector to the class label; and / or, the generation method of the high-mismatch label is represented as: where I denotes the vector of mismatch scores defined using the confusion matrix, element I q denotes the proportion of mismatch between the predictions of the specialized student model and the generalist teacher model when the prediction of any one model is q, denotes a pixel screening operation that only selects pixels for which the mismatch score vector is above a threshold. 5.The method of claim 1, wherein, The contrastive learning loss comprises an ANL loss and a PCSL loss, and the calculation method is represented as: wherein C represents the number of classes; c represents a class; v i represents the general education student prediction feature, i is a natural number; P c represents the feature of pixels in each batch; F c represents the feature of C classes in the memory library; σ c represents the anchor point index corresponding to the C classes; τ represents the temperature coefficient; Preferably, the distance between the plurality of anchor points is optimized by minimizing the loss, and the minimizing loss is represented as: wherein j is a natural number; Preferably, prototypes of each class are computed in the early phase of training and updated iteratively by EMA, prototypes c i with anchor index match, with the following constraints, and shortest optimization by the Hungarian algorithm: Preferably, the sampling method of the training method specifically comprises: first, a threshold φ is predefined to select features with high confidence; Then, features closer to the corresponding anchor points are selected, and only the top-K closest features are used to update the sampling memory bank. 6.The method of claim 1, wherein, The calculation method of the supervised loss comprises: L sup = H(P G , Y l ) + H(P P , Y l ) H(P G , Y l ) represents a supervised sub-loss of a general student model; H(P P , Y l ) represents a supervised sub-loss of a professional student model; Y l represents the true value label; L sup represents the supervised loss; represents the labeled prediction feature predicted by the corresponding model; HxW represents the pixel size of the image, h and w represent the longitudinal and transverse coordinates of the pixel respectively; c represents classification. 7.The method of claim 1, wherein, The total loss of the semi-supervised semantic segmentation model training method is represented as: wherein L total represents the total loss, λ sup , λ unsup , λ con respectively represent the corresponding weight coefficients.

8. The semantic segmentation model obtained by the semi-supervised semantic segmentation model training method of any one of claims 1-7, wherein the semantic segmentation model is used for semantic segmentation of an image. 9.A dual-teacher based semi-supervised semantic segmentation model training system, characterized in that, Comprising: a supervised loss module, a general education module, a professional education module, a contrastive learning module and an iterative training module; The general education module comprises a general student model and a professional teacher model, is configured to perform supervised training of the general student model by taking prediction provided by the professional teacher model as pseudo-label, and calculate a first cross-entropy loss; the professional education module comprises a general teacher model and a professional student model, is configured to perform supervised training of the professional student model by taking prediction provided by the general teacher model as pseudo-label, and calculate a second cross-entropy loss; The contrast learning module is configured to perform contrast learning on prediction of the general teacher model and output feature of the general student model, and calculate a contrast learning loss; The supervised loss module is configured to input the labeled data into the initial model to obtain labeled prediction feature, and calculate a supervised loss based on the corresponding true value label; The iterative training module is configured to iteratively update parameters of the initial model based on the supervised loss, the first cross-entropy loss, the second cross-entropy loss and the contrast learning loss.

10. A readable storage medium, characterized by, The readable storage medium stores a computer program, the computer program is executed to perform the steps of the semi-supervised semantic segmentation model training method in any one of claims 1-7, or stores the semantic segmentation model in claim 8. The readable storage medium stores a computer program, the computer program is executed to perform the steps of the semi-supervised semantic segmentation model training method in any one of claims 1-7, or stores the semantic segmentation model in claim 8.

Images