Model training method and device, equipment and storage medium

Abstract

The invention provides a model training method and device, equipment and a storage medium, and the method comprises the steps: carrying out the preprocessing of an image data set, obtaining a target data set, and enabling the target data set to comprise a source domain data set and a target domain data set; training the student network of image segmentation based on the source domain data set to obtain a trained student network; performing index moving average on the parameters of the trained student network to obtain a teacher network; generating a corresponding pseudo tag for each pixel of each target domain data in the target domain data set based on a teacher network; and carrying out secondary training on the trained student network based on the pseudo tag to obtain a target student network. According to the embodiment of the invention, the student network is trained by using the high-confidence pseudo tag generated by the teacher network, so that the performance of the student network on the target domain is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a model training method, apparatus, device, and storage medium. Background Technology

[0002] Semantic segmentation models are deep learning models that can classify each pixel in an image, thereby identifying the boundaries of different objects and scenes within the image. This type of model has wide applications in multiple fields.

[0003] Currently, semantic segmentation models can be trained using supervised learning. However, supervised learning relies too heavily on large-scale labeled datasets, and semantic segmentation models are generally deep neural networks. Due to cross-domain mismatch, deep neural networks typically perform poorly in new domains. Summary of the Invention

[0004] This application proposes a model training method, apparatus, device, and storage medium that can solve the technical problem that current semantic segmentation models often perform poorly in new domains due to cross-domain mismatch.

[0005] The first aspect of this application proposes a model training method, including:

[0006] The image dataset is preprocessed to obtain the target dataset, which includes the source domain dataset and the target domain dataset;

[0007] The image segmentation student network is trained based on the source domain dataset to obtain a trained student network;

[0008] The teacher network is obtained by performing an exponential moving average on the parameters of the trained student network.

[0009] Based on the teacher network, corresponding pseudo-labels are generated for each target domain data in the target domain dataset;

[0010] Determine the confidence level of each target domain data;

[0011] The trained student network is then trained a second time based on the pseudo-labels and confidence scores of each target domain data to obtain the target student network.

[0012] An embodiment of the second aspect of this application provides a model training apparatus, comprising:

[0013] The preprocessing module is used to preprocess the image dataset to obtain the target dataset, which includes the source domain dataset and the target domain dataset.

[0014] The training module is used to train the image segmentation student network based on the source domain dataset to obtain the trained student network.

[0015] The exponential moving average module is used to perform an exponential moving average on the parameters of the trained student network to obtain the teacher network;

[0016] The generation module is used to generate corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset based on the teacher network;

[0017] The training module is also used to perform secondary training on the trained student network based on the pseudo-labels to obtain the target student network.

[0018] An embodiment of the third aspect of this application provides a computing device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first or second aspect above.

[0019] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first or second aspect above.

[0020] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0021] This application proposes a model training method, device, and storage medium, comprising: preprocessing an image dataset to obtain a target dataset, the target dataset including a source domain dataset and a target domain dataset; training a student network for image segmentation based on the source domain dataset to obtain a trained student network; performing an exponential moving average on the parameters of the trained student network to obtain a teacher network; generating corresponding pseudo-labels for each target domain data in the target domain dataset based on the teacher network; determining the confidence level of each target domain data; and performing secondary training on the trained student network based on the pseudo-labels and confidence levels of each target domain data to obtain a target student network. The embodiments of this application improve the performance of the student network on the target domain by using high-confidence pseudo-labels generated by the teacher network to train the student network.

[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0024] In the attached diagram:

[0025] Figure 1 A flowchart of a model training method provided in an embodiment of this application is shown;

[0026] Figure 2 This paper shows a schematic diagram of the structure of a model training device provided in one embodiment of the present application;

[0027] Figure 3 This invention provides a schematic diagram of the structure of a computing device according to an embodiment of the present application.

[0028] Figure 4 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0029] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0030] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0031] Building upon the aforementioned background, semantic segmentation models based on Fully Convolutional Networks (FCNs) are the most promising methods for achieving pixel-level classification. To accurately understand scenes, semantic segmentation using Convolutional Neural Networks (CNNs) has been continuously improved in many aspects. Spatial Pyramid Pooling (SPP) modules have been proven to effectively utilize multi-scale representations to gather scene cues from local to global environments.

[0032] Generally, the semantic segmentation model described above can be trained using supervised learning. However, the training set typically includes a source domain dataset and a target domain dataset. The source domain dataset is a labeled dataset on which the model is trained, while the target domain dataset is the dataset on which the model is expected to perform well. This target domain dataset may be unlabeled or have a very limited number of labels.

[0033] Deep neural networks typically perform poorly in the target domain due to cross-domain mismatch.

[0034] To address the aforementioned technical problems, this application proposes a model training method, apparatus, device, and storage medium, comprising: preprocessing an image dataset to obtain a target dataset, the target dataset including a source domain dataset and a target domain dataset; training a student network for image segmentation based on the source domain dataset to obtain a trained student network; performing an exponential moving average on the parameters of the trained student network to obtain a teacher network; generating corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset based on the teacher network; and performing secondary training on the trained student network based on the pseudo-labels to obtain a target student network. This application embodiment improves the performance of the student network on the target domain by using high-confidence pseudo-labels generated by the teacher network to train the student network.

[0035] The model training method or interface generation method of this application can be executed by a computing device, which can be a server, such as a single server, multiple servers, a server cluster, a cloud computing platform, etc. Optionally, the computing device can also be a terminal device, such as a mobile phone, tablet computer, game console, portable computer, desktop computer, advertising machine, all-in-one machine, etc. This application does not limit the type or number of computing devices.

[0036] The following describes a model training method according to an embodiment of this application, with reference to the accompanying drawings.

[0037] See Figure 1 The method specifically includes the following steps:

[0038] S101. Preprocess the image dataset to obtain the target dataset.

[0039] The target dataset includes the source domain dataset and the target domain dataset.

[0040] In some embodiments, the image dataset is preprocessed to obtain the target dataset, including:

[0041] The images in the image dataset are transformed and segmented sequentially by random rotation, multi-scale cropping, and uniform segmentation to obtain the segmented dataset.

[0042] The segmented dataset is then normalized to obtain a normalized dataset.

[0043] The images in the normalized dataset are scaled, rotated, and / or their brightness and contrast are adjusted to obtain the enhanced target dataset.

[0044] The images in the image dataset are transformed and segmented sequentially by random rotation, multi-scale cropping, and uniform segmentation to obtain the segmented dataset.

[0045] In this embodiment, each source image is evenly divided into four parts, thereby obtaining four times the number of training set images.

[0046] Furthermore, the segmented object detection dataset is normalized to obtain a normalized object detection dataset. The normalization formula is:

[0047]

[0048] Where x represents the input data, x * This represents the normalized output data, where max indicates the maximum value and min indicates the minimum value.

[0049] By employing a normalization method, all data in both the source and target domain datasets are within the range [0,1]. This normalization process can improve the convergence speed and enhance the model's accuracy.

[0050] The images in the normalized object detection dataset are scaled, rotated, and / or their brightness and contrast are adjusted to obtain the enhanced object detection dataset.

[0051] Specifically, when scaling the images in the normalized object detection dataset, a multi-scale training strategy was employed. The dataset was scaled to different scales for input to improve the network's adaptability to object detection of different sizes. In this embodiment, during multi-scale training, the training and testing input images were scaled to six scales using wrapAffine to avoid losing too much source image information.

[0052] The specific formula for scaling using wrapAffine is as follows:

[0053]

[0054] Among them, f x and f y represents the focal length (scaling factor) of the x-axis and y-axis respectively, x and y represent the width and height of the input before scaling, and x′ and y′ represent the width and height after scaling.

[0055] Specifically, wrapAffine is used to perform rotation enhancement on the image dataset, and the formula is as follows:

[0056]

[0057] Where θ represents the rotation angle, x and y represent the width and height of the input before scaling, and x′ and y′ represent the width and height after scaling.

[0058] Specifically, when adjusting the brightness and contrast of images in the normalized object detection dataset, it is considered that the brightness and contrast variations between the training and test sets are significant, and such large differences could affect model training. Therefore, this embodiment enhances the brightness and contrast of the training and test sets to avoid the impact of brightness and contrast variations on the model training and prediction process. Adjusting the contrast involves increasing or decreasing the difference between bright and dark points while keeping the average brightness constant. The adjustment ratio for each point is applied to the difference between this value and the average brightness, thereby ensuring that the calculated average brightness remains constant.

[0059] Therefore, the formula for adjusting brightness and contrast is:

[0060] Out=Average+(In-Average)*(1+percent);

[0061] Where In represents the original pixel brightness; Average represents the average brightness of the entire image; Out represents the adjusted brightness; and percent∈[-1,1] represents the adjustment range.

[0062] Simultaneously, network parameter augmentation algorithms specifically include: firstly, analyzing the target data in the training and validation sets...

[0063] The height-to-width ratio of the target is the height-to-width ratio of the generated bounding box. Then, the area of ​​the target in the training set is analyzed, and the size ratio of the generated bounding box is adjusted according to the target area. The larger the target area, the larger the size ratio of the generated bounding box should be. The adjustment range of the generated bounding box is 1 to 10.

[0064] S102. Train the student network for image segmentation based on the source domain dataset to obtain the trained student network.

[0065] The source domain dataset typically contains labeled data, which allows us to train the student network using supervised learning methods. By training on the source domain, the student network can learn rich feature representations, which are crucial for understanding the underlying structure of the data.

[0066] Specifically, training uses the source domain image X S And One-hot label Y S Supervised learning is performed. The student network is trained using the classification cross-entropy loss, with the following formula:

[0067]

[0068] in, Let H be the loss for the i-th source image, and H×W be the product of the image's height and width, representing the total number of pixels in the image. For one-hot encoding, represents the probability that the j-th pixel in the i-th sample belongs to class c, where C is the number of classes. Let be the probability predicted by the student network that the j-th pixel in the i-th sample belongs to the c-th class.

[0069] In some embodiments, the student network can be trained with the aim of minimizing the classification cross-entropy loss, resulting in a well-trained student network.

[0070] S103. Perform an exponential moving average on the parameters of the trained student network to obtain the teacher network.

[0071] In some embodiments, student networks often fail to generalize well to the target domain, resulting in poor performance on target images. To address this domain gap, teacher networks are introduced for domain knowledge transfer.

[0072] Among them, by applying an exponential moving average to the parameters of the trained student network, the teacher network can be implemented as follows:

[0073] First, initialize the parameters of the teacher network. Typically, these parameters are initialized to the parameters of the trained student network.

[0074] Choose an Exponential Moving Average (EMA) parameter α, which determines the rate at which the teacher network parameters are updated. The value of α is typically between (0,1). A smaller α value means the teacher network parameters are updated more slowly, while a larger α value means the updates are faster.

[0075] In each training iteration, the parameters of the teacher network are updated using the following formula:

[0076] θteacher←α·θteacher+(1-α)·θstudent

[0077] Where: θteacher are the parameters of the teacher network, θstudent are the parameters of the student model, and α is the EMA parameter.

[0078] S104. Based on the teacher network, generate corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset.

[0079] In this model, the teacher network, by smoothing the parameter updates of the student network, tends to learn more generalized feature representations. These representations have better generalization ability for unseen data, thus generating pseudo-labels that are usually more accurate and reliable. During training, the parameters of the student network are continuously updated, and the teacher network dynamically reflects these updates through EMA (Effective Mapping). This means that the teacher network can gradually adapt to the latest state of the student network, thereby generating pseudo-labels that better match the current predictions of the student network.

[0080] Furthermore, image segmentation aims to classify each pixel in an image, thus requiring pixel-level detail. Each pixel may belong to a different object or background, therefore, a pseudo-label needs to be generated for each pixel individually.

[0081] Therefore, the teacher network can generate corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset, and then use these pseudo-labels to retrain the student network, thereby improving the performance of the student network on the target domain.

[0082] In some embodiments, a teacher network φ is used for the target domain data X. T The formula for generating pseudo-tags is shown below:

[0083]

[0084] in, For target domain data X T The pseudo-label of the j-th pixel of the i-th target domain sample, h φ For the prediction function of the teacher network, The j-th pixel of the i-th target domain sample in the target domain data.

[0085] In this context, the pseudo-labels generated by the teacher network typically represent the probability distribution of the category to which each pixel or sample belongs. In image segmentation tasks, this might be a two-dimensional array, where each element is a vector representing the probability that the pixel belongs to each category; in classification tasks, it might be a vector representing the probability that a sample belongs to each category.

[0086] S105. The trained student network is trained a second time based on pseudo-labels to obtain the target student network.

[0087] In some embodiments, pseudo-labels can be used as target pseudo-labels for the target dataset. A classification cross-entropy loss is constructed based on the pseudo-labels and predicted labels to minimize the classification cross-entropy loss, thereby performing secondary training on the trained student network to obtain the target student network.

[0088] Understandably, while the pseudo-labels generated by the teacher network contain rich information, directly using these pseudo-labels to train the student network may introduce problems. For example, the pseudo-labels generated by the teacher network may contain noise, especially when the distribution of the target domain data differs significantly from that of the source domain data. Directly using these potentially inaccurate pseudo-labels for training may cause the student network to learn incorrect patterns, thereby affecting its performance.

[0089] In some embodiments, since the pseudo-label in the image segmentation task may be a two-dimensional array, where each element is a vector representing the probability that the pixel belongs to each category, the pseudo-label can be transformed into a probability distribution. The pseudo-label can be transformed into a softmax probability vector using the Softmax function.

[0090] For each pixel, the model calculates the probability of it belonging to each category and selects the one with the highest probability as the most likely category for that pixel. This maximum probability value represents the model's confidence in classifying that pixel.

[0091] In some cases, we are only interested in predictions that the model is very confident about. Therefore, we set a confidence threshold τ. Only when the maximum softmax probability of a pixel exceeds this threshold do we consider the model's classification of that pixel to be reliable.

[0092] By comparing the maximum softmax probability of each pixel with a threshold τ, we can filter out pixels that the model is highly confident in. These pixels' pseudo-labels are generally considered to be of high quality and can be used for training or evaluating the model.

[0093] Suppose we have an image segmentation model that outputs a softmax probability vector [0.1, 0.2, 0.7] for a pixel, representing the probability that the pixel belongs to one of the three categories.

[0094] The maximum softmax probability is 0.7. If we set the threshold τ = 0.5, then the maximum probability of this pixel exceeds the threshold, and we consider the model's classification of this pixel to be reliable.

[0095] Furthermore, pseudo-labels with a maximum softmax probability greater than a threshold are selected, and the trained student network is trained a second time based on these pseudo-labels to obtain the target student network.

[0096] This application proposes a model training method, apparatus, device, and storage medium, comprising: preprocessing an image dataset to obtain a target dataset, the target dataset including a source domain dataset and a target domain dataset; training a student network for image segmentation based on the source domain dataset to obtain a trained student network; performing an exponential moving average on the parameters of the trained student network to obtain a teacher network; generating corresponding pseudo-labels for each target domain data in the target domain dataset based on the teacher network; determining the confidence level of each target domain data; and performing secondary training on the trained student network based on the pseudo-labels and confidence levels of each target domain data to obtain a target student network. The embodiments of this application improve the performance of the student network on the target domain by using high-confidence pseudo-labels generated by the teacher network to train the student network.

[0097] In some embodiments, training a student network for image segmentation based on a source domain dataset to obtain a trained student network includes: extracting features at different levels from each source domain data in the source domain dataset; using a multi-level feature fusion module to fuse features at the same level and features at different levels to obtain fused features; performing a depthwise separable convolution operation on the fused features to obtain target features; and training the student network for image segmentation based on the target features corresponding to each source domain data to obtain a trained student network.

[0098] In some embodiments, to improve the accuracy of the image segmentation model, the student network can be trained using contextual information from the image dataset.

[0099] Specifically, the student network first extracts features at different levels from the input image. These features typically include low-level features (such as edges and textures) and high-level features (such as semantic information). These feature maps contain local information about the image.

[0100] Furthermore, these features from different levels are fused through a multi-level feature fusion module. Specifically, the Transformer uses a self-attention mechanism to capture long-distance dependencies in the feature maps, thereby introducing contextual information. The self-attention mechanism allows the model to consider information from other locations in the image when processing each pixel, thus better understanding the global structure of the image. To further enhance the expressive power of the features, the Transformer module performs feature fusion at multiple levels. This means that the Transformer is applied not only to high-level semantic features but also to low-level features, in order to capture both local details and global context simultaneously.

[0101] The multi-level feature fusion module can first fuse features at the same level and then fuse features at different levels, or it can first fuse features at different levels and then fuse features at the same level. The order of fusion is not limited in this embodiment.

[0102] After feature fusion, depthwise separable convolution (DSC) is used to further process the feature maps. DSC is an efficient convolution operation that decomposes standard convolution into depthwise convolution (DC) and pointwise convolution (PC). This decomposition reduces the number of model parameters and computational cost while preserving feature expressiveness. Through these steps, the Transformer module not only fuses features from different levels but also utilizes the image's contextual information. This contextual information helps the model understand the relationships between pixels in the image, thereby making more accurate predictions in image segmentation tasks.

[0103] Furthermore, the student network for image segmentation is trained based on the target features corresponding to each source domain data, resulting in a well-trained student network. This well-trained student network can understand the relationships between pixels in the image, thereby making more accurate predictions in the image segmentation task.

[0104] It should be noted that the multi-level feature fusion module can be used to fuse features at the same level and features at different levels to obtain fused features. The fused features can also be used to perform depthwise separable convolution operations to obtain target features. This can also be applied to the process of retraining a pre-trained student network based on pseudo-labels to obtain a target student network. This helps the model understand the relationship between pixels in the image, thereby making more accurate predictions in image segmentation tasks.

[0105] In some embodiments, before training the image segmentation student network based on the source domain dataset to obtain the trained student network, the method further includes: in the preheating training phase, obtaining a preset initial number of iterations and a base learning rate; determining the current learning rate based on the initial number of iterations, the base learning rate, and the current number of iterations; and ending the preheating training phase if the current learning rate and the base learning rate are consistent.

[0106] In some implementations, the network weights are typically initialized randomly in the early stages of training. If a large learning rate is used at this time, the gradient updates may be too large, causing the network weights to update too drastically and making it difficult to converge.

[0107] Generally, the training phase includes a warm-up phase and a normal training phase. During the warm-up phase, the learning rate is gradually increased to help the model start the training process stably and avoid instability or divergence caused by an excessively large initial learning rate.

[0108] The preset initial iteration count and basic learning rate can be flexibly set based on the actual situation, and will not be elaborated here.

[0109] In each iteration of the Warmup phase, the learning rate is calculated based on the current iteration number and the total number of iterations in the Warmup phase. The formula for calculating the learning rate is typically as follows:

[0110] η t =η base ·t / t warm

[0111] ηt is the learning rate for the t-th iteration, ηbase is the base learning rate, t is the current iteration number, and twarm is the total number of iterations in the Warmup phase.

[0112] In each iteration, the calculated learning rate is applied to the optimizer of the student network to update the network weights.

[0113] Monitor the performance of the student network during training, and adjust the length of the Warmup phase or the base learning rate if necessary.

[0114] Furthermore, the image segmentation student network is trained based on the source domain dataset to obtain a trained student network, including: training the image segmentation student network based on the base learning rate and the source domain dataset to obtain a trained student network.

[0115] The gradient of the student network can be obtained based on the base learning rate; the gradient points in the direction of the fastest increase in the loss function. In the gradient descent algorithm, the gradient is used to determine the direction and magnitude of parameter updates.

[0116] The learning rate is a hyperparameter in the gradient descent algorithm that controls the step size of parameter updates in each iteration. It determines how far we move in the negative direction of the gradient (i.e., the direction in which the loss function decreases), in the direction indicated by the gradient.

[0117] In the gradient descent algorithm, the basic rules for parameter updates are as follows:

[0118]

[0119] Where: θ represents the model parameters, and η represents the learning rate. This represents the gradient of the loss function J with respect to the parameter θ.

[0120] After the gradient is determined, the student network can be trained based on the direction and magnitude of parameter updates determined by the gradient.

[0121] In some embodiments, the trained student network is retrained based on pseudo-labels to obtain the target student network, including: determining the target pixel with a maximum confidence score greater than a preset confidence threshold in each pixel of each target domain data; and retraining the trained student network based on the target pseudo-labels corresponding to the target pixels to obtain the target student network.

[0122] The preset confidence level can be flexibly set based on actual conditions. The confidence level is the probability that a pixel belongs to one of several categories, and the maximum confidence level is the highest probability among the various categories corresponding to that pixel. For example, if the probability of a pixel being a cat is 70%, a dog is 20%, and a pig is 10%, then the confidence level of that pixel is: cat 70%, dog 20%, pig 10%, and the maximum confidence level is cat 70%.

[0123] When generating pseudo-labels, only pixels whose predicted probabilities exceed a threshold are selected, as these pixels are considered reliable. Therefore, we can obtain the maximum confidence score for each target domain and compare it with a preset confidence threshold to filter out target pixels with a maximum confidence score greater than the preset threshold.

[0124] The screening process can be represented by the following formula:

[0125]

[0126] in, H represents the number of pixels in the i-th target domain sample whose confidence exceeds a threshold, used for weighted loss calculation. H×W is the product of the image's height and width, representing the total number of pixels in the image. φ For the prediction function of the teacher network, Let τ be the j-th pixel of the i-th target domain sample in the target domain data, and τ be the confidence threshold.

[0127] The above process yields the target pixels with a maximum confidence score greater than a preset confidence threshold, as well as the number of target pixels. Furthermore, based on the target pixel pseudo-labels, the trained student network is trained a second time to obtain the target student network.

[0128] In some embodiments, the trained student network is retrained based on the target pseudo-labels corresponding to the target pixels to obtain the target student network, including:

[0129] For the first target domain data, the first target domain data is input into the trained student network to obtain the predicted label corresponding to each pixel in the first target domain data. The first target domain data is any target domain data in the target domain dataset. Based on the predicted label and the target pseudo label, the loss function value is calculated. Based on the loss function value, the model parameters of the trained student network are adjusted, and training continues until the training completion condition is met to obtain the target student network.

[0130] In some embodiments, the process of calculating the loss function value based on the predicted label and the target pseudo-label can be represented as:

[0131]

[0132] in The loss for the i-th target domain sample, H×W: the product of the image height and width, representing the total number of pixels in the image, C: the number of classes. The number of pixels in the i-th target domain sample whose confidence exceeds the threshold is used for weighted loss calculation. The probability that the j-th pixel in the i-th target domain sample belongs to the c-th class of pseudo-labels. The student network predicts the probability that the j-th pixel in the i-th target domain sample belongs to the c-th class.

[0133] This specific process can be implemented as follows:

[0134] Double summation: The outer summation is performed on every pixel in the image, while the inner summation is performed on each category.

[0135] Target pseudo-label: PT(i,j,c) is the pseudo-label probability generated by the teacher network for the j-th pixel in the i-th target domain sample, representing the probability that the pixel belongs to the c-th class.

[0136] Predicted label: g∈((xT(i))(j,c)) is the probability predicted by the student network that the j-th pixel in the i-th target domain sample belongs to the c-th class.

[0137] Log loss: Calculates the log loss of the predicted probability, measuring the difference between the student network's predictions and the pseudo-labels.

[0138] Weighted loss: The loss is weighted by qT(i), and only pixels with confidence exceeding the threshold are considered, thereby reducing the impact of unreliable pseudo-labels.

[0139] In some embodiments, the goal is to minimize the classification cross-entropy loss, i.e., minimize The goal is to perform secondary training on the trained student network to obtain the target student network.

[0140] This application also provides a model training apparatus for executing the model training method provided in any of the above embodiments. Figure 2 As shown, the device includes: a preprocessing module 201, a training module 202, an exponential moving average module 203, and a generation module 204.

[0141] Preprocessing module 201 is used to preprocess the image dataset to obtain a target dataset, wherein the target dataset includes a source domain dataset and a target domain dataset;

[0142] Training module 202 is used to train the image segmentation student network based on the source domain dataset to obtain the trained student network;

[0143] The exponential moving average module 203 is used to perform an exponential moving average on the parameters of the trained student network to obtain the teacher network;

[0144] The generation module 204 is used to generate corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset based on the teacher network;

[0145] The training module 202 is also used to perform secondary training on the trained student network based on the pseudo-labels to obtain the target student network.

[0146] This application proposes a model training method, apparatus, device, and storage medium, comprising: preprocessing an image dataset to obtain a target dataset, the target dataset including a source domain dataset and a target domain dataset; training a student network for image segmentation based on the source domain dataset to obtain a trained student network; performing an exponential moving average on the parameters of the trained student network to obtain a teacher network; generating corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset based on the teacher network; and performing secondary training on the trained student network based on the pseudo-labels to obtain a target student network. The embodiments of this application improve the performance of the student network on the target domain by using high-confidence pseudo-labels generated by the teacher network to train the student network.

[0147] In some embodiments, the training module 202 is specifically used for

[0148] Extract features at different levels from each source domain data within the source domain dataset;

[0149] The multi-level feature fusion module is used to fuse features at the same level and features at different levels to obtain fused features.

[0150] The fused features are subjected to a depthwise separable convolution operation to obtain the target features;

[0151] The image segmentation student network is trained based on the target features corresponding to each source domain data to obtain a trained student network.

[0152] In some embodiments, the above-described apparatus further includes a preheating training module for...

[0153] During the warm-up training phase, the preset initial number of iterations and the base learning rate are obtained;

[0154] The current learning rate is determined based on the initial number of iterations, the base learning rate, and the current number of iterations.

[0155] If the current learning rate and the base learning rate are consistent, the warm-up training phase ends.

[0156] In some embodiments, the training module 202 is further specifically used for:

[0157] The image segmentation student network is trained based on the base learning rate and the source domain dataset to obtain a trained student network.

[0158] In some embodiments, the preprocessing module 201 is specifically used for

[0159] The images in the image dataset are transformed and segmented sequentially by random rotation, multi-scale cropping, and uniform segmentation to obtain the segmented dataset.

[0160] The segmented dataset is then normalized to obtain a normalized dataset.

[0161] The images in the normalized dataset are scaled, rotated, and / or their brightness and contrast are adjusted to obtain the enhanced target dataset.

[0162] In some embodiments, the training module 202 is further specifically used for

[0163] In each pixel of each target domain data, the target pixel with a maximum confidence score greater than a preset confidence threshold is identified;

[0164] The trained student network is then trained a second time based on the target pseudo-labels corresponding to the target pixels to obtain the target student network.

[0165] In some embodiments, the training module 202 is not further specifically used for:

[0166] For the first target domain data, the first target domain data is input into the trained student network to obtain the predicted label corresponding to each pixel in the first target domain data. The first target domain data is any target domain data in the target domain dataset.

[0167] Calculate the loss function value based on the predicted label and the target pseudo-label;

[0168] The model parameters of the trained student network are adjusted based on the loss function value, and training continues until the training completion condition is met to obtain the target student network.

[0169] This application also provides a computing device for performing the above-described model training method. Please refer to... Figure 3 It illustrates a schematic diagram of a computing device provided by some embodiments of this application. For example... Figure 3 As shown, the computing device 7 includes a processor 700, a memory 701, a bus 702, and a communication interface 703. The processor 700, the communication interface 703, and the memory 701 are connected via the bus 702. The memory 701 stores a computer program that can run on the processor 700. When the processor 700 runs the computer program, it executes the model training method provided in any of the foregoing embodiments of this application.

[0170] The memory 701 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 703 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0171] Bus 702 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 701 is used to store programs. After receiving execution instructions, processor 700 executes the programs. The model training method disclosed in any of the aforementioned embodiments of this application can be applied to processor 700, or implemented by processor 700.

[0172] The processor 700 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 700 or by instructions in software form. The processor 700 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 701. Processor 700 reads the information in memory 701 and, in conjunction with its hardware, completes the steps of the above method.

[0173] The computing device and the model training method provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0174] This application also provides a computer-readable storage medium corresponding to the model training method provided in the foregoing embodiments. Please refer to... Figure 4 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the model training method provided in any of the aforementioned embodiments.

[0175] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0176] The computer-readable storage medium provided in the above embodiments of this application and the model training method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0177] It should be noted that:

[0178] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0179] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0180] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0181] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method, characterized in that, include: The image dataset is preprocessed to obtain the target dataset, which includes the source domain dataset and the target domain dataset; The image segmentation student network is trained based on the source domain dataset to obtain a trained student network; The teacher network is obtained by performing an exponential moving average on the parameters of the trained student network. Based on the teacher network, corresponding pseudo-labels are generated for each pixel of each target domain data in the target domain dataset; The trained student network is then trained a second time based on the pseudo-labels to obtain the target student network.

2. The method according to claim 1, characterized in that, The process of training the student network for image segmentation based on the source domain dataset to obtain a trained student network includes: Extract features at different levels from each source domain data within the source domain dataset; The multi-level feature fusion module is used to fuse features at the same level and features at different levels to obtain fused features. The fused features are subjected to a depthwise separable convolution operation to obtain the target features; The image segmentation student network is trained based on the target features corresponding to each source domain data to obtain a trained student network.

3. The method according to claim 1, characterized in that, Before training the student network for image segmentation based on the source domain dataset to obtain the trained student network, the method further includes: During the warm-up training phase, the preset initial number of iterations and the base learning rate are obtained; The current learning rate is determined based on the initial number of iterations, the base learning rate, and the current number of iterations. If the current learning rate and the base learning rate are consistent, the warm-up training phase ends.

4. The method according to claim 3, characterized in that, The process of training the student network for image segmentation based on the source domain dataset to obtain a trained student network includes: The image segmentation student network is trained based on the base learning rate and the source domain dataset to obtain a trained student network.

5. The method according to claim 1, characterized in that, The preprocessing of the image dataset to obtain the target dataset includes: The images in the image dataset are transformed and segmented sequentially by random rotation, multi-scale cropping, and uniform segmentation to obtain the segmented dataset. The segmented dataset is then normalized to obtain a normalized dataset. The images in the normalized dataset are scaled, rotated, and / or their brightness and contrast are adjusted to obtain the enhanced target dataset.

6. The method according to claim 1, characterized in that, The process of performing secondary training on the trained student network based on the pseudo-labels and confidence scores of each target domain data to obtain the target student network includes: In each pixel of each target domain data, the target pixel with a maximum confidence score greater than a preset confidence threshold is identified; The trained student network is then trained a second time based on the target pseudo-labels corresponding to the target pixels to obtain the target student network.

7. The method according to claim 6, characterized in that, The step of performing secondary training on the trained student network based on the target pseudo-labels corresponding to the target pixels to obtain the target student network includes: For the first target domain data, the first target domain data is input into the trained student network to obtain the predicted label corresponding to each pixel in the first target domain data. The first target domain data is any target domain data in the target domain dataset. Calculate the loss function value based on the predicted label and the target pseudo-label; The model parameters of the trained student network are adjusted based on the loss function value, and training continues until the training completion condition is met to obtain the target student network.

8. A model training device, characterized in that, The device includes: The preprocessing module is used to preprocess the image dataset to obtain the target dataset, which includes the source domain dataset and the target domain dataset. The training module is used to train the image segmentation student network based on the source domain dataset to obtain the trained student network. The exponential moving average module is used to perform an exponential moving average on the parameters of the trained student network to obtain the teacher network; The generation module is used to generate corresponding pseudo-labels for each pixel of each target domain data in the target domain dataset based on the teacher network; The training module is also used to perform secondary training on the trained student network based on the pseudo-labels to obtain the target student network.

9. A computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.

Images