Image segmentation model training and distillation methods, electronic devices, and storage media

Abstract

This invention provides a training and distillation method for an image segmentation model, an electronic device, and a storage medium. The training method includes: acquiring training sample images; inputting the training sample images into an image segmentation model, and iteratively training the image segmentation model based on a third preset loss function; wherein the third preset loss function includes multiple probability values ​​corresponding to multiple intersections of the set of real sample pixels and the set of actual predicted sample pixels under the target category, each probability value being related to the real and predicted values ​​of pixels belonging to the target category in the corresponding intersection, and the predicted value of each probability value being non-linearly decayed under the action of a third preset decay parameter in the third preset loss function. This helps to accelerate the convergence speed of the model, facilitating the acquisition of a high-performance image segmentation model in a shorter time.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a training method for an image segmentation model, a knowledge distillation method applied to an image segmentation model, an electronic device, and a storage medium. Background Technology

[0002] Image segmentation has been an active research area in recent years, with applications such as addressing vulnerabilities in the medical field and benefiting the general public. In image segmentation, image segmentation models are typically used to segment images. These models are usually trained using training sample images and based on a loss function.

[0003] Currently, training image segmentation models suffers from the following problems: not only is the model convergence speed slow and training time-consuming, but also, in scenarios with an imbalance in the number of foreground and background samples, the trained model performs poorly, with a high false negative rate, making it impossible to obtain a model with good performance. Summary of the Invention

[0004] The present invention addresses the aforementioned problems. It provides a training method for an image segmentation model, a knowledge distillation method applied to the image segmentation model, an electronic device, and a storage medium.

[0005] According to one aspect of the present invention, a training method for an image segmentation model is provided. The training method includes: acquiring training sample images; inputting the training sample images into an image segmentation model, and iteratively training the image segmentation model based on a third preset loss function; wherein the third preset loss function includes: multiple probability values ​​corresponding to multiple intersections of the set of real sample pixels and the set of actual predicted sample pixels under the target category, each probability value being related to the real value and predicted value of the pixels in the corresponding intersection belonging to the target category, and the predicted value of each probability value being non-linearly attenuated under the action of a third preset attenuation parameter in the third preset loss function.

[0006] For example, the set of real sample pixels includes the set of real positive sample pixels and the set of real negative sample pixels, the set of actual predicted sample pixels includes the set of actual predicted positive sample pixels and the set of actual predicted negative sample pixels, and the multiple intersections include a first intersection, a second intersection, and a third intersection, wherein the first intersection is the intersection of the set of real positive sample pixels and the set of actual predicted positive sample pixels under the target category; the second intersection is the intersection of the set of real positive sample pixels and the set of actual predicted negative sample pixels under the target category; and the third intersection is the intersection of the set of real negative sample pixels and the set of actual predicted positive sample pixels under the target category.

[0007] For example, the probability values ​​corresponding to the second intersection and the third intersection have corresponding weight coefficients, and the weight coefficient of the probability value corresponding to the second intersection is greater than the weight coefficient of the probability value corresponding to the third intersection.

[0008] For example, the third preset loss function = 1 - preset loss coefficient, where the preset loss coefficient is the cross-union ratio function, and multiple probability values ​​exist in the denominator of the cross-union ratio function.

[0009] For example, the third preset loss function is the Dice loss function.

[0010] For example, the third preset loss function is only related to the rare class used as the target class among all categories contained in all training sample images, or the third preset loss function is related to all categories used as the target class among all training sample images, where the rare class refers to the class whose number of annotations or annotation ratio is less than a preset threshold among all categories contained in the training sample images.

[0011] For example, the numerator and denominator of the Dice loss coefficient in the Dice loss function are both based on the probability value plus a smoothing term coefficient, and the smoothing term coefficient is not equal to 0.

[0012] For example, the denominator of the preset loss coefficient includes the sum of TP, αFN, and βFP, where α and β are weighting coefficients; TP = T c *P c n1 TP represents the true values ​​T of all pixels in the first intersection under target category c. c and predicted value P c The product of n1 powers and summation; FN = (1 - P c ) n2 *T c FN represents the true values ​​T of all pixels in the second intersection under target category c. c and predicted value (1-P) c The product of n² and summation; FP = P c n3 *(1-Tc), where FP represents the true value 1-Tc and the predicted value P of all pixels in the third intersection under target category c. c The product of n3 and summation; n1, n2 and n3 are the third preset attenuation parameters, and are all greater than 1.

[0013] For example, n1 = n2 = n3.

[0014] For example, n1 = n2 = n3 = 2.

[0015] For example, the rarer the target category, the larger the weight coefficient of the probability value corresponding to the second intersection under that target category.

[0016] For example, the weight coefficients of the probability values ​​corresponding to the second and third intersections under different target categories are set separately.

[0017] According to another aspect of the present invention, a knowledge distillation method for image segmentation models is also provided. The knowledge distillation method includes: acquiring training sample images; inputting the training sample images into a student segmentation model and a teacher segmentation model respectively for model training, and calculating the loss based on a distillation loss function during the training process; wherein the distillation loss function includes a soft loss function calculated based on soft targets and a hard loss function calculated based on hard targets, wherein the hard loss function is implemented using the aforementioned third preset loss function.

[0018] For example, the soft loss function is implemented using either a first preset loss function or a second preset loss function.

[0019] The first preset loss function includes the sum of the following cross-entropies:

[0020] The cross-entropy corresponding to the first category: The cross-entropy value is calculated based on the predicted value of the first category under the preset logits adjustment parameter;

[0021] The cross-entropy corresponding to the second category with the decay factor: the decay factor will non-linearly decay (1 - the predicted value of the second category) under the first preset decay parameter;

[0022] The second preset loss function includes the sum of cross-entropy corresponding to multiple categories;

[0023] The cross-entropy of each category has a preset decay factor, which decays non-linearly by (1 - the predicted value of that category) under a second preset decay parameter;

[0024] For each category, a predicted value is obtained under the preset logits adjustment parameter, and the cross-entropy of the category is calculated based on the predicted value.

[0025] For example, the first category is a rare class and the second category is a non-rare class. The rare class refers to the category whose number of annotations or annotation ratio is less than a preset threshold among all categories included in the training sample image.

[0026] For example, the first preset attenuation parameter is γ, and the attenuation factor is (1 - the predicted value of the second category) raised to the power of γ, where γ is a non-negative number; or,

[0027] The second preset attenuation parameter is γ, and the attenuation factor is (1 - the predicted value corresponding to the category) raised to the power of γ, where γ is a non-negative number.

[0028] According to another aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the above-described image segmentation model training method and the above-described knowledge distillation method.

[0029] According to another aspect of the present invention, a storage medium is also provided, storing a computer program / instructions that, when executed by a processor, implement the above-described image segmentation model training method and the above-described knowledge distillation method.

[0030] According to the multi-image segmentation model training method, knowledge distillation method applied to the image segmentation model, electronic device and storage medium of the present invention, the predicted value is nonlinearly attenuated by a preset attenuation parameter, which helps to accelerate the convergence speed of the model and facilitates obtaining a high-performance image segmentation model in a short time, especially when the number of foreground and background samples is unbalanced, resulting in a low false negative rate. Attached Figure Description

[0031] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0032] Figure 1 A schematic flowchart illustrating a training method for an image segmentation model according to an embodiment of the present invention is shown.

[0033] Figure 2 A schematic flowchart illustrating a training method for an image segmentation model according to an embodiment of the present invention is shown.

[0034] Figure 3 A schematic flowchart illustrating a training method for an image segmentation model according to an embodiment of the present invention is shown.

[0035] Figure 4 A schematic flowchart of a knowledge distillation method applied to an image segmentation model according to an embodiment of the present invention is shown;

[0036] Figure 5 A schematic diagram of a knowledge distillation model applied to an image segmentation model according to an embodiment of the present invention is shown;

[0037] Figure 6 This invention illustrates the use of a concrete crack dataset based on an embodiment of the present invention. Figure 1 The diagram illustrates the trial results of training the image segmentation model using the training method shown; and

[0038] Figure 7 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0040] To at least partially solve the above problems, embodiments of the present invention provide a training method for an image segmentation model. Figure 1 A schematic flowchart of a training method 100 for an image segmentation model according to an embodiment of the present invention is shown. Figure 1 As shown, the training method 100 may include the following steps S110 and S120.

[0041] Step S110: Obtain training sample images.

[0042] For example, training sample images can be images containing any target object, such as a vehicle, a person or part of a person (e.g., a face), an animal, a building, etc. Training sample images can be still images or any video frame from a moving video. Training sample images can be raw images captured by an image acquisition device (e.g., an image sensor in a camera), or images obtained after preprocessing the raw images (e.g., digitization, normalization, smoothing, etc.). It is understood that preprocessing the raw images can include extracting sub-images containing the target objects from the raw images captured by the image acquisition device to obtain the training sample images.

[0043] For example, in an industrial setting, the training sample images can be images that include defects, referred to as foreground (targets), and the regions of the image other than the foreground are referred to as background or background.

[0044] It should be noted that the training sample images are labeled with labeled data (or referred to as label values, ground truth values, labeled data, ground truth data, etc.).

[0045] Step S120: Input the training sample images into the image segmentation model and iteratively train the image segmentation model based on the third preset loss function. The third preset loss function includes multiple probability values ​​corresponding to various intersections of the set of real sample pixels and the set of actual predicted sample pixels under the target category. Each probability value is related to the real and predicted values ​​of pixels belonging to the target category in the corresponding intersection. Furthermore, the predicted value of each probability value is configured to undergo non-linear decay under the action of the third preset decay parameter within the third preset loss function.

[0046] For example, the training sample images obtained in step S110 can be input into the image segmentation model, and the image segmentation model can be iteratively trained based on the third preset loss function. After multiple iterations, an image segmentation model that meets the requirements can be obtained. The training sample images are used to train the image segmentation model. Those skilled in the art will understand that when obtaining the training sample images, the corresponding labeled data can also be obtained. The labeled data are pre-labeled, real image segmentation results. When iteratively training the image segmentation model, the labeled data and the prediction results of the image segmentation model for the training sample images can be substituted into the third preset loss function to calculate the loss value, and based on the loss value, the parameters in the image segmentation model (including the weights and biases of each network layer, etc.) can be optimized using backpropagation and gradient descent algorithms.

[0047] The third preset loss function may include: multiple probability values ​​corresponding to multiple intersections of the set of real sample pixels and the set of actual predicted sample pixels under the target category. Each probability value is related to the real value and predicted value of the pixel in the corresponding intersection belonging to the target category. Furthermore, the predicted value of each probability value is configured in the third preset loss function to undergo non-linear decay under the action of the third preset decay parameter.

[0048] The image segmentation model training method according to embodiments of the present invention uses a third preset attenuation parameter to nonlinearly attenuate the predicted values. This helps accelerate the convergence speed of the model, facilitating the acquisition of a high-performance image segmentation model in a shorter time, especially in situations where the number of foreground and background samples is unbalanced. This accelerates the training of small target segmentation tasks, and the trained model has a low false negative rate. A small target can be understood as a foreground element that occupies a small proportion of the image, such as a small crack appearing in a large image. Since the number of real training samples obtained in real industrial scenarios is small, resulting in an imbalance between foreground and background samples, the image segmentation model training method of the present invention can be well applied in industrial scenarios.

[0049] For example, the set of real sample pixels may include a set of real positive sample pixels and a set of real negative sample pixels, and the set of actual predicted sample pixels may include a set of actual predicted positive sample pixels and a set of actual predicted negative sample pixels. The above-mentioned multiple intersections may include a first intersection, a second intersection, and a third intersection, wherein the first intersection is the intersection of the set of real positive sample pixels and the set of actual predicted positive sample pixels under the target category; the second intersection is the intersection of the set of real positive sample pixels and the set of actual predicted negative sample pixels under the target category; and the third intersection is the intersection of the set of real negative sample pixels and the set of actual predicted positive sample pixels under the target category.

[0050] The first intersection can be represented by TP, which represents the pixels that are correctly identified as positive samples after the image segmentation model. The second intersection can be represented by FN, which represents the pixels that are incorrectly identified as negative samples after the image segmentation model. The third intersection can be represented by FP, which represents the pixels that are incorrectly identified as positive samples after the image segmentation model.

[0051] For example, TP, FN, and FP can be calculated in the following way:

[0052] TP = T c *P c That is, TP can be the true values ​​T of all pixels in the first intersection under the target category c. c and predicted value P c It is expressed as the product of and sum of the products.

[0053] FN=(1-P c )*T c That is, FN can be calculated using the true values ​​T of all pixels in the second intersection of the target class c. c and predicted value (1-P) c It is expressed as the product of () and summed.

[0054] FP = P c *(1-T c That is, FP can use the true values ​​(1-T) of all pixels in the third intersection under the target class c. c ) and predicted value P c It is expressed as a product of the products and summed.

[0055] T c This refers to the true value of class C (also known as the label value; typically, if the pixel is class C, then T...). c It's 1, otherwise T c That is, 0), P c This refers to the predicted value for class c. It can be understood that for a true positive sample, its true value can be represented by T. cThis means that for a true negative sample, its true value can be represented by (1-T). c This means that, for the actual predicted positive sample, its predicted value can be represented by P. c This means that for the actual predicted negative sample, its predicted value can be expressed as (1-P). c )express.

[0056] This embodiment, by simultaneously employing the above three intersections, can comprehensively consider various prediction scenarios for positive and negative samples, which helps in training an image classification model with better prediction performance.

[0057] For example, the probability values ​​corresponding to the second intersection and the third intersection have corresponding weight coefficients, and the weight coefficient of the probability value corresponding to the second intersection is greater than the weight coefficient of the probability value corresponding to the third intersection.

[0058] In one embodiment, the probability values ​​corresponding to the second intersection may have corresponding weight coefficients α and β, respectively, where the weight coefficient α of the probability values ​​corresponding to the second intersection is greater than the weight coefficient β of the probability values ​​corresponding to the third intersection. Also, exemplarily, α + β = 1.

[0059] In this embodiment, based on the weight coefficients α and β, the focus on different data intersections during the loss calculation process can be easily adjusted, further helping to solve the false negative problem caused by imbalanced positive and negative samples. Furthermore, assigning different weights to FN and FP (a larger α, a smaller β) allows the third preset loss function to focus more on training FN and less on FP, thus improving the recall rate for small targets.

[0060] For example, the third preset loss function = 1 - preset loss coefficient, where the preset loss coefficient is the cross-union ratio function, and multiple probability values ​​exist in the denominator of the cross-union ratio function.

[0061] In one embodiment, the aforementioned preset loss coefficient refers to the sum of the loss coefficients of all target categories for all samples, that is: the third preset loss function can be equal to 1 - the sum of the loss coefficients of all target categories for all samples.

[0062] In another embodiment, the aforementioned preset loss coefficient refers to the sum of the loss coefficients of all target categories for all samples, based on the mean of all samples. That is, the third preset loss function can also be equal to 1 - 1 / N (the sum of the loss coefficients of all target categories for all samples), where N is the total number of samples in a single iteration. By using the intersection-union function, the prediction accuracy of the image segmentation model can be measured relatively well based on multiple intersections, which helps to improve the training effect of the image segmentation model.

[0063] For example, the third preset loss function can be the Dice loss function.

[0064] In one embodiment, the third preset loss function can be the Dice loss function. The Dice loss function for binary classification can be represented as TPL(P,T), as follows:

[0065]

[0066] Where c represents the target category, c=1 represents the foreground region (i.e., positive sample), and c=0 represents the background region (i.e., negative sample); N represents the total number of samples in each iteration, and T c P represents the true value of class c. c This represents the predicted value for class c, where γ is the smoothing term coefficient; α and β represent weighting coefficients, n represents the third preset decay parameter, and α + β = 1. The value of γ can be set manually as needed.

[0067] For example, the third preset loss function is only related to the rare class used as the target class among all categories contained in all training sample images, or the third preset loss function is related to all categories used as the target class among all training sample images, where the rare class refers to the class whose number of annotations or annotation ratio is less than a preset threshold among all categories contained in the training sample images.

[0068] In this embodiment, a rare class can represent a class among all categories included in the training sample images whose number of annotations or annotation ratio is less than a preset threshold. The number of annotations can be understood as the number of labeled training sample images. The annotation ratio of any category is the proportion of the number of annotations for the current category to the total number of annotations for all categories. The preset threshold can be set to any suitable value as needed, and this invention does not limit this. For example, all categories included in the training sample images may include people, trees, and vehicles. Wherein, the number of annotations belonging to the category of people is 100, the number of annotations belonging to the category of trees is 80, and the number of annotations belonging to the category of vehicles is 20, and the preset threshold is 80 or 2 / 5. The number of annotations belonging to the category of people accounts for 1 / 2 of the total number of annotations for all categories, the number of annotations belonging to the category of trees accounts for 2 / 5 of the total number of annotations for all categories, and the number of annotations belonging to the category of vehicles accounts for 1 / 10 of the total number of annotations for all categories. Therefore, it can be determined that the number of annotations or annotation ratio of people and trees is greater than or equal to the preset threshold, and the number of annotations or annotation ratio of vehicles is less than the preset threshold; thus, vehicles belong to the rare class. The third preset loss function can be associated only with the rare class used as the target class among all categories contained in all training sample images. In the binary image segmentation scenario, the rare class is the foreground and the non-rare class is the background. For example, in the above Dice loss function calculation formula, only the true value and predicted value corresponding to the rare class can be substituted into the formula for calculation, while ignoring the information of other categories. Of course, optionally, the third preset loss function can also be associated with all categories contained in all training sample images used as the target class. That is, it is not necessary to distinguish between rare and non-rare classes, but to add the true value and predicted value of all categories to the calculation of the third preset loss function. For multi-class image segmentation, compared with using only rare classes as the target class, the intersection-union ratio (IOU) is better and the model training effect is better. For binary image segmentation, the training effect of using only rare classes as the target class and using all classes as the target class is not much different. Therefore, only rare classes are usually used as the target class, as shown in the above formula (1).

[0069] In one embodiment, the numerator and denominator of the Dice loss coefficient in the Dice loss function are both based on the probability value plus a smoothing term coefficient, and the smoothing term coefficient is not equal to 0.

[0070] The smoothing term coefficient can be represented by γ. This effectively avoids the situation where the denominator of the DICE loss coefficient is 0. Furthermore, adding the γ term allows users to manually adjust the smoothing term coefficient according to their needs and the application scenario of the DICE loss function.

[0071] For example, the denominator of the preset loss coefficient includes the sum of TP, αFN, and βFP, where α and β are weighting coefficients; TP = T c *P c n1 TP represents the true values ​​T of all pixels in the first intersection under target category c. c and predicted value P c The product of n1 powers and summation; FN = (1 - P c ) n2 *T c FN represents the true values ​​T of all pixels in the second intersection under target category c. c and predicted value (1-P) c The product of n² and summation; FP = P c n3 *(1-T c ), where FP represents the true values ​​(1-T) of all pixels in the third intersection under target category c. c and predicted value P c The product of n3 and summation; n1, n2 and n3 are the third preset attenuation parameters, and are all greater than 1.

[0072] In the above formula (1) for calculating the Dice loss function, the third preset attenuation parameter corresponding to TP, FN, and FP is shown to be n. However, this is only a simplified representation. These three intersections can each have their own corresponding third preset attenuation parameters n1, n2, and n3. That is to say, the denominator of the loss coefficient in the above Dice loss function can also include: T c *P c n1 +α(1-P c ) n2 *T c +βP c n3 *(1-T c ), where n1, n2, and n3 are all greater than 1.

[0073] Based on the aforementioned smoothing term coefficients, the denominator of the loss coefficients in the above Dice loss function can be further expressed as: T c *P c n1 +α(1-P c ) n2 *T c +βP c n3 *(1-T c )+γ.

[0074] The third preset attenuation parameters n1, n2, and n3 are set separately, which makes it convenient to adjust the attenuation rate of TP, FN, and FP independently as needed. This scheme is flexible in implementation and has a wider range of applications.

[0075] For example, n1 = n2 = n3.

[0076] Preferably, n1, n2, and n3 are equal. The value of n can affect the convergence speed and accuracy of the image segmentation model. The higher the value of n, the more obvious the model convergence acceleration. However, at the same time, too high a value of n will affect the accuracy of the image segmentation model. Therefore, the values ​​of n1, n2, and n3 can be set as needed. The scheme with n1 = n2 = n3 has fewer parameters, which is convenient for calculation and parameter adjustment.

[0077] For example, n1 = n2 = n3 = 2.

[0078] Preferably, n1, n2, and n3 are equal to 2. As mentioned above, the higher the value of n, the more obvious the acceleration of the image segmentation model convergence, but it will affect the accuracy of the image segmentation model. Therefore, when n1 = n2 = n3 = 2, a better balance can be achieved between model acceleration and result accuracy.

[0079] For example, the rarer the target category, the larger the weight coefficient of the probability value corresponding to the second intersection under that target category.

[0080] Preferably, the rarer the target category, the larger the weight coefficient α of the probability value corresponding to the second intersection under that target category. By adjusting the weight coefficient of the probability value corresponding to the second intersection, the attention given to the probability value corresponding to the second intersection in the loss calculation process can be further adjusted. The rarer the target category, the larger the weight coefficient, and the higher the attention given to it, resulting in better segmentation performance of the image segmentation model.

[0081] For example, the weight coefficients of the probability values ​​corresponding to the second and third intersections under different target categories are set separately.

[0082] In one embodiment, the probability values ​​α and β of the second intersection and the third intersection for different target categories can be set independently. For example, the probability value corresponding to the second intersection for the first target category can be α1, and the probability value corresponding to the third intersection can be β1; the probability value corresponding to the second intersection for the second target category can be α2, and the probability value corresponding to the third intersection can be β2. α1 and α2 are independent of each other, and β1 and β2 are also independent of each other. This scheme of setting weight coefficients independently is flexible and facilitates independent adjustment of the attention given to various categories during the training process.

[0083] The above describes an exemplary third pre-defined loss function for binary classification, namely formula (1). Furthermore, for both binary and multi-class classification (more than 2 classes), the following general formula for the third pre-defined loss function also exists:

[0084]

[0085] Where i represents the i-th category as the target category c, M represents the total number of target categories, and the meanings of the other parameters are the same as the corresponding parameters in the above formula (1), and will not be repeated here.

[0086] According to a second aspect of the present invention, a method for training an image segmentation model is provided. Figure 2 A schematic flowchart of a training method 200 for an image segmentation model according to an embodiment of the present invention is shown. Figure 2 As shown, the training method 200 may include the following steps S210 and S220.

[0087] Step S210: Obtain training sample images.

[0088] For example, the method for obtaining training sample images is similar to step S110 above, and will not be repeated here for the sake of brevity.

[0089] Step S220: Input the training sample images into the image segmentation model and iteratively train the image segmentation model based on the first preset loss function; wherein, the first preset loss function includes the sum of the following cross-entropies: cross-entropy corresponding to the first category: the cross-entropy value is calculated based on the predicted value of the first category under the preset logits adjustment parameter; cross-entropy corresponding to the second category with a decay factor: the decay factor nonlinearly decays (1-the predicted value of the second category) under the first preset decay parameter.

[0090] For example, the training sample images obtained in step S210 can be input into the image segmentation model to be trained. The image segmentation model is iteratively trained based on the first preset loss function. After multiple iterations, a satisfactory image segmentation model can be obtained. The iterative training method of the image segmentation model has been described above and will not be repeated here. For distinction, the preset loss function involved in the image segmentation model training method 100 can be called the third preset loss function, the preset loss function involved in the image segmentation model training method 200 can be called the first preset loss function, and the preset loss function involved in the image segmentation model training method 300 described below can be called the second preset loss function.

[0091] In one embodiment, the first preset loss function used by the image segmentation model can be an asymmetric loss function. An asymmetric loss function refers to using different loss calculation methods for different categories. For example, the first preset loss function may include the sum of the cross-entropy corresponding to the first category and the second category. The calculation methods for the cross-entropy corresponding to the first category and the second category are different, as described below.

[0092] The cross-entropy corresponding to the first category may include the cross-entropy value calculated based on the predicted value of the first category under the preset logits adjustment parameter; the cross-entropy corresponding to the second category may include the attenuation factor that non-linearly attenuates (1 - the predicted value of the second category) under the first preset attenuation parameter.

[0093] By way of example, but not limitation, the cross-entropy corresponding to the first category can be the cross-entropy value calculated based on the product of the true value and the predicted value corresponding to the first category. By adjusting the size of the preset logits adjustment parameter, compared with the prior art which does not set a preset logits adjustment parameter to improve the predicted value of the rare class, the loss of the rare class samples with fewer samples can be completely preserved, and the decision boundary is shifted towards the non-rare class. The trained model has better generalization ability and prevents overfitting. Therefore, it helps to solve the problem of overfitting in image segmentation models caused by imbalanced samples. The prior art does not set a preset logits adjustment parameter, that is, the predicted value corresponding to the first category is the predicted value calculated based on the logits value of the first category, which can be understood as P c This refers to the predicted value calculated based on the logits value of the first category in conventional techniques. However, in this embodiment, the predicted value corresponding to the first category is the predicted value obtained under a preset logits adjustment parameter. In this embodiment, by presetting the logits adjustment parameter, the predicted value of the first category can be improved compared to the predicted value in the prior art.

[0094] By way of example, but not limitation, the cross-entropy corresponding to the second category can be a cross-entropy value calculated based on the product of the decay term, the true value corresponding to the second category, and the predicted value corresponding to the second category, where the decay term is less than 1. The decay term can be understood as a decay factor.

[0095] For example, the first category can be a rare class, and the second category can be a non-rare class. Optionally, in a scenario where the image segmentation model is applied to binary classification, the rare class can be the foreground class, and the non-rare class can be the background class. Optionally, in a scenario where the image segmentation model is applied to multi-class classification (more than 2 categories), the rare class can be a subset of multiple categories, and the non-rare class can be another subset of multiple categories.

[0096] According to the image segmentation model training method 200 of this invention, by adjusting the preset logtis parameter in the loss function, the loss of rare class samples with fewer samples can be completely preserved, and the decision boundary is shifted towards the non-rare class. The trained model has better generalization ability and prevents overfitting. Therefore, it helps to solve the problem of overfitting in image segmentation models caused by imbalanced samples. By using the first preset decay parameter in the loss function to decay the weight of the second class, the losses of the first and second classes can be balanced, that is, the losses of rare and non-rare classes can be balanced. This is used to solve the problem of insufficient loss for small targets caused by class imbalance, thereby helping to reduce the false negative rate and false positive rate. In summary, the image segmentation model trained in this way has strong universality for image segmentation of small targets under imbalanced samples or small sample learning.

[0097] In one embodiment, the formula for the first preset loss function for multi-class classification can be as follows:

[0098]

[0099] Where N represents the total number of training sample images in each training iteration; T c＝i This represents the true value when the current category c is the first category i; This represents the predicted value obtained based on the preset logits adjustment parameter m when the current category c is the first category i; T c≠i This represents the true value when the current category c is any of the second categories; P c≠i This represents the predicted value when the current category c is any of the second categories; γ represents the first preset decay parameter.

[0100] In another embodiment, the formula for the first preset loss function for binary classification can be as follows:

[0101]

[0102] Where c represents the current category, c=1 represents the foreground region (i.e., the first category), and c=0 represents the background region (i.e., the second category); N represents the total number of training sample images in each training iteration; Tc represents the true value when the current category c is the first category; This represents the predicted value obtained by adjusting the parameter m based on the preset logits when the current category c is the first category; (1-T) c This represents the true value when the current category c is the second category; This represents the predicted value when the current category c is the second category; γ represents the first preset decay parameter.

[0103] It's important to note that in binary classification, only foreground and background classes are included. The first category can be either foreground or background, and the second category can be the other. In multi-class classification, multiple categories are included. The first category can be some of the multiple categories, and the second category is another set of categories. That is, in multi-class classification, one or more categories can belong to the first category, and one or more categories can belong to the second category.

[0104] For example, the first category is a rare class, and the second category is a non-rare class. A rare class refers to a category whose number of annotations or annotation ratio is less than a preset threshold among all categories contained in the training sample image. All categories contained in the training sample image refer to all categories to be classified, and these categories are usually preset.

[0105] The definitions of rare and non-rare classes can be found in the description above, and will not be repeated here.

[0106] For example, the first preset attenuation parameter is γ; the attenuation factor is (1 - the predicted value of the second category) to the power of γ, where γ is a non-negative number.

[0107] Referring to formula (3) above, the attenuation factor can be expressed as (1-P) c≠i ) γ This approach allows for a faster decay of the second-class weights, which can further help address the problem of imbalanced samples.

[0108] For example, the value range of γ is [1, 5].

[0109] Preferably, the value of γ can be in the range of [1, 5]. The value of γ is related to the imbalance of the sample, where the higher the imbalance of the sample, the larger the value of γ can be.

[0110] For example, the predicted value of the first category is obtained based on the difference between the logits value of each first category and the preset logits adjustment parameter m, where m is a non-negative number.

[0111] Optionally, the predicted value for the first category can be a predicted value obtained based on the corresponding logits-m value for that category. Those skilled in the art will understand that logits is the output value of the last layer of the image segmentation model, which is processed by the final activation function to obtain the predicted value. The predicted value obtained based on the logits-m value is the result obtained by replacing the logits value with the logits-m value and inputting it into the activation function for processing. For example, when the image segmentation model is applied to binary classification, the activation function used by the image segmentation model can be the Sigmoid function; when the image segmentation model is applied to multi-class classification, the activation function used by the image segmentation model can be the softmax function.

[0112] By adjusting the parameter m by presetting logits, the loss function can retain the loss of the first class of samples (e.g., rare class) as much as possible, and shift the decision boundary towards the second class (e.g., non-rare class). This can give the trained image segmentation model better generalization ability and prevent overfitting.

[0113] For example, the value range of m is [0.5, 1.5].

[0114] Preferably, the value of m can be in the range of [0.5, 1.5], and the current range of m can correspond to a sample imbalance of 3% to 10%. Here, a sample imbalance of 3% means that the number of samples belonging to the first category (e.g., the foreground class) accounts for only 3% of all samples, and other proportions are similar. The higher the sample imbalance, the larger the value of m can be.

[0115] By way of example and not limitation, the above m and γ can be set and adjusted manually to adjust the effect of model training.

[0116] According to a third aspect of the present invention, a method for training an image segmentation model is also provided. Figure 3 A schematic flowchart of a training method 300 for a multi-class image segmentation model according to an embodiment of the present invention is shown. Figure 3 As shown, the training method 300 may include the following steps S310 and S320.

[0117] Step S310: Obtain training sample images.

[0118] For example, the method for obtaining training sample images is similar to step S110 above, and will not be repeated here for the sake of brevity.

[0119] Step S320: Input the training sample images into the image segmentation model and iteratively train the image segmentation model based on the second preset loss function; wherein, the second preset loss function includes the sum of cross-entropy corresponding to multiple categories; the cross-entropy of each category has a preset decay factor, and the decay factor will non-linearly decay (1-the predicted value of the category) under the second preset decay parameter; the predicted value of each category is obtained under the preset logits adjustment parameter, and the cross-entropy of the category is calculated based on the predicted value.

[0120] For example, the training sample images obtained in step S310 can be input into the image segmentation model. The image segmentation model is then iteratively trained based on the second preset loss function. After multiple iterations, a satisfactory image segmentation model can be obtained. The training sample images are used to train the image segmentation model. Those skilled in the art will understand that when obtaining the training sample images, corresponding labeled data can also be obtained. The labeled data are pre-labeled, real image segmentation results. During iterative training of the image segmentation model, the labeled data and the prediction results of the image segmentation model for the training sample images can be substituted into the second preset loss function to calculate the loss value. Based on the loss value, backpropagation and gradient descent algorithms are used to optimize the parameters in the image segmentation model (including the weights and biases of each network layer).

[0121] In one embodiment, the second preset loss function used by the image segmentation model can be a symmetric loss function. A symmetric loss function means that the same loss calculation method is used for different categories. For example, the second preset loss function may include the sum of the cross-entropies corresponding to each of the multiple categories. The calculation method for the cross-entropies corresponding to the multiple categories is the same, which can be understood with reference to the description below. The above calculation method is the same, meaning that the calculation formulas corresponding to different categories are the same, but the magnitudes of the parameters in the calculation formulas for different categories, such as the magnitudes of the second preset attenuation parameter and / or the preset logits adjustment parameter, can be different.

[0122] The cross-entropy corresponding to each category has the following conditions: it has a preset decay factor, which decays (1 - the predicted value of the category) non-linearly under a second preset decay parameter; each category obtains the predicted value of the category under a preset logits adjustment parameter, and the cross-entropy of the category can be calculated based on the predicted value.

[0123] By way of example, but not limitation, the cross-entropy corresponding to each category can be the cross-entropy value calculated based on the product of the decay term, the true value corresponding to that category, and the predicted value corresponding to that category. By adjusting the size of the preset logits adjustment parameter, compared with the prior art which does not set the preset logits adjustment parameter to improve the predicted value of rare classes, the loss of rare class samples with fewer samples can be completely preserved, and the decision boundary is shifted towards non-rare classes. The trained model has better generalization ability and prevents overfitting. Therefore, it helps to solve the problem of overfitting in image segmentation models caused by imbalanced samples. The decay term can be understood as the decay factor.

[0124] In the prior art, the predicted value without a preset logits adjustment parameter can be understood as P. cIn conventional techniques, the predicted value is calculated based on the logits value of the current category. However, in this embodiment, the predicted value for any category is the predicted value obtained under a preset logits adjustment parameter. In this embodiment, by using the preset logits adjustment parameter, the predicted value for the current category can be improved compared to the predicted value in existing techniques.

[0125] The image segmentation model training method 300 according to an embodiment of the present invention helps to solve problems such as overfitting of the image segmentation model caused by sample imbalance by adjusting the preset logtis parameter in the loss function, which is very suitable for the segmentation of small targets. By using a second preset decay parameter in the loss function to attenuate the weights of each category, it is easy to balance the loss of each category by setting the second preset decay parameter, adjust the sample imbalance, and help reduce the false negative rate. The image segmentation model trained in this way has strong universality for the segmentation of images of small targets.

[0126] In another embodiment, the formula for the second preset loss function can be as follows:

[0127]

[0128] Where N represents the total number of training sample images in each training iteration; T c This represents the true value of the current category c; This represents the predicted value of the current category c based on the preset logits adjustment parameter m; γ represents the second preset decay parameter; M represents the total number of categories; c = i indicates that the i-th category is the current category c.

[0129] For example, the second preset attenuation parameter is γ; the attenuation factor is (1 - the predicted value corresponding to the category) to the power of γ, where γ is a non-negative number.

[0130] Referring to formula (5) above, the attenuation factor can be expressed as (1-P) c ) γ This approach allows for a relatively rapid decay of the current category weights, which can further help address the problem of imbalanced samples.

[0131] For example, the value range of γ is [1, 5].

[0132] Preferably, the value of γ can be in the range of [1, 5]. The value of γ is related to the imbalance of the sample, where the higher the imbalance of the sample, the larger the value of γ can be.

[0133] For example, the predicted value for each category is obtained based on the difference between the logits value of each category and the preset logits adjustment parameter m, where m is a non-negative number.

[0134] The meanings of the predicted values ​​obtained based on the logits value corresponding to any category and the predicted values ​​obtained based on the logits-m value corresponding to any category can be understood by referring to the description above, and will not be repeated here.

[0135] By adjusting the parameter m by presetting logits, the classification can be made more rigorous, giving the trained image segmentation model better generalization ability and preventing overfitting.

[0136] For example, the value range of m is [0.5, 1.5].

[0137] Preferably, the value of m can be in the range of [0.5, 1.5]. The current range of m values ​​corresponds to a sample imbalance of 3% to 10%. Here, a sample imbalance of 3% means that the number of foreground classes accounts for only 3%, so the higher the imbalance, the larger the value of m should be.

[0138] As mentioned above, the first preset loss function is an asymmetric loss function, while the second preset loss function is a symmetric loss function. The symmetric loss function is more suitable for scenarios with mild imbalance, while the asymmetric formula is more effective in scenarios with severe imbalance.

[0139] According to a fourth aspect of the present invention, a knowledge distillation method for image segmentation models is also provided. Figure 4 A schematic flowchart of a knowledge distillation method 400 applied to an image segmentation model according to an embodiment of the present invention is shown, such as Figure 4 As shown, the knowledge distillation method 400 may include the following steps.

[0140] Step S410: Obtain training sample images.

[0141] For example, the method for obtaining training sample images is similar to step S110 above, and will not be repeated here for the sake of brevity.

[0142] Step S420: Input the training sample images into the student segmentation model and the teacher segmentation model respectively for model training. During the training process, the loss is calculated based on the distillation loss function. The distillation loss function includes a soft loss function calculated based on soft targets and a hard loss function calculated based on hard targets. The hard loss function is implemented using the third preset loss function described in the image segmentation model training method 100.

[0143] Figure 5 A schematic diagram of a knowledge distillation model applied to an image segmentation model according to an embodiment of the present invention is shown. Figure 5As shown, the knowledge distillation model can include a student segmentation model, a teacher segmentation model, ground truth, a soft target, a soft loss function, a hard target, a hard loss function, and a distillation loss function.

[0144] For example, the training sample images obtained in step S410 can be input into the student segmentation model and the teacher segmentation model respectively for distillation training, and the model loss can be calculated based on the distillation loss function. Those skilled in the art will understand the principle and implementation of distillation training, which will not be elaborated here. The distillation loss function may include a soft loss function calculated based on soft targets and a hard loss function calculated based on hard targets.

[0145] The distillation loss function can be expressed as L distillation ,as follows:

[0146]

[0147] Where S refers to the logits output of the student segmentation model, T refers to the logits output of the teacher segmentation model, t is the temperature during model distillation, and G (Ground truth) is the true value.

[0148] For example, the soft loss function can be expressed as as follows:

[0149]

[0150] in, This represents the predicted value of the difference between the logits output S and m of the student segmentation model at distillation temperature t.

[0151] T c This represents the predicted value of the logits output T of the teacher segmentation model at distillation temperature t.

[0152] P c This represents the predicted value of the logits output S of the student segmentation model at distillation temperature t. Note that γ in this soft loss function formula is a decay parameter, different from the smoothing coefficient γ in the hard loss function. The decay parameter γ in the soft loss function formula can take values ​​in the range [1, 5].

[0153] For example, the hard loss function can be represented as TPL(P,T), as follows:

[0154]

[0155] P cTc represents the predicted value of the logits output S of the student segmentation model at distillation temperature t=1; Tc represents the ground truth, corresponding to G above.

[0156] It can be seen that the hard loss function is the third preset loss function used in the training method 100 of the image segmentation model. Furthermore, it can be seen that the soft loss function can be the first preset loss function used in the training method 200 of the image segmentation model or the second preset loss function used in the training method 300 of the image segmentation model. Of course, this is just an example; the soft loss function can also be implemented using any other suitable form of loss function.

[0157] In the above technical solution, a complex and large image segmentation model can be used as the teacher segmentation model, while the student segmentation model has a simpler structure. The teacher segmentation model is used to assist the student segmentation model in distillation training. The teacher segmentation model has a stronger learning ability and can transfer its learned knowledge to the relatively weaker student segmentation model, thereby enhancing the generalization ability of the student segmentation model. This enables the integration and transfer of datasets from two different domains.

[0158] Note that the above loss function The loss function TPL(P,T) can be used not only in the context of knowledge distillation, but also in the training context of any other suitable image segmentation model.

[0159] For example, the soft loss function is implemented using either a first preset loss function or a second preset loss function.

[0160] The first preset loss function includes the sum of the following cross-entropies:

[0161] The cross-entropy corresponding to the first category: The cross-entropy value is calculated based on the predicted value of the first category under the preset logits adjustment parameter;

[0162] The cross-entropy corresponding to the second category with the decay factor: the decay factor will non-linearly decay (1 - the predicted value of the second category) under the first preset decay parameter;

[0163] The second preset loss function includes the sum of cross-entropy corresponding to multiple categories;

[0164] The cross-entropy of each category has a preset decay factor, which decays non-linearly by (1 - the predicted value of that category) under a second preset decay parameter;

[0165] For each category, a predicted value is obtained under the preset logits adjustment parameter, and the cross-entropy of the category is calculated based on the predicted value.

[0166] For example, the first category is a rare class and the second category is a non-rare class. The rare class refers to the category whose number of annotations or annotation ratio is less than a preset threshold among all categories included in the training sample image.

[0167] For example, the first preset attenuation parameter is γ, and the attenuation factor is (1 - the predicted value of the second category) raised to the power of γ, where γ is a non-negative number; or,

[0168] The second preset attenuation parameter is γ, and the attenuation factor is (1 - the predicted value corresponding to the category) raised to the power of γ, where γ is a non-negative number.

[0169] Referring to the above description, the soft loss function can be either the first preset loss function used in the image segmentation model training method 200 or the second preset loss function used in the image segmentation model training method 300. Examples of such preset loss functions can be found in the above descriptions of image segmentation model training methods 200 or 300, and will not be repeated here.

[0170] Figure 6 This diagram illustrates the trial results of training an image segmentation model using a training method 100 based on a concrete crack (DeepCrack) dataset according to an embodiment of the present invention. Figure 6 In addition to showing the trial results of training the image segmentation model based on training method 100 (the scheme of this application), the paper also shows the trial results of training the image segmentation model based on two other existing training methods (existing scheme 1 and existing scheme 2). Figure 6 As shown, the vertical axis represents the training IOU (Intersection over Union), and the horizontal axis represents the number of training epochs. It can be seen that the proposed scheme converges to a relatively good result in about 10 training epochs, while the other two existing schemes converge after 20 training epochs.

[0171] According to a fifth aspect of the present invention, an electronic device is also provided. Figure 7 A schematic block diagram of an electronic device 700 according to an embodiment of the present invention is shown, such as... Figure 7 As shown, the electronic device 700 may include a processor 710 and a memory 720. The memory 720 stores a computer program, and the processor 710 executes the computer program to implement the above-described image segmentation model training method or the above-described knowledge distillation method applied to the image segmentation model.

[0172] According to a sixth aspect of the present invention, a storage medium is also provided. This medium stores a computer program / instructions that, when executed by a processor, implement the above-described training method for the image segmentation model or the above-described knowledge distillation method applied to the image segmentation model. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0173] Those skilled in the art can understand the specific implementation scheme of the above-mentioned electronic device and storage medium by reading the relevant descriptions of the training method 100 of the image segmentation model and the knowledge distillation method 400 applied to the image segmentation model. For the sake of brevity, they will not be described in detail here.

[0174] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0175] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0176] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0177] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A training method for an image segmentation model, characterized in that, include: Obtain training sample images; The training sample images are input into the image segmentation model, and the image segmentation model is iteratively trained based on the third preset loss function; The third preset loss function includes multiple probability values ​​corresponding to multiple intersections of the set of real sample pixels and the set of actual predicted sample pixels under the target category. Each probability value is related to the real value and predicted value of the pixels in the corresponding intersection belonging to the target category. The predicted value of each probability value is configured to undergo non-linear decay under the action of a third preset decay parameter in the third preset loss function. The third preset loss function = 1 - preset loss coefficient, the preset loss coefficient is the intersection-union ratio function, the multiple probability values ​​exist in the denominator of the intersection-union ratio function, and the third preset loss function is the Dice loss function.

2. The method according to claim 1, characterized in that, The set of real sample pixels includes a set of real positive sample pixels and a set of real negative sample pixels; the set of actual predicted sample pixels includes a set of actual predicted positive sample pixels and a set of actual predicted negative sample pixels; the multiple intersections include a first intersection, a second intersection, and a third intersection, wherein... The first intersection is the intersection of the set of real positive sample pixels and the set of actual predicted positive sample pixels under the target category; the second intersection is the intersection of the set of real positive sample pixels and the set of actual predicted negative sample pixels under the target category; and the third intersection is the intersection of the set of real negative sample pixels and the set of actual predicted positive sample pixels under the target category.

3. The method according to claim 2, characterized in that, The probability values ​​corresponding to the second intersection and the third intersection have corresponding weight coefficients, and the weight coefficient of the probability value corresponding to the second intersection is greater than the weight coefficient of the probability value corresponding to the third intersection.

4. The method according to any one of claims 1-3, characterized in that, The third preset loss function is only related to the rare class used as the target class among all categories contained in all training sample images, or the third preset loss function is related to all categories contained in all training sample images used as the target class, wherein the rare class refers to the class whose number of annotations or annotation ratio is less than a preset threshold among all categories contained in the training sample images.

5. The method according to any one of claims 1-3, characterized in that, The numerator and denominator of the Dice loss coefficient in the Dice loss function are both based on the probability value plus a smoothing term coefficient, and the smoothing term coefficient is not equal to 0.

6. The method according to any one of claims 1-3, characterized in that, The multiple intersections include a first intersection, a second intersection, and a third intersection. The denominator of the preset loss coefficient includes the sum of TP, αFN, and βFP, where α and β are weighting coefficients. TP = T c * P c n1 , TP denotes the product of the true values T c and the predicted values P c of the n1 powers and summed up for all pixels in the first intersection under the target class c; FN = (1-P c ) n2 *T c ,FN represents the product of the true value T c and the predicted value (1-P c ) raised to the power of n2 for all pixels in the second intersection under the target class c and summed. FP = P c n3 *(1-Tc), FP represents the true value (1-Tc) of all pixels in the third intersection under the target category c. c and predicted value P c Sum the products of n^3. n1, n2, and n3 are the third preset attenuation parameters, and each is greater than 1.

7. The method according to claim 6, characterized in that, n1 = n2 = n3.

8. The method according to claim 7, characterized in that, n1 = n2 = n3 = 2.

9. The method according to claim 3, characterized in that, The rarer the target category, the greater the weighting coefficient of the probability value corresponding to the second intersection under that target category.

10. The method according to claim 3, characterized in that, The weighting coefficients of the probability values ​​corresponding to the second and third intersections under different target categories are set separately.

11. A knowledge distillation method applied to image segmentation models, characterized in that, include: Obtain training sample images; The training sample images are input into the student segmentation model and the teacher segmentation model respectively for model training, and the loss is calculated based on the distillation loss function during the training process; The distillation loss function includes a soft loss function calculated based on soft targets and a hard loss function calculated based on hard targets. The hard loss function is implemented using the third preset loss function in the training method of the image segmentation model as described in any one of claims 1-10.

12. The method according to claim 11, characterized in that, The soft loss function is implemented using either a first preset loss function or a second preset loss function. The first preset loss function includes the sum of the following cross-entropies: The cross-entropy corresponding to the first category: The cross-entropy value is calculated based on the predicted value of the first category under the preset logits adjustment parameter; The cross-entropy corresponding to the second category with a decay factor: the decay factor will non-linearly decay (1 - the predicted value of the second category) under a first preset decay parameter; Wherein, the second preset loss function includes the sum of cross-entropy corresponding to multiple categories; The cross-entropy of each category has a preset decay factor, which decays non-linearly by (1 - the predicted value of the category) under a second preset decay parameter; Each category obtains a predicted value for that category under a preset logits adjustment parameter, and the cross-entropy of that category is calculated based on the predicted value.

13. The method as described in claim 12, characterized in that, The first category is a rare class, and the second category is a non-rare class. A rare class refers to a category whose number of annotations or annotation ratio is less than a preset threshold among all categories contained in the training sample image.

14. The method as described in claim 12, characterized in that, The first preset attenuation parameter is γ, and the attenuation factor is (1 - the predicted value of the second category) raised to the power of γ, where γ is a non-negative number; or, The second preset attenuation parameter is γ, and the attenuation factor is (1 - the predicted value corresponding to the category) to the power of γ, where γ is a non-negative number.

15. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the training method of the image segmentation model as described in any one of claims 1-10 or the knowledge distillation method as described in any one of claims 11-14.

16. A storage medium, characterized in that, The system stores a computer program / instructions that, when executed by a processor, implement the training method for the image segmentation model as described in any one of claims 1-10 or the knowledge distillation method as described in any one of claims 11-14.

Images