An adversarial knowledge distillation semantic segmentation method based on local structure consistency

Abstract

The application provides an adversarial knowledge distillation semantic segmentation method based on local structure consistency, comprising: obtaining a task to be segmented; inputting the task to be segmented into a student network to obtain a semantic segmentation result; the training process of the student network is as follows: obtaining a task data set; obtaining a first data set probability map according to the task data set and a teacher network; obtaining a second data set probability map according to the task data set and a candidate student network; obtaining local structure consistency according to the first data set probability map and the second data set probability map; obtaining cross entropy according to the second data set probability map; inputting the first data set probability map and the second data set probability map into an adversarial training discriminator to obtain a first discrimination probability map and a second discrimination probability map; obtaining network weights according to the cross entropy, the local structure consistency, the first discrimination probability map and the second discrimination probability map; and further obtaining the student network. By taking the local structure consistency as patch-level knowledge and the adversarial training discriminator, the precision of semantic segmentation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer machine vision, and particularly relates to an adversarial knowledge distillation semantic segmentation method based on local structure consistency. BACKGROUND

[0002] Semantic segmentation is a basic and challenging task of image understanding, which aims to predict a unique class label for each pixel of an input image. Semantic segmentation has been widely applied in autonomous driving, robot perception, image editing, video surveillance, virtual reality and medical image analysis. In recent years, the main solution to the segmentation task is the semantic segmentation model based on the fully convolutional neural network, which has made significant improvement in segmentation accuracy. However, these high-precision models usually contain millions of parameters and require high computing resources, which limits their application on low-resource devices such as embedded devices, Internet of Things devices, edge devices, AR / VR devices and smartphones. Due to the need for real-time applications, some works tend to focus on neural networks with small model size, low computational cost and high precision, which requires a lightweight semantic segmentation method that can run quickly on low-resource devices. The existing lightweight semantic segmentation method usually adopts the way of transferring pixel-level and image-level knowledge, which leads to poor prediction effect on the boundaries of images, small targets and weak classes, and further makes the semantic segmentation accuracy not high. SUMMARY

[0003] In view of the defects in the prior art, the present application provides an adversarial knowledge distillation semantic segmentation method based on local structure consistency to improve the accuracy of semantic segmentation.

[0004] In the first aspect, the present application provides an adversarial knowledge distillation semantic segmentation method based on local structure consistency, which comprises: obtaining a segmentation task; inputting the segmentation task into a student network to obtain a semantic segmentation result; the training process of the student network is as follows:

[0005] Obtain the task dataset; obtain the first dataset probability map based on the task dataset and the teacher network; obtain the second dataset probability map based on the task dataset and the candidate student network; obtain local structural consistency based on the first and second dataset probability maps; local structural consistency is used to characterize the structural similarity between the teacher network and the candidate student network; obtain cross-entropy based on the second dataset probability map; input the first and second dataset probability maps into the adversarial training discriminator to obtain the first and second discriminant probability maps; the first discriminant probability map is used to characterize the dataset probability map that the adversarial training discriminator judges as true; the second discriminant probability map is used to characterize the dataset probability map that the adversarial training discriminator judges as false; obtain network weights based on cross-entropy, local structural consistency, the first and second discriminant probability maps; obtain the student network based on the network weights and the candidate student network.

[0006] Optionally, obtaining the first dataset probability map based on the task dataset and the teacher network includes: selecting a teacher network based on the task to be segmented; freezing the parameters of the teacher network and inputting the task dataset into the teacher network with the parameters frozen to obtain the first dataset probability map.

[0007] Optionally, before obtaining the local structural consistency based on the probability maps of the first and second datasets, the method further includes: softening the probability maps of the first and second datasets.

[0008] Optionally, obtaining local structural consistency based on the probability maps of the first and second datasets includes: calculating... Achieving local structural consistency; where, This represents the KL divergence (Kullback–Leibler divergence, relative entropy) between the probability maps of the first and second datasets after softening. This is the probability map of the first dataset after softening. This is the probability map of the second dataset after softening. , The width of the probability graph. Let be the length of the probability graph.

[0009] Optionally, the cross-entropy is obtained from the probability graph of the second dataset, including: by calculating Obtain the cross-entropy; where, , For task datasets The first in One tag, The sample set of the task dataset. The set of labels for the task dataset. This is the probability map of the second dataset after softening. , is a width of the probability graph, is a length of the probability graph.

[0010] Optionally, the network weight is obtained according to the cross entropy, the local structure consistency, the first discriminant probability graph and the second discriminant probability graph, comprising: obtaining a first loss function according to the first discriminant probability graph and the second discriminant probability graph; obtaining a second loss function according to the second discriminant probability graph; obtaining a whole loss function according to the local structure consistency, the cross entropy, the first loss function and the second loss function; and obtaining the network weight according to the whole loss function.

[0011] Optionally, the first loss function is obtained according to the first discriminant probability graph and the second discriminant probability graph, comprising: obtaining the first loss function by calculating ; wherein, is the first loss function, E represents expectation, is a network function of the discriminator, is the first discriminant probability graph, is the second discriminant probability graph.

[0012] Optionally, the second loss function is obtained according to the second discriminant probability graph, comprising: obtaining the second loss function by calculating ; wherein, E represents expectation, is a network function of the discriminator, is the second loss function, is the second discriminant probability graph.

[0013] Optionally, the whole loss function is obtained according to the local structure consistency, the cross entropy, the first loss function and the second loss function, comprising: obtaining the whole loss function by calculating ; wherein, is the whole loss function, is the local structure consistency is a first hyperparameter, is a second hyperparameter of the cross entropy is a third hyperparameter of the first loss function is a fourth hyperparameter of the second loss function , , , , , , are all real numbers.

[0014] Optionally, the network weight is obtained according to the whole loss function, comprising: obtaining the network weight by using a minimum value of the whole loss function according to a gradient descent method or a gradient-free optimization algorithm.

[0015] From the above technical solutions, the beneficial technical effects of the semantic segmentation method based on local structure consistency provided by the present application are as follows:

[0016] 1. Obtain local structure consistency from the first data set probability map and the second data set probability map, then use the local structure consistency as patch-level knowledge to improve the feature extraction capability of the teacher network and the student network, and then obtain the overall loss function by using cross-entropy, local structure consistency, first discriminant probability map and second discriminant probability map, and then obtain the network weight, and obtain the student network according to the network weight and the candidate student network. Then obtain the segmentation task; input the segmentation task into the student network to obtain the semantic segmentation result. Such a semantic segmentation method based on local structure consistency provides local spatial constraints between the outputs of the student network and the teacher network, and can effectively perform knowledge distillation, improve the accuracy of semantic segmentation, and provide a student network with high accuracy and fast running on low-resource devices.

[0017] 2. Input the first data set probability map and the second data set probability map into the adversarial training discriminator to obtain the first discriminant probability map and the second discriminant probability map, and then obtain the overall loss function by using cross-entropy, local structure consistency, first discriminant probability map and second discriminant probability map, and then obtain the network weight, and use the network weight for semantic segmentation. In this way, the adversarial training discriminator is used to enable the student network to more effectively obtain the knowledge of the teacher network, thereby further improving the accuracy of semantic segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application, the drawings required to be used in the specific embodiments or prior art description will be briefly introduced below. In all the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0019] Figure 1 A schematic diagram of a semantic segmentation method based on local structure consistency provided by an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the training process of a student network provided by an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a first data set probability map output by a teacher network provided by an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a probability map output by a student network provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the technical solutions of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0024] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should be understood as the usual meaning understood by the skilled person in the field of the present application.

[0025] In addition, the terms "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0026] In conjunction with Figure 1 As shown in the figure, the embodiment provided based on the local structure consistency of the adversarial knowledge distillation semantic segmentation method includes:

[0027] Step S01, obtaining a task to be segmented;

[0028] Step S02, inputting the task to be segmented into the student network to obtain a semantic segmentation result.

[0029] In conjunction with Figure 2 As shown in the figure, the training process of the student network provided by the embodiment includes:

[0030] Step S03-1, obtaining a task data set;

[0031] Step S03-2, obtaining a first data set probability map according to the task data set and the teacher network;

[0032] Step S03-3, obtaining a second data set probability map according to the task data set and the alternative student network;

[0033] Step S03-4, obtaining a local structure consistency according to the first data set probability map and the second data set probability map; the local structure consistency is used to represent the structural similarity of the teacher network and the alternative student network;

[0034] Step S03-5, obtaining a cross-entropy according to the second data set probability map;

[0035] Step S03-6, inputting the first data set probability map and the second data set probability map into the adversarial training discriminator to obtain a first discrimination probability map and a second discrimination probability map; the first discrimination probability map is used to represent the data set probability map judged as true by the adversarial training discriminator; the second discrimination probability map is used to represent the data set probability map judged as false by the adversarial training discriminator;

[0036] Step S03-7, obtaining the network weight according to the cross entropy, the local structure consistency, the first discriminant probability map and the second discriminant probability map;

[0037] Step S03-8, obtaining the student network according to the network weight and the alternative student network.

[0038] The local structure consistency is obtained according to the first data set probability map and the second data set probability map, and then the local structure consistency is taken as patch-level knowledge, which improves the feature extraction capability of the teacher network and the student network. Then, the cross entropy, the local structure consistency, the first discriminant probability map and the second discriminant probability map are used to obtain the overall loss function, and then the network weight is obtained. The student network is obtained according to the network weight and the alternative student network. Then, the task to be segmented is obtained; the task to be segmented is input into the student network to obtain the semantic segmentation result. Such a local structure consistency-based adversarial knowledge distillation semantic segmentation method takes the local structure consistency as patch-level knowledge, can provide local spatial constraints between the outputs of the student network and the teacher network, and can more effectively perform knowledge distillation, thereby improving the accuracy of semantic segmentation and providing a high-precision student network that can run quickly on a low-resource device. The discriminator is trained adversarially, so that the student network can more effectively obtain the knowledge of the teacher network, further improving the accuracy of semantic segmentation.

[0039] Optionally, the first data set probability map is obtained according to the task data set and the teacher network, including: selecting the teacher network according to the task to be segmented; freezing the parameters of the teacher network, and inputting the task data set into the teacher network with the frozen parameters to obtain the first data set probability map.

[0040] Optionally, the task to be segmented includes a plurality of to-be-segmented pictures.

[0041] Optionally, the task data set includes a plurality of to-be-segmented picture samples and probability map labels corresponding to each to-be-segmented picture sample.

[0042] Optionally, the teacher network is selected according to the task to be segmented, including: selecting the teacher network from a plurality of alternative teacher networks according to the task to be segmented.

[0043] Optionally, the alternative teacher network is obtained by training in the following manner: initializing a network model of a transformer architecture; training the network model using an ImageNet training set, adjusting the weight of the alternative teacher network, and stopping training when the accuracy of the trained network model is greater than or equal to a first threshold value, and determining the trained network model as the alternative teacher network. Using the latest transformer as the backbone network architecture makes the feature extraction capability of the teacher network and the student network more powerful.

[0044] In some embodiments, the transformer network, such as the Swin-S series, is trained using the ImageNet training set, the weights of the teacher network are adjusted to obtain the teacher network. The parameters of the teacher network are frozen, so that the parameters of the teacher network are fixed, and the data in the task data set is input into the teacher network to obtain the first data set probability map. The size of the picture in the task data set is , , , , , , , , , , , ,

[0045] In some embodiments, the task data set is the cityspaces (cityscape) data set, and the size of the picture in the task data set is , , , ,

[0046] , ,

[0047] , ,

[0048] , ,

[0049] In some embodiments, the network architecture of the alternative student network and the network architecture of the teacher network both use the low-end series of transformer networks, for example, the teacher network architecture selects the Swin-S network, the alternative student network architecture selects the Swin-T network, and the alternative student network is obtained by He initialization on the Swin-T network.

[0050] In some embodiments, after the parameters of the alternative student network are unfrozen, the task data set data input alternative student network , obtaining a second dataset probability map of the alternative student network output .

[0051] Optionally, before obtaining the local structure consistency according to the first dataset probability map and the second dataset probability map, further comprising: softening the first dataset probability map and the second dataset probability map.

[0052] Optionally, the softening the first dataset probability map and the second dataset probability map comprises: obtaining the softened first dataset probability map or the softened second dataset probability map; wherein, is the first dataset probability map or the second dataset probability map before softening, is the first dataset probability map or the second dataset probability map after softening, , represents the student network, represents the teacher network, represents adding all elements of the matrix, and τ is the temperature.

[0053] Optionally, the temperature is used to control the smoothness of the probability map distribution; the greater the temperature, the smaller the value of the probability map, that is, the distribution is softer. In this way, when processing the first dataset probability map and the second dataset probability map, the temperature is used to soften the probability of each class, thereby improving the performance of knowledge distillation.

[0054] In some embodiments, during the training process, the same temperature is applied to the alternative student network and the teacher network, and the temperature is set to 1 to restore the standard soft-max normalization. Compared with the prior art of directly using the probability map obtained by the teacher network and the alternative student network to transfer the probability mapping of each class, the present embodiment adds temperature to soften the probability of each class when processing the probability map, thereby improving the performance of knowledge distillation through softening the first dataset probability map and the second dataset probability map.

[0055] Optionally, obtaining the local structure consistency according to the first dataset probability map and the second dataset probability map comprises: obtaining the local structure consistency by calculating is the KL divergence of the softened first dataset probability map and the softened second dataset probability map, is the softened first dataset probability map, is the softened second dataset probability map, , is the width of the probability map, ​The length is a probability map. Compared with the pixel level and image level knowledge in the prior art, the embodiment takes the local consistency as patch level knowledge, and uses a smaller semantic segmentation network to achieve more accurate segmentation results, so that the semantic segmentation can be quickly run on a real-time device.

[0056] Optionally, the KL divergence is used to measure the similarity between the two types of probability maps.

[0057] Optionally, the cross entropy is obtained according to the second data set probability map, including: obtaining the cross entropy by calculating the cross entropy; wherein, , is the i-th label in the task data set , is a sample set of the task data set, is a label set of the task data set.

[0058] Optionally, the adversarial training discriminator is composed of a convolutional neural network and a fully connected network. In some embodiments, the convolutional neural network is a Resnet (deep residual network). The adversarial training discriminator is used to enable the student network to more effectively obtain the teacher network knowledge, and further improve the accuracy of the semantic segmentation network.

[0059] Optionally, the network weight is obtained according to the cross entropy, the local structure consistency, the first discriminant probability map and the second discriminant probability map, including: obtaining the first loss function according to the first discriminant probability map and the second discriminant probability map; obtaining the second loss function according to the second discriminant probability map; obtaining the overall loss function according to the local structure consistency, the cross entropy, the first loss function and the second loss function; obtaining the network weight according to the overall loss function. The adversarial training method is used to improve the effectiveness of the student network to obtain the teacher network knowledge, and provides a faster and smaller knowledge distillation semantic segmentation method for related fields such as edge computing and service computing.

[0060] Optionally, the first loss function is obtained according to the first discriminant probability map and the second discriminant probability map, including:

[0061] the first loss function, i.e. the Wasserstein loss function, is obtained by calculating ; wherein, is the first loss function, E represents the expectation, is a network function of the adversarial training discriminator, is the first discriminant probability map, is the second discriminant probability map.

[0062] Optionally, the output size of the adversarial training discriminator is consistent with the size of the input probability map. In some embodiments, the output size of the adversarial training discriminator is​ .

[0063] Optionally, the second loss function is obtained according to the second discriminant probability map, comprising: calculating obtaining the second loss function, i.e., the Holistic loss function; wherein E represents expectation, is a network function of the discriminator for adversarial training, is the second loss function, is the second discriminant probability map.

[0064] Optionally, the overall loss function is obtained according to the local structure consistency, the cross entropy, the first loss function and the second loss function, comprising: calculating obtaining the overall loss function; wherein is the overall loss function, is the local structure consistency is the first hyperparameter, is the second hyperparameter of the cross entropy is the third hyperparameter of the first loss function is the fourth hyperparameter of the second loss function , , , , , , all are real numbers.

[0065] Optionally, is a hyperparameter for balancing the local structure consistency function, . The initial value of is 0.25, and the value of is determined by the value range of the Loss curve of the local structure consistency function before the verification of the candidate student network after completing the initial training of the candidate student network.

[0066] Optionally, is a hyperparameter for balancing the cross entropy function, . The initial value of is 0.25, and the value of is determined by the value range of the Loss curve of the cross entropy function before the verification of the candidate student network after completing the initial training of the candidate student network.

[0067] Optionally, is a hyperparameter for balancing the first loss function, . The initial value of an initial value of 0.25, and a value range of a Loss curve of the first loss function before the verification of the alternative student network after completing the initial training of the alternative student network is determined .

[0068] Optionally, is a hyperparameter for balancing the second loss function, . The preset an initial value of 0.25, and a value range of a Loss curve of the second loss function before the verification of the alternative student network after completing the initial training of the alternative student network is determined .

[0069] Optionally, the network weight is obtained according to the overall loss function, including: the network weight is obtained by using a minimum value of the overall loss function according to a gradient descent method or a derivation-free optimization algorithm.

[0070] Optionally, the network weight is obtained by calculating , that is, the network weight is solved to make the value of the overall loss function minimum; wherein, is the network weight, is the overall loss function.

[0071] Optionally, the network weight is obtained by using the gradient descent method, including: the network weight is obtained by calculating ; wherein, is a learning rate, is the network weight of the i th iteration training, is the network weight of the j th iteration training, ≥ 1, is a positive integer. Optionally, the network weight is obtained, including: the network parameter of the adversarial training discriminator is updated according to the first loss function obtained in the i th iteration; the network parameter of the adversarial training discriminator after the update is frozen. The weight of the j th iteration is loaded into the alternative student network, the task data set is input into the alternative student network, and the second data set probability graph of the j th iteration is obtained. The local structure consistency of the i th iteration and the cross entropy of the j th iteration are obtained according to the first data set probability graph and the second data set probability graph of the j th iteration. The first data set probability graph and the second data set probability graph of the j th iteration are input into the updated adversarial training discriminator, and the network weight of the k th iteration is obtained.

[0072] Optionally, the network weight is obtained, including: the network parameter of the adversarial training discriminator is updated according to the first loss function obtained in the i th iteration; the network parameter of the adversarial training discriminator after the update is frozen. The weight of the j th iteration is loaded into the alternative student network, the task data set is input into the alternative student network, and the second data set probability graph of the j th iteration is obtained. The local structure consistency of the i th iteration and the cross entropy of the j th iteration are obtained according to the first data set probability graph and the second data set probability graph of the j th iteration. The first data set probability graph and the second data set probability graph of the j th iteration are input into the updated adversarial training discriminator, and the network weight of the k th iteration is obtained. ​​​​​​​​The first discriminant probability map of the next iteration and the... The second discriminant probability map of the next iteration; based on the... The local structural consistency of the iteration, the first iteration The cross-entropy of the nth iteration, the th The first discriminant probability map of the next iteration and the... The second discriminant probability map of the next iteration obtains the first... The candidate network weights in the second iteration of training are used to stop training if the accuracy of the candidate student network is greater than or equal to the second threshold. The candidate network weights for the next iteration of training are determined as the network weights.

[0073] Optionally, obtaining a student network based on network weights and candidate student networks includes: loading network weights into candidate student networks to obtain a student network.

[0074] In some embodiments, the task to be segmented is a size The image to be segmented is input into the teacher's network to obtain the following results: Figure 3 The first dataset probability plot is shown. (Regarding hyperparameters...) Learning rate In this case, the overall loss function is The gradient descent method is used to calculate The optimal network weights are obtained and loaded into the student network. The parameters of the student network are then frozen, with the student network having two-thirds the number of parameters as the teacher network. The task to be segmented is then input into the student network to obtain the following results: Figure 4 The probability graph shown. Compare. Figure 3 and Figure 4 The segmentation accuracy of the two is almost the same. It can be seen that the language segmentation method provided in this embodiment can still guarantee sufficient segmentation accuracy when the number of parameters of the student network is two-thirds that of the teacher network. The smaller number of parameters allows the student network to run quickly on low-resource devices, thus obtaining a high-precision student network that can run quickly on low-resource devices.

[0075] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction, and the combination is within the scope of the present application.

[0076] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the specification of the present application.

Claims

1. A semantic segmentation method based on adversarial knowledge distillation and local structural consistency, characterized in that, include: Obtain the tasks to be split; The task to be segmented is input into the student network to obtain semantic segmentation results; the training process of the student network is as follows: Obtain the task dataset; Obtain the probability graph of the first dataset based on the task dataset and the teacher network; Based on the task dataset and the teacher network, obtain the probability graph of the first dataset, including: Select the teacher network based on the task to be segmented; The step of selecting the teacher network based on the task to be segmented includes: Select a teacher network from several candidate teacher networks based on the task to be segmented; The candidate teacher network was obtained through training in the following manner: Initialize the network model of the transformer architecture; The network model is trained using a Swin-S network, and the weights of the network model are adjusted. If the accuracy of the trained network model is greater than or equal to the first threshold, training is stopped, and the trained network model is determined as a candidate teacher network. The parameters of the teacher network are frozen, and the task dataset is input into the teacher network with the parameters frozen to obtain the first dataset probability map; Obtain a probability graph of the second dataset based on the task dataset and the candidate student network; The probability maps of the first and second datasets are softened, including: Through calculation Obtain the probability maps of the first and second datasets after softening. in, This is the probability map of the first dataset or the second dataset before softening. This is the probability map of either the first or second dataset after softening. , Indicates student network, Indicates the teacher's network, This represents adding all elements of the matrix, where τ is the temperature; Local structural consistency is obtained based on the probability graphs of the first and second datasets; the local structural consistency is used to characterize the structural similarity between the teacher network and the candidate student network. Obtain the cross-entropy based on the probability graph of the second dataset; The first dataset probability map and the second dataset probability map are input into the adversarial training discriminator to obtain a first discriminant probability map and a second discriminant probability map; the first discriminant probability map is used to represent the probability map of the dataset that the adversarial training discriminator judges as true; the second probability map is used to represent the probability map of the dataset that the adversarial training discriminator judges as false. The network weights are obtained based on the cross-entropy, the local structural consistency, the first discriminant probability map, and the second discriminant probability map. The student network is obtained based on the network weights and the candidate student networks.

2. The method according to claim 1, characterized in that, Obtaining local structural consistency based on the probability maps of the first and second datasets includes: Through calculation Achieving local structural consistency; where, The KL divergence between the probability maps of the first and second datasets after softening is given. Here are the probability maps for the first dataset after softening, and for the second dataset after softening. , The width of the probability graph, Let be the length of the probability graph.

3. The method according to claim 2, characterized in that, The cross-entropy is obtained from the probability graph of the second dataset, including: Through calculation Obtain the cross-entropy; where, , For task datasets The Middle A tag, For the sample set of the task dataset, The set of labels for the task dataset. This is the probability map of the second dataset after softening. , The width of the probability graph, Let be the length of the probability graph.

4. The method according to claim 1, characterized in that, The network weights are obtained based on the cross-entropy, the local structural consistency, the first discriminant probability map, and the second discriminant probability map, including: The first loss function is obtained based on the first discrimination probability map and the second discrimination probability map; The second loss function is obtained based on the second discrimination probability map; The overall loss function is obtained based on the local structural consistency, the cross-entropy, the first loss function, and the second loss function. The network weights are obtained based on the overall loss function.

5. The method according to claim 4, characterized in that, The first loss function is obtained based on the first and second discriminant probability maps, including: Through calculation Obtain the first loss function; where, Let E be the first loss function, and E represent the expectation. For the network function of the adversarial training discriminator, This is the first discriminant probability map. This is the second discriminant probability diagram.

6. The method according to claim 4, characterized in that, The second loss function is obtained based on the second discriminant probability map, including: Through calculation Obtain the second loss function; where E represents the expectation. For the network function of the adversarial training discriminator, For the second loss function, This is the second discriminant probability diagram.

7. The method according to claim 4, characterized in that, The overall loss function is obtained based on the local structural consistency, the cross-entropy, the first loss function, and the second loss function, including: Through calculation Obtain the overall loss function; where, For the overall loss function, For local structural consistency The first hyperparameter, Cross-entropy The second hyperparameter, The first loss function The third hyperparameter, For the second loss function The fourth hyperparameter, , , , All are real numbers.

8. The method according to claim 4, characterized in that, The network weights are obtained based on the overall loss function, including: The network weights are obtained by using the minimum value of the overall loss function, either by gradient descent or gradient-free optimization.

Images