Image segmentation method and device, electronic equipment and storage medium

Abstract

The invention discloses an image segmentation method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining an input image; inputting the input image into a preset multi-decision aggregation network for image segmentation prediction to obtain a segmentation prediction map; constructing a first loss function based on the weighted score, the prediction result and the annotation information, and pre-training the multi-decision aggregation network; obtaining segmentation data by using a segmentation model in the pre-trained multi-decision aggregation network; performing index evaluation on all the segmentation results based on a preset index to obtain a reward score of each segmentation result; on the basis of the reward score, a preference data pair is constructed according to the segmentation result; constructing a second loss function based on the preference data pair, and performing second training on the segmentation model; and performing image segmentation on the to-be-processed image by using the second trained multi-decision aggregation network to obtain an image segmentation result. The method can balance multiple indexes to realize accurate image segmentation, and can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image segmentation method, device, electronic equipment and storage medium. Background Art

[0002] The goal of medical image segmentation is to classify each pixel in the input medical image into a certain category in a set of categories. In this task, various evaluation indicators (such as pixel accuracy, Dice similarity coefficient, Hausdorff distance, sensitivity, etc.) are used to evaluate the segmentation performance from different perspectives. Existing methods generally design specific differentiable proxy loss functions for different indicators to optimize the segmentation model.

[0003] However, in the field of medical image segmentation, the evaluation indicators are very diverse, and some of them are not differentiable, which makes it very challenging to design a specific proxy loss function for each indicator. In addition, it is also very difficult to optimize multiple indicators at the same time during training, which requires a delicate balance of different loss functions during training. Summary of the invention

[0004] The main purpose of the embodiments of the present invention is to propose an image segmentation method, device, electronic device and storage medium in order to solve at least one problem of the prior art. The present invention can balance the loss optimization of different indicators to achieve accurate image segmentation.

[0005] To achieve the above object, an embodiment of the present invention provides an image segmentation method, which includes:

[0006] Get an input image; the input image carries the annotation information of the category of each pixel;

[0007] Input the input image into a preset multi-decision aggregation network for image segmentation prediction to obtain a segmentation prediction map;

[0008] The multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image;

[0009] A first loss function is constructed based on the weighted scores, the prediction results and the annotation information, and then the multi-decision aggregation network is pre-trained;

[0010] Based on the input image, the segmentation model in the pre-trained multi-decision aggregation network is used to obtain segmentation data; the segmentation data includes the segmentation results of each decision sub-network;

[0011] Perform an indicator evaluation on all segmentation results based on the preset indicators to obtain the reward score for each segmentation result;

[0012] Based on the reward score, a preference data pair is constructed according to the segmentation result; the preference data pair includes a positive sample and a negative sample, and the reward score of the positive sample is greater than the reward score of the negative sample;

[0013] Constructing a second loss function based on the preference data pair, and then performing a second training on the segmentation model;

[0014] The second trained multi-decision aggregation network is used to perform image segmentation on the image to be processed to obtain an image segmentation result.

[0015] In some embodiments, inputting the input image into a preset multi-decision aggregation network to perform image segmentation prediction to obtain a segmentation prediction map includes the following steps:

[0016] Visually encode the input image through the visual encoder and extract the visual embedding;

[0017] Based on visual embedding, the corresponding first segmentation probability map is generated through each decision sub-network; random inactivation operation is used in each decision sub-network;

[0018] A weighted score is assigned to the first segmentation probability map corresponding to each decision subnetwork through a sparse router, and the products of all the first segmentation probability maps and their corresponding weighted scores are accumulated and summed to obtain a second segmentation probability map; the sparse router includes a full connection and an activation function;

[0019] Find the maximum value index in the channel dimension of the second segmentation probability map to obtain the segmentation prediction map.

[0020] In some embodiments, constructing a first loss function based on the weighted score and the prediction result and the annotation information includes the following steps:

[0021] Construct an entropy penalty loss function based on the weighted scores;

[0022] Construct a cross entropy loss function based on the prediction results and annotation information;

[0023] The first loss function is obtained by adding the entropy penalty loss function and the cross entropy loss function.

[0024] In some embodiments, constructing an entropy penalty loss function based on the weighted scores comprises the following steps:

[0025] Solve the logarithmic function for the weighted scores assigned to each decision subnetwork;

[0026] The products of the weighted scores assigned to all decision sub-networks and their corresponding logarithmic function solutions are accumulated and summed to obtain the entropy penalty loss function.

[0027] In some embodiments, performing an indicator evaluation on all segmentation results based on a preset indicator to obtain a reward score for each segmentation result includes the following steps:

[0028] Based on the preset indicators, the segmentation results are evaluated to obtain the function value of the evaluation indicator function corresponding to each preset indicator of the segmentation result;

[0029] The function values ​​of the segmentation results in all evaluation index functions are weighted summed to obtain the reward score corresponding to the segmentation result.

[0030] In some embodiments, constructing a second loss function based on the preference data pair comprises the following steps:

[0031] Construct contrast loss and cross entropy loss functions based on preference data pairs;

[0032] The contrast loss is balanced based on the preset hyperparameters, and then the second loss function is constructed by combining it with the cross entropy loss function.

[0033] In some embodiments, constructing a contrastive loss based on a preference data pair comprises the following steps:

[0034] Get the generation probability of each segmentation result;

[0035] Construct contrast loss based on the generation probabilities of positive and negative samples in all preference data pairs;

[0036] Among them, the expression of contrast loss is:

[0037]

[0038] In the formula, represents contrast loss; Y a represents a positive sample; Y b represents negative samples; π φ (Y a |I) means that given an input image I, the segmentation model generates Y a The probability of generation; π φ (Y b |I) means that given an input image I, the segmentation model generates Y b ; Ω represents the preference sorting set composed of all segmentation results, and the segmentation results in the preference sorting set are arranged in the order of their corresponding reward scores.

[0039] To achieve the above object, another aspect of an embodiment of the present invention provides an image segmentation device, the device comprising:

[0040] The first module is used to obtain an input image; the input image carries the annotation information of the category of each pixel;

[0041] The second module is used to input the input image into a preset multi-decision aggregation network to perform image segmentation prediction and obtain a segmentation prediction map;

[0042] The multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image;

[0043] The third module is used to construct a first loss function based on the weighted scores, the prediction results and the annotation information, and then pre-train the multi-decision aggregation network;

[0044] The fourth module is used to obtain segmentation data based on the input image using the segmentation model in the pre-trained multi-decision aggregation network; the segmentation data includes the segmentation results of each decision sub-network;

[0045] The fifth module is used to evaluate all segmentation results based on preset indicators and obtain the reward score of each segmentation result;

[0046] The sixth module is used to construct a preference data pair based on the reward score according to the segmentation result; the preference data pair includes a positive sample and a negative sample, and the reward score of the positive sample is greater than the reward score of the negative sample;

[0047] A seventh module is used to construct a second loss function based on the preference data pair, and then perform a second training on the segmentation model;

[0048] The eighth module is used to perform image segmentation on the image to be processed by using the multi-decision aggregation network after the second training to obtain an image segmentation result.

[0049] To achieve the above objective, another aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0050] To achieve the above objective, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0051] The embodiments of the present invention include at least the following beneficial effects: the present invention provides an image segmentation method, device, electronic device and storage medium, the scheme acquires an input image; the input image carries the annotation information of the category of each pixel; the input image is input into a preset multi-decision aggregation network for image segmentation prediction to obtain a segmentation prediction map; wherein the multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image; based on the weighted score and the prediction The first loss function is constructed based on the results and annotation information, and then the multi-decision aggregation network is pre-trained; based on the input image, the segmentation data is obtained by using the segmentation model in the pre-trained multi-decision aggregation network; the segmentation data includes the segmentation results of each decision subnetwork; based on the preset indicators, all segmentation results are evaluated to obtain the reward score of each segmentation result; based on the reward score, a preference data pair is constructed according to the segmentation result; the preference data pair includes positive samples and negative samples, and the reward score of the positive sample is greater than the reward score of the negative sample; based on the preference data pair, a second loss function is constructed, and then the segmentation model is trained for the second time; the multi-decision aggregation network after the second training is used to perform image segmentation on the processed image to obtain the image segmentation result. The embodiment of the present invention integrates preference optimization and expert hybrid strategy, and uses a unified method to optimize any evaluation index; the multi-decision aggregation network is used as the backbone network of the segmentation task, which is responsible for generating diversified segmentation results of the input image, and then constructing segmentation data and optimizing the segmentation model accordingly. The embodiment of the present invention can balance multiple indicators to achieve accurate image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of an image segmentation method provided by an embodiment of the present invention;

[0053] Figure 2 is a schematic diagram of a multi-decision preference optimization network structure provided by an embodiment of the present invention;

[0054] Figure 3 It is a schematic diagram of a multi-decision preference optimization process provided by an embodiment of the present invention;

[0055] Figure 4 is a schematic diagram of the architecture of an image segmentation device provided by an embodiment of the present invention;

[0056] Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0058] It is understood that the terms "first", "second", etc. used in the present invention can be used to describe various concepts in the present invention, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0059] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0060] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0061] The image segmentation method provided by the embodiment of the present invention relates to the field of image processing technology. The image segmentation method provided by the embodiment of the present invention can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the image segmentation method, etc., but is not limited to the above forms.

[0062] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0063] Figure 1 is an optional flowchart of the image segmentation method provided by an embodiment of the present invention. Figure 1 The method may include but is not limited to steps S100 to S800.

[0064] S100, obtaining an input image;

[0065] Among them, the input image carries the annotation information of the category of each pixel;

[0066] S200, inputting the input image into a preset multi-decision aggregation network to perform image segmentation prediction to obtain a segmentation prediction map;

[0067] The multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image;

[0068] It should be noted that, in some embodiments, step S200 may include the following steps: visually encode the input image through a visual encoder to extract a visual embedding; based on the visual embedding, generate a corresponding first segmentation probability map through each decision sub-network; use a random inactivation operation in each decision sub-network; assign a weighted score to the first segmentation probability map corresponding to each decision sub-network through a sparse router, and accumulate and sum the products of all first segmentation probability maps and their corresponding weighted scores to obtain a second segmentation probability map; the sparse router includes a full connection and an activation function; and find the maximum value index in the channel dimension of the second segmentation probability map to obtain a segmentation prediction map.

[0069] For example, in some specific embodiments, Figure 2 The process on the left, the processing flow of the multi-decision aggregation network can be implemented as follows:

[0070] Input a medical image I, the multi-decision preference optimization network first passes through the visual encoder Extracting visual embeddings:

[0071]

[0072] in, is the visual embedding of the input image I. Here, S and C represent the number of tokens and latent dimension, respectively. is a visual encoder. Then, the present invention designs multiple decoder decision subnetworks, defined as {ε m (·)|m=1,…,M}, used to generate diverse segmentation results:

[0073]

[0074] in, Y represents the segmentation probability map generated by the mth decision subnetwork for X. m Represents the corresponding segmentation result prediction map. Find the index of the maximum value in the channel dimension, and get the corresponding segmentation prediction map Y m Each decision subnetwork ε m (·) are expected to adaptively learn expertise for medical image segmentation, thus generating diverse segmentation results Y for X m In order to further enhance the diversity of the decision subnetwork, the present invention adds m (·) Dropout operation (random inactivation) is used.

[0075] Since each decision subnetwork ε m (·) can capture diverse information from X and generate different segmentation results. Therefore, the present invention designs a sparse router s(·). The router function s(·) is implemented by two layers of full connection + Softmax (FC-ReLU-FC-ReLU-Softmax) layers. It is used to aggregate ε from different decision sub-networks. m Different segmentation results of (·):

[0076]

[0077] in is the final segmentation probability map, and Y is the final prediction map. The router function s(·) is for each corresponding decision subnetwork ε m (·) Assign score sm (X) (i.e., a weighted score) that indicates how suitable the mth decision subnetwork is for processing X.

[0078] S300, constructing a first loss function based on the weighted scores, the prediction results and the annotation information, and then pre-training the multi-decision aggregation network;

[0079] It should be noted that, in some embodiments, constructing a first loss function based on weighted scores and prediction results and annotation information may include the following steps: constructing an entropy penalty loss function based on the weighted scores; constructing a cross-entropy loss function based on the prediction results and annotation information; and obtaining the first loss function based on the result of adding the entropy penalty loss function and the cross-entropy loss function.

[0080] Exemplarily, in some specific implementations, in order to encourage effective sparse routing, the present invention designs an entropy penalty loss function to minimize the entropy of router scores of different decision subnetworks:

[0081]

[0082] The entropy penalty loss function can make the decision subnetwork ε that best processes X m (·) is highlighted. Therefore, during the training process, different decision subnetworks focus on different images, so that specific and complementary medical knowledge can be learned. The final prediction map Y is obtained through the probability map {ε m (X)|m=1,…,M} are aggregated.

[0083] During the training process, the present invention applies the cross entropy loss function to the final prediction graph Y:

[0084]

[0085] Among them, N and They represent the number of pixels and the number of categories of image I respectively. Represents the true label that the i-th pixel belongs to category c.

[0086] The present invention utilizes multiple parallel decision subnetworks to generate diverse segmentation results, which can be used for the subsequent construction of segmentation data. It is worth noting that the sparse router and dropout operations ensure that for the same input, {Y m |m=1,…,M} has sufficient diversity.

[0087] S400, based on the input image, using the segmentation model in the pre-trained multi-decision aggregation network to obtain segmentation data;

[0088] Among them, the segmentation data includes the segmentation results of each decision sub-network;

[0089] Exemplarily, in some specific implementations, a trained multi-decision aggregation network is used to construct segmentation data.

[0090] S500, performing an indicator evaluation on all segmentation results based on a preset indicator to obtain a reward score for each segmentation result;

[0091] It should be noted that, in some embodiments, performing an indicator evaluation on all segmentation results based on preset indicators to obtain a reward score for each segmentation result may include the following steps: performing an indicator evaluation on the segmentation results based on preset indicators to obtain the function value of the evaluation indicator function of the segmentation result corresponding to each preset indicator; performing a weighted summation on the function values ​​of all evaluation indicator functions of the segmentation result to obtain a reward score corresponding to the segmentation result.

[0092] For example, in some specific implementations, for multiple segmentation results {Y m |m=1,…,M}, the present invention proposes a segmentation optimization module based on index preference to learn the knowledge of evaluation preference. The segmentation optimization module based on index preference is used to calculate the value of each result Y. m The reward score can be used to construct segmentation data for various target evaluation indicators. The segmentation data is then used for preference supervision to guide the optimization of the segmentation model through contrastive learning, encouraging the generation of high-quality results and suppressing poor-quality results.

[0093] The present invention first designs a reward function to evaluate different segmentation results {Y m |m=1,…,M}. This reward function can be used to score each segmentation result based on any evaluation index, and its definition is as follows:

[0094]

[0095] Among them, r m Represents the segmentation result Y of the input image I m Bonus score. It can be any differentiable or non-differentiable evaluation index function, such as Dice coefficient, Hausdorff distance, etc. The calculation results of these evaluation index functions are standardized to the range of 0 to 1. The higher the score, the better the quality of the segmentation result. k Represents the kth evaluation index function The weight of .

[0096] S600, based on the reward score, constructing a preference data pair according to the segmentation result;

[0097] The preferred data pairs include positive samples and negative samples, and the reward score of the positive sample is greater than the reward score of the negative sample.

[0098] For example, in some specific implementations, according to {r m |m=1,…,M}, the present invention divides the segmentation result {Y m |m=1,…,M} to sort and rewrite as where t m represents the order of sorting, and Finally, the present invention constructs a segmentation preference sorting set Ω for the input image I, which contains a series of preference data pairs (for example, one of the preference data pairs is

[0099] Using the constructed preference ranking set Ω, the present invention optimizes the segmentation model of the present invention by contrastive learning. Ω contains multiple preference data pairs, such as in, represents the high-quality segmentation result of a given input image I as a positive sample, and Indicates a relatively poor segmentation result. As a negative sample, the definition of positive and negative samples is relative. For two results, the one with a higher evaluation index is the positive sample, and the poor one is the negative sample.

[0100] S700, constructing a second loss function based on the preference data, and then performing a second training on the segmentation model;

[0101] It should be noted that, in some embodiments, constructing a second loss function based on the preference data pair may include the following steps: constructing a contrast loss and a cross entropy loss function based on the preference data pair; balancing the contrast loss based on preset hyperparameters, and then constructing a second loss function in combination with the cross entropy loss function.

[0102] In some embodiments, constructing the contrast loss based on the preference data pairs may include the following steps: obtaining the generation probability of each segmentation result; constructing the contrast loss based on the generation probabilities corresponding to the positive samples and the negative samples in all the preference data pairs;

[0103] Among them, the expression of contrast loss is:

[0104]

[0105] In the formula, represents contrast loss; Y a represents a positive sample; Y b represents negative samples; π φ (Y a |I) means that given an input image I, the segmentation model generates Y a The probability of generation; π φ (Y b|I) means that given an input image I, the segmentation model generates Y b ; Ω represents the preference sorting set composed of all segmentation results, and the segmentation results in the preference sorting set are arranged in the order of their corresponding reward scores.

[0106] Exemplarily, in some specific implementations, the network is optimized by the following contrast loss:

[0107]

[0108] Here, π φ (Y t |I) means that given an input image I, the segmentation model generates Y t probability. The purpose is to ensure that the network generates The probability of generating The probability of effectively combining positive and negative supervision signals. This formula can be further extended to the entire preference ranking set Ω:

[0109]

[0110] Different from the loss function generally used in medical image segmentation, By ensuring that the probability of generating a better result is higher than the probability of generating a worse result, the segmentation prediction is aligned with the evaluation preference. This loss function is independent of the specific form of the evaluation metric and provides flexibility to adapt to various metrics.

[0111] S800, performing image segmentation on the image to be processed by using the second trained multi-decision aggregation network to obtain an image segmentation result.

[0112] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0113] First of all, it should be noted that in the medical field, image segmentation technology plays an important role in the accurate diagnosis of diseases and subsequent treatment strategies. In order to measure the quality of image segmentation, many evaluation indicators have been generated in the field of medical image segmentation. However, in order to optimize these indicators, existing methods need to design specific proxy loss functions for the indicators. This process is not only cumbersome but also very time-consuming, especially when some indicators are non-differentiable and complex. This problem is particularly serious.

[0114] In view of this, the present invention proposes a multi-decision preference optimization network, such as Figure 3As shown, the network adopts preference optimization and hybrid expert algorithm to design a unified optimization scheme that can optimize any evaluation index. Unlike existing methods, the multi-decision preference optimization network does not design a specific proxy loss function for each evaluation index, but guides the segmentation model through comparative learning to generate results consistent with the evaluation index preference. In addition, the multi-decision preference optimization network uses a single preference optimization for supervised optimization of all evaluation indicators, eliminating the problem of balancing different loss functions during training in existing methods. On this basis, the multi-decision preference optimization network further constructs segmentation data that can reflect the evaluation index preference, designs a multi-decision aggregation network and segmentation optimization based on index preference to achieve effective segmentation preference optimization. Experiments on multiple public benchmark data sets have demonstrated the advancement of the multi-decision preference optimization network in segmentation performance. The contributions of the present invention can be summarized as follows:

[0115] 1. A multi-decision preference optimization network is proposed, which designs a unified framework through preference optimization and hybrid expert algorithm to optimize any evaluation index.

[0116] 2. Two key modules, multi-decision aggregation network and segmentation optimization based on indicator preference, were introduced to establish an effective preference optimization paradigm for medical image segmentation.

[0117] 3. Experiments on multiple public benchmarks have verified the effectiveness of the method of the present invention, and the method of the present invention can achieve the most advanced medical segmentation performance.

[0118] The goal of medical image segmentation is to classify each pixel in the input medical image I into a set of categories In this task, various evaluation indicators (such as pixel accuracy, Dice similarity coefficient, Hausdorff distance, sensitivity, etc.) are used to evaluate the segmentation performance from different perspectives. Existing methods generally design specific differentiable proxy loss functions for different indicators to optimize the segmentation model.

[0119] However, in the field of medical image segmentation, the evaluation indicators are very diverse, and some of them are not differentiable, which makes it very challenging to design a specific proxy loss function for each indicator. In addition, it is also very difficult to optimize multiple indicators at the same time during training, which requires a delicate balance of different loss functions during training.

[0120] In order to overcome these difficulties, the present invention proposes an innovative medical image segmentation framework - a multi-decision preference optimization network, which combines preference optimization and expert hybrid strategy to optimize any evaluation index in a unified way. The network consists of two core parts: a multi-decision aggregation network and a segmentation optimization module based on index preference. As the backbone network of the segmentation task, the multi-decision aggregation network is responsible for generating diversified segmentation results of the input image. The segmentation optimization module based on index preference focuses on constructing segmentation data and optimizing the segmentation model accordingly. The entire workflow of the multi-decision preference optimization network is described in detail in the following sections. Figure 2 There is a detailed demonstration in . The specific implementation can be as follows:

[0121] Input a medical image I, the multi-decision preference optimization network first passes through the visual encoder Extracting visual embeddings:

[0122]

[0123] in, is the visual embedding of the input image I. Here, S and C represent the number of tokens and latent dimension, respectively. is a visual encoder. Then, the present invention designs multiple decoder decision subnetworks, defined as {ε m (·)|n=1,…,M}, used to generate diverse segmentation results:

[0124]

[0125] in, Y represents the segmentation probability map generated by the mth decision subnetwork for X. m Represents the corresponding segmentation result prediction map. Find the index of the maximum value in the channel dimension, and get the corresponding segmentation prediction map Y m Each decision subnetwork ε m (·) are expected to adaptively learn expertise for medical image segmentation, thus generating diverse segmentation results Y for X m In order to further enhance the diversity of the decision subnetwork, the present invention adds m (·) Dropout operation (random inactivation) is used.

[0126] Since each decision subnetwork ε m (·) can capture diverse information from X and generate different segmentation results. Therefore, the present invention designs a sparse router s(·). The router function s(·) is implemented by two layers of full connection + Softmax (FC-ReLU-FC-ReLU-Softmax) layers. It is used to aggregate ε from different decision sub-networks.m Different segmentation results of (·):

[0127]

[0128] in is the final segmentation probability map, and Y is the final prediction map. The router function s(·) is for each corresponding decision subnetwork ε m (·) Assign score s m (X), which indicates the suitability of the mth decision subnetwork for processing X. In order to encourage effective sparse routing, the present invention designs an entropy penalty loss function to minimize the entropy of the router scores of different decision subnetworks:

[0129]

[0130] The entropy penalty loss function can make the decision subnetwork ε that best processes X m (·) is highlighted. Therefore, during the training process, different decision subnetworks focus on different images, so that specific and complementary medical knowledge can be learned. The final prediction map Y is obtained through the probability map {ε m (X)|m=1,…,M} are aggregated.

[0131] During the training process, the present invention applies the cross entropy loss function to the final prediction graph Y:

[0132]

[0133] Among them, N and They represent the number of pixels and the number of categories of image I respectively. Represents the true label that the i-th pixel belongs to category c

[0134] The present invention utilizes multiple parallel decision subnetworks to generate diverse segmentation results, which can be used for the subsequent construction of segmentation data. It is worth noting that the sparse router and dropout operations ensure that for the same input, {Y m |m=1,…,M} has sufficient diversity.

[0135] Next, for multiple segmentation results {Y m |m=1,…,M}, the present invention proposes a segmentation optimization module based on index preference to learn the knowledge of evaluation preference. The segmentation optimization module based on index preference is used to calculate the value of each result Y. m The reward score can be used to construct segmentation data for various target evaluation indicators. The segmentation data is then used for preference supervision to guide the optimization of the segmentation model through contrastive learning, encouraging the generation of high-quality results and suppressing poor-quality results.

[0136] The present invention first designs a reward function to evaluate different segmentation results {Y m |m=1,…,M}. This reward function can be used to score each segmentation result based on any evaluation index, and its definition is as follows:

[0137]

[0138] Among them, r m Represents the segmentation result Y of the input image I m Bonus score. It can be any differentiable or non-differentiable evaluation index function, such as Dice coefficient, Hausdorff distance, etc. The calculation results of these evaluation index functions are standardized to the range of 0 to 1. The higher the score, the better the quality of the segmentation result. k Represents the kth evaluation index function The weight of .

[0139] Then, according to {r m |m=1,…,M}, the present invention divides the segmentation result {Y m |m=1,…,M} to sort and rewrite as where t m represents the order of sorting, and Finally, the present invention constructs a segmentation preference sorting set Ω for the input image I, which contains a series of preference data pairs (for example, one of the preference data pairs is

[0140] Using the constructed preference ranking set Ω, the present invention optimizes the segmentation model of the present invention by contrastive learning. Ω contains multiple preference data pairs, such as in, represents the high-quality segmentation result of a given input image I as a positive sample, and Relatively poor segmentation results are used as negative samples. The definition of positive and negative samples is relative. For two results, the one with a higher evaluation index is the positive sample, and the one with a poor evaluation index is the negative sample. The present invention optimizes the network through the following contrast loss:

[0141]

[0142] Here, π φ (Y t |I) means that given an input image I, the segmentation model generates Y t probability. The purpose is to ensure that the network generates Y t1 The probability of generating Y t2The probability of effectively combining positive and negative supervision signals. This formula can be further extended to the entire preference ranking set Ω:

[0143]

[0144] Different from the loss function generally used in medical image segmentation, By ensuring that the probability of generating a better result is higher than the probability of generating a worse result, the segmentation prediction is aligned with the evaluation preference. This loss function is independent of the specific form of the evaluation metric and provides flexibility to adapt to various metrics.

[0145] Finally, the overall framework of the present invention follows a two-stage training process, including a pre-training stage and a preference alignment stage. In the pre-training stage, the present invention uses the cross entropy loss function of Formula 5 and the entropy penalty loss function of Formula 4 added together as the total loss function to train the multi-decision aggregation network. This training method enables the multi-decision aggregation network to produce diverse segmentation results, laying the foundation for the subsequent construction of segmentation data.

[0146] In the preference alignment stage, the present invention uses the trained multi-decision aggregation network to construct segmentation data, and combines the contrast loss in formula (8) with the cross entropy loss function in formula (5) to learn the evaluation preference knowledge and optimize the network:

[0147]

[0148] Here, γ is a hyperparameter used to balance different loss terms. The purpose is to stabilize the training process and ensure that the network of the present invention does not significantly deviate from its pre-trained state during the preference alignment phase.

[0149] After a total of 200 epochs of training, the network of the present invention has achieved the best results in three widely used medical image segmentation benchmark datasets. The present invention uses the Dice similarity coefficient (DSC), Hausdorff distance (HD), sensitivity (SE), specificity (SP) and accuracy (ACC) indicators. Specifically, the method of the present invention is verified on the following three medical image segmentation benchmark datasets: Synapse multi-organ CT dataset, ISIC 2017 and ISIC 2018. The Synapse multi-organ CT dataset consists of 30 abdominal CT scan cases, a total of 3,779 slices, covering eight abdominal organs such as aorta, gallbladder, spleen, left kidney, right kidney, liver, pancreas and stomach. The dataset is divided into a training set (18 cases, 2,212 axial slices) and a test set (12 cases, 1,567 axial slices). The ISIC 2017 and ISIC 2018 datasets focus on skin lesion segmentation and provide annotations for cancerous and non-cancerous lesions. The ISIC 2017 dataset contains 2,000 training dermoscopic images, 150 validation images, and 600 test images. Based on the settings of existing studies, the ISIC 2018 dataset is divided into 1,815 training images, 259 validation images, and 520 test images.

[0150] The performance on all datasets surpasses the existing methods. In particular, on the Synapse multi-organ CT dataset, the method of the present invention outperforms other methods in all five evaluation indicators, with improvements of 0.31%, 0.01%, 0.97%, 0.01% and 1.0% respectively. On the ISIC 2017 dataset. The method of the present invention is ahead of the second-place method in DSC, HD, SE and SP indicators, with improvements of 0.82%, 0.2172%, 0.09% and 0.04% respectively, and also shows the best performance comparable to the best existing method in the ACC indicator. On the ISIC 2018 dataset, the method of the present invention achieved the highest performance in DSC, HD, SP and ACC indicators, with improvements of 0.59%, 0.1552, 0.69% and 0.02% respectively, and also showed performance comparable to the best method in the SE indicator.

[0151] In summary, the present invention proposes an innovative medical image segmentation framework - a multi-decision preference optimization network, which can optimize various evaluation indicators in a unified framework without the need to design a specific proxy loss function for each evaluation indicator. It is simpler and more practical. Specifically, the multi-decision preference optimization network designs multiple parallel decision sub-networks to generate multiple segmentation results, and scores and sorts these segmentation results through a reward function based on the evaluation indicator. This sorting result can reflect the evaluation indicator's preference for the result. Based on this evaluation preference, the multi-decision preference optimization network designs a segmentation preference optimization mechanism, which guides the segmentation model through comparative learning, so that the segmentation model can ultimately generate results consistent with the evaluation indicator. The present invention has conducted experiments on multiple challenging public datasets and proved that the multi-decision preference optimization network can achieve superior segmentation performance compared with existing methods.

[0152] like Figure 4 As shown, the embodiment of the present invention further provides an image segmentation device 900, which may include:

[0153] The first module 901 is used to obtain an input image; the input image carries the annotation information of the category of each pixel;

[0154] The second module 902 is used to input the input image into a preset multi-decision aggregation network to perform image segmentation prediction and obtain a segmentation prediction map;

[0155] The multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image;

[0156] The third module 903 is used to construct a first loss function based on the weighted scores, the prediction results and the annotation information, and then pre-train the multi-decision aggregation network;

[0157] The fourth module 904 is used to obtain segmentation data based on the input image using the segmentation model in the pre-trained multi-decision aggregation network; the segmentation data includes the segmentation results of each decision sub-network;

[0158] The fifth module 905 is used to perform an indicator evaluation on all segmentation results based on a preset indicator to obtain a reward score for each segmentation result;

[0159] The sixth module 906 is used to construct a preference data pair based on the reward score according to the segmentation result; the preference data pair includes a positive sample and a negative sample, and the reward score of the positive sample is greater than the reward score of the negative sample;

[0160] The seventh module 907 is used to construct a second loss function based on the preference data, and then perform a second training on the segmentation model;

[0161] The eighth module 908 is used to perform image segmentation on the image to be processed by using the second trained multi-decision aggregation network to obtain an image segmentation result.

[0162] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0163] The embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned image segmentation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0164] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0165] See also Figure 5 , Figure 5 The hardware structure of an electronic device 1000 of another embodiment is illustrated. The electronic device 1000 includes:

[0166] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0167] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the image segmentation method of the embodiment of the present invention;

[0168] Input / output interface 1003, used to implement information input and output;

[0169] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0170] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0171] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0172] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image segmentation method is implemented.

[0173] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0174] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The image segmentation method, image segmentation device, electronic device and storage medium provided by the embodiments of the present invention are as follows: an input image is acquired; the input image carries the annotation information of the category of each pixel; the input image is input into a preset multi-decision aggregation network for image segmentation prediction to obtain a segmentation prediction map; wherein the multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image; based on the weighted score and the prediction result and the annotation information Construct a first loss function, and then pre-train the multi-decision aggregation network; based on the input image, use the segmentation model in the pre-trained multi-decision aggregation network to obtain segmentation data; the segmentation data includes the segmentation results of each decision subnetwork; perform an indicator evaluation on all segmentation results based on preset indicators to obtain a reward score for each segmentation result; based on the reward score, construct a preference data pair according to the segmentation result; the preference data pair includes positive samples and negative samples, and the reward score of the positive sample is greater than the reward score of the negative sample; construct a second loss function based on the preference data pair, and then perform a second training on the segmentation model; use the multi-decision aggregation network after the second training to perform image segmentation on the processed image to obtain the image segmentation result. The embodiment of the present invention integrates preference optimization and expert hybrid strategy, and uses a unified method to optimize any evaluation index; the multi-decision aggregation network is used as the backbone network of the segmentation task, which is responsible for generating diversified segmentation results of the input image, and then constructs segmentation data and optimizes the segmentation model accordingly. The embodiment of the present invention can balance multiple indicators to achieve accurate image segmentation.

[0176] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0177] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0178] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] Those skilled in the art will appreciate that all or some of the steps, devices, and functional modules / units in the above disclosed methods may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0180] The terms "first", "second", "third", "fourth", etc. (if any) in the description of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0181] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0182] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0186] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. An image segmentation method, characterized in that: The method comprises the following steps: Acquire an input image; the input image carries the annotation information of the category of each pixel; Inputting the input image into a preset multi-decision aggregation network to perform image segmentation prediction to obtain a segmentation prediction map; The multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image; Constructing a first loss function based on the weighted score, the prediction result and the annotation information, and then pre-training the multi-decision aggregation network; Based on the input image, segmentation data is obtained using the segmentation model in the pre-trained multi-decision aggregation network; the segmentation data includes the segmentation result of each decision sub-network; Performing an indicator evaluation on all the segmentation results based on a preset indicator to obtain a reward score for each segmentation result; Based on the reward score, construct a preference data pair according to the segmentation result; the preference data pair includes a positive sample and a negative sample, and the reward score of the positive sample is greater than the reward score of the negative sample; Constructing a second loss function based on the preference data pair, and then performing a second training on the segmentation model; The multi-decision aggregation network after the second training is used to perform image segmentation on the image to be processed to obtain an image segmentation result.

2. The image segmentation method according to claim 1, characterized in that: The step of inputting the input image into a preset multi-decision aggregation network for image segmentation prediction to obtain a segmentation prediction map comprises the following steps: Visually encode the input image by using the visual encoder to extract a visual embedding; Based on the visual embedding, a corresponding first segmentation probability map is generated by each of the decision sub-networks; a random deactivation operation is used in each of the decision sub-networks; Assigning a weighted score to the first segmentation probability map corresponding to each decision subnetwork through the sparse router, and accumulating and summing the products of all the first segmentation probability maps and the corresponding weighted scores to obtain a second segmentation probability map; the sparse router includes a full connection and an activation function; The maximum value index is found in the channel dimension of the second segmentation probability map to obtain the segmentation prediction map.

3. The image segmentation method according to claim 1 or 2, characterized in that: The constructing a first loss function based on the weighted score, the prediction result and the annotation information comprises the following steps: constructing an entropy penalty loss function based on the weighted scores; Constructing a cross entropy loss function based on the prediction result and the annotation information; The first loss function is obtained according to the result of adding the entropy penalty loss function and the cross entropy loss function.

4. The image segmentation method according to claim 3, characterized in that: The step of constructing an entropy penalty loss function based on the weighted score comprises the following steps: Solving the logarithmic function for the weighted scores assigned to each of the decision subnetworks; The products of the weighted scores obtained by all the decision sub-networks and the corresponding logarithmic function solution values ​​are cumulatively summed to obtain the entropy penalty loss function.

5. The image segmentation method according to claim 1, characterized in that: The step of performing an indicator evaluation on all the segmentation results based on a preset indicator to obtain a reward score for each segmentation result comprises the following steps: Performing an index evaluation on the segmentation result based on preset indexes to obtain a function value of an evaluation index function corresponding to each preset index of the segmentation result; The function values ​​of the segmentation result in all the evaluation index functions are weightedly summed to obtain the reward score corresponding to the segmentation result.

6. The image segmentation method according to claim 1, characterized in that: The step of constructing a second loss function based on the preference data pair comprises the following steps: Constructing contrast loss and cross entropy loss functions based on the preference data pairs; The contrast loss is balanced based on preset hyperparameters, and then the second loss function is constructed in combination with the cross entropy loss function.

7. The image segmentation method according to claim 6, characterized in that: The step of constructing the contrast loss based on the preference data comprises the following steps: Obtaining a generation probability of each of the segmentation results; Constructing the contrast loss based on the generation probabilities corresponding to the positive samples and the negative samples in all the preference data pairs; Wherein, the expression of the contrast loss is: In the formula, represents contrast loss; Y a represents a positive sample; Y b represents negative samples; π φ (Y a |I) means that given an input image I, the segmentation model generates Y a The probability of generation; π φ (Y b |I) means that given an input image I, the segmentation model generates Y b ; Ω represents the preference sorting set composed of all segmentation results, and the segmentation results in the preference sorting set are arranged in the order of their corresponding reward scores.

8. An image segmentation device, characterized in that: The device comprises: The first module is used to obtain an input image; the input image carries the annotation information of the category of each pixel; The second module is used to input the input image into a preset multi-decision aggregation network to perform image segmentation prediction to obtain a segmentation prediction map; The multi-decision aggregation network includes a segmentation model and a sparse router; the segmentation model includes a visual encoder and a preset number of decision sub-networks connected in parallel after the visual encoder; the decision sub-network uses a decoder; the sparse router is used to assign a weighted score to the data result of each decision sub-network; the segmentation prediction map includes the prediction result of the category of each pixel of the input image; A third module is used to construct a first loss function based on the weighted score, the prediction result and the annotation information, so as to pre-train the multi-decision aggregation network; A fourth module is used to obtain segmentation data based on the input image using the segmentation model in the pre-trained multi-decision aggregation network; the segmentation data includes the segmentation result of each decision subnetwork; A fifth module is used to perform an indicator evaluation on all the segmentation results based on a preset indicator to obtain a reward score for each segmentation result; A sixth module is used to construct a preference data pair based on the reward score and the segmentation result; the preference data pair includes a positive sample and a negative sample, and the reward score of the positive sample is greater than the reward score of the negative sample; A seventh module is used to construct a second loss function based on the preference data, and then perform a second training on the segmentation model; The eighth module is used to perform image segmentation on the image to be processed by using the multi-decision aggregation network after the second training to obtain an image segmentation result.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images