Model generation, image segmentation method, system, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a model generation, image segmentation method, system, electronic device and storage medium. The model generation method may include: obtaining a sample image, an enhanced image and a segmentation label of the sample image, and using the sample image and the segmentation label, and the enhanced image and the segmentation label as a group of training samples respectively; training the original segmentation model based on multiple groups of training samples to obtain an image segmentation model; wherein the enhanced image includes a first enhanced image generated according to the sample image and the image template, and / or a second enhanced image generated according to the target feature map of the sample image and the target feature map template, the image template includes pixel points with a first value, the target feature map template includes pixel points with a second value, and the first value and the second value are less than the preset value. The technical solution of the embodiment of the present invention is to obtain an image segmentation model with strong robustness based on sample image training under a finite data distribution.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a model generation, image segmentation method, system, electronic device and storage medium. Background Art

[0002] Accurately segmenting anatomical structures from medical images can serve as an important reference for medical professionals in diagnosis, treatment planning, and clinical research. In recent years, deep convolutional neural networks have been widely used to automate segmentation tasks. However, a major difficulty in deploying deep learning-based segmentation methods in real-world applications is the difference in data distribution during clinical deployment (also known as domain shift), which includes variations in image appearance and contrast across medical centers and scanners, as well as various imaging artifacts.

[0003] Recent work on domain generalization offers a promising approach to addressing these challenges. However, much of this work requires collecting data from multiple data distributions (e.g., data from different scanners, data from different medical centers, etc.) to help the network learn domain-invariant features for segmentation. However, due to data privacy concerns and collection costs, collecting data from multiple data distributions is not always feasible. Therefore, developing robust image segmentation models trained from data from limited data distributions is of great practical value for medical imaging research. Summary of the Invention

[0004] Embodiments of the present invention provide a model generation, image segmentation method, system, electronic device and storage medium to achieve the effect of training a highly robust image segmentation model based on data under a finite data distribution (i.e., training samples).

[0005] In a first aspect, an embodiment of the present invention provides a model generation method, which may include:

[0006] Obtain a sample image, an enhanced image, and a segmentation label of the sample image, and use the sample image and the segmentation label, and the enhanced image and the segmentation label as a set of training samples respectively;

[0007] The original segmentation model is trained based on multiple sets of training samples to obtain an image segmentation model;

[0008] In which, the enhanced image includes a first enhanced image generated based on the sample image and the image template, and / or a second enhanced image generated based on the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of the first numerical value, and the target feature map template includes pixel points with pixel values ​​of the second numerical value, and both the first numerical value and the second numerical value are smaller than the preset numerical value.

[0009] In a second aspect, an embodiment of the present invention further provides an image segmentation method, which may include:

[0010] Acquire a target image and an image segmentation model generated according to the model generation method provided by any embodiment of the present invention;

[0011] The target image is input into the image segmentation model, and the object to be segmented in the target image is obtained according to the output result of the image segmentation model.

[0012] In a third aspect, an embodiment of the present invention further provides a model generation system, which may include:

[0013] A training sample acquisition module is used to obtain a sample image, an enhanced image, and a segmentation label of the sample image, and use the sample image and the segmentation label, and the enhanced image and the segmentation label as a set of training samples respectively;

[0014] A model generation module is used to train the original segmentation model based on multiple sets of training samples to obtain an image segmentation model;

[0015] In which, the enhanced image includes a first enhanced image generated based on the sample image and the image template, and / or a second enhanced image generated based on the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of the first numerical value, and the target feature map template includes pixel points with pixel values ​​of the second numerical value, and both the first numerical value and the second numerical value are smaller than the preset numerical value.

[0016] In a fourth aspect, an embodiment of the present invention further provides an image segmentation system, which may include:

[0017] An image segmentation model acquisition module, configured to acquire a target image and an image segmentation model generated by the model generation method provided by any embodiment of the present invention;

[0018] The image segmentation module is used to input the target image into the image segmentation model and obtain the target to be segmented in the target image based on the output result of the image segmentation model.

[0019] In a fifth aspect, an embodiment of the present invention further provides an electronic device, which may include:

[0020] one or more processors;

[0021] a memory for storing one or more programs;

[0022] When one or more programs are executed by one or more processors, the one or more processors implement the model generation method or image segmentation method provided by any embodiment of the present invention.

[0023] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the model generation method or image segmentation method provided by any embodiment of the present invention can be implemented.

[0024] The technical solution of the embodiment of the present invention obtains a sample image, an enhanced image, and a segmentation label of the sample image, and uses the sample image and the segmentation label, and the enhanced image and the segmentation label as a group of training samples respectively, wherein the enhanced image includes a first enhanced image generated according to the sample image and the image template, and / or a second enhanced image generated according to the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of a first numerical value, and the target feature map template includes pixel points with pixel values ​​of a second numerical value, and the first numerical value and the second numerical value are both less than a preset numerical value, which means that the first enhanced image generated after the image template is applied to the sample image can be understood as a sample image with some image details missing, and the missing feature map generated after the target feature map template is applied to the target feature map can be understood as a target feature map with some feature details missing, and it is precisely because of this missing that the data distribution of the sample image and the first enhanced image, as well as the data distribution of the second enhanced image and the sample image generated according to the missing feature map may be different; further, the original segmentation model is trained based on multiple groups of training samples. Since the multiple groups of training samples are samples under multiple data distributions, the image segmentation model obtained by training has strong robustness. The above technical solution generates an enhanced image with a data distribution different from that of the sample image by performing a deletion operation on the sample image or the target feature map of the sample image. Then, the multiple groups of training samples including the sample image and the enhanced image are training samples under multiple data distributions, thereby achieving the effect of training an image segmentation model with strong robustness while only needing to obtain sample images under a limited data distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1a is a schematic diagram of a first network training method in an embodiment of the present invention;

[0026] Figure 1b is a schematic diagram of a second network training method in an embodiment of the present invention;

[0027] Figure 1c is a schematic diagram of a third network training method in an embodiment of the present invention;

[0028] Figure 2 is a flow chart of a model generation method in an embodiment of the present invention;

[0029] Figure 3a is a first schematic diagram of generating a first enhanced image in an embodiment of the present invention;

[0030] Figure 3b is a second schematic diagram of generating a first enhanced image in an embodiment of the present invention;

[0031] Figure 4 is a flow chart of another model generation method in an embodiment of the present invention;

[0032] Figure 5 is a schematic diagram of generating a second enhanced image in an embodiment of the present invention;

[0033] Figure 6a is a schematic diagram of generating a random feature map template in an embodiment of the present invention;

[0034] Figure 6b This is a schematic diagram of generating a channel-type feature map template in an embodiment of the present invention;

[0035] Figure 6c This is a schematic diagram of generating a spatial feature map template in an embodiment of the present invention;

[0036] Figure 7 is a flow chart of another model generation method in an embodiment of the present invention;

[0037] Figure 8 Schematic diagram of series-parallel collaborative training in an embodiment of the present invention;

[0038] Figure 9 is a flow chart of an image segmentation method in an embodiment of the present invention;

[0039] Figure 10 is a structural block diagram of a model generation system in an embodiment of the present invention;

[0040] Figure 11 is a structural block diagram of an image segmentation system in an embodiment of the present invention;

[0041] Figure 12 It is a structural diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0043] Before introducing the embodiments of the present invention, the application scenarios of the embodiments of the present invention are first exemplified: the image segmentation task can be completed by a single image segmentation network, or it can be completed by at least two image segmentation networks together. Among them, being completed by a single image segmentation network means using only one image segmentation network to directly learn the final image segmentation task; being completed by at least two image segmentation networks together means, as the name implies, splitting the final image segmentation task into at least two image segmentation subtasks, and each image segmentation subtask is completed by an image segmentation network. Therefore, the final image segmentation task is completed by at least two image segmentation networks together, where the front image segmentation network performs coarse segmentation, and the back image segmentation network performs segmentation refinement based on the previous coarse segmentation results.

[0044] In order to obtain the above-mentioned image segmentation network (hereinafter referred to as the network), the network training methods mainly include the following:

[0045] (1) Single Network, One-stage Training (SNOT)

[0046] For example, see Figure 1a , only one network S is used to directly learn the final image segmentation task, and the selected loss function L is the final image segmentation loss function. In each training, (X, Y) is randomly selected from the training sample set, where X = [x1, x2, ..., x n ] is a sample image set with a capacity of n (batch size, i.e., the number of sample images used for one iterative training) in the training sample set, Y = [y1, y2, ..., y n ] is each sample image x in the sample image set i The corresponding segmentation labels are composed of a segmentation label set. Select the data enhancement method T (T = [t1, t2, ..., t n ], t i (a certain data transformation method) transforms (X,Y) to obtain (X',Y'), where X' is the set of sample images after X is transformed based on T, and Y' is the set of segmentation labels for S learning after Y uses the same transformation as X. X' is input into S for prediction to obtain P, and the difference between P and Y' is calculated based on the loss function L. Then, the backpropagation is completed based on the backpropagation gradient calculated by L to update the network parameters in S. The training is iteratively performed in this way, and finally the training ends when L meets a certain preset condition, and the final image segmentation model is obtained. Because the above training method only uses one network to complete the final image segmentation task, when the image segmentation task is more complex, the segmentation performance of the resulting image segmentation model sometimes cannot meet the actual needs, and post-processing operations are required to meet the actual needs.

[0047] (2) Multiple Networks, One-stage Training (MNOT)

[0048] MNOT can be used to update the network parameters of multiple networks simultaneously in each training stage. For example, see Figure 1b , take the example of using two networks to learn the final image segmentation task, select networks S1 and S2, and select L1 as the loss function when training S1, and L2 as the loss function when training S2, then the final loss function L is the weighted sum of L1 and L2. In each iterative training, randomly select (X, Y) from the training sample set, where X = [x1, x2, ..., x n ] is a sample image set with a capacity of n in the training sample set, Y=[y1,y2,...,y n ] is each sample image x in this sample image set i The corresponding segmentation labels are composed of a segmentation label set. Select the data enhancement method T (where T = [t1, t2, ..., t n ], t i Transform (X, Y) to obtain (X1, Y1, Y2), where X1 is the image set of X after the transformation based on T, Y1 is the segmentation label set of Y after the same transformation as X for learning in S1, and Y2 is the segmentation label set of Y after the same transformation as X for learning in S2. X1 is input into S1 for prediction to obtain P1. The difference between P1 and Y1 is calculated based on L1. P1 and X1 are combined into a combination C (this combination C can be, but is not limited to, pixel-wise multiplication of P1 and X1 or channel-wise merging of P1 and X1) to obtain X2. X2 is input into S2 for prediction to obtain P. The difference between P and Y2 is calculated based on L2. The calculated L1 and L2 are weighted to obtain L. The network parameters of S1 and S2 are then updated by backpropagating the gradients based on the calculated L. Training is repeated in this way until L meets a certain preset condition, resulting in the two desired image segmentation models. Because the above training method requires training multiple networks simultaneously in a single training phase, it places high demands on the hardware required for training. Even if training is possible with less powerful hardware, it may still cause the trained network to underfit.

[0049] (3) Multiple Networks, Multi-stage Training (MNMT)

[0050] MNMT can be a multi-network separate network parameter update, that is, several networks are trained in several stages. For example, Figure 1c As shown in the figure, we use two networks to learn the final image segmentation task. We select networks S1 and S2, and choose L1 as the loss function for S1 training and L2 as the loss function for S2 training. The entire training process is divided into two stages: the first stage trains only S1, and the second stage trains only S2.

[0051] In the first stage of training, each iteration of training randomly selects (X, Y) from the training sample set, where X = [x1, x2, ..., x n ] is a sample image set with a capacity of n (batch size set for S1) in the training sample set, Y = [y1,y2,...,y n ] is each sample image x in this sample image set i The corresponding segmentation labels are composed of a segmentation label set. Select the data enhancement method T (T = [t1, t2, ..., t n ], t i Transform (X, Y) to obtain (X1, Y1), where X1 is the set of sample images after X is transformed based on T, and Y1 is the set of segmentation labels for S1 learning after Y is transformed using the same transformation as X. X1 is input into S1 for prediction to obtain P1. The difference between P1 and Y1 is calculated using the loss function L1. Backpropagation is then performed based on the backpropagated gradient calculated by L1 to update the network parameters of S1. Training is iteratively repeated until L1 meets certain preset conditions, resulting in the image segmentation model required for the image segmentation subtask. The trained image segmentation model is also referred to as S1.

[0052] In the second stage of training, the training samples in the training sample set are first trimmed, and then the trimmed training sample set is used to train S2, where the trimming is based on the prediction results of model S1 obtained in the first stage. In each iterative training, (XX, YY) is randomly selected from the trimmed training sample set, where XX = [xx1, xx2, ..., xx m ] is a sample image set with a capacity of m (batch size set for S2) in the trimmed training sample set, YY=[yy1,yy2,...,yy m ] is each sample image xx in this sample image set i The corresponding segmentation label set is composed of the trimmed segmentation labels. Select the data enhancement method T'(T'=[t'1,t'2,...,t' m ], t' i(a certain data transformation method) transforms (XX, YY) to obtain (XX2, YY2), where XX2 is the set of sample images after XX is transformed based on T', and YY2 is the set of segmentation labels for S2 learning after YY uses the same transformation as XX. XX2 is input into S2 for prediction to obtain PP2, and the difference between PP2 and YY2 is calculated based on L2. Then, the network parameters of S2 are updated through backpropagation based on the backpropagation gradient calculated by L2. Iterative training is performed in this way, and finally, when L2 meets a certain preset condition, the training ends and the final image segmentation model is obtained. The image segmentation model obtained here is also recorded as S2. Since each network is trained separately in the above training method, the training time required is relatively long.

[0053] In summary, all of the above network training methods can be considered as "serial" training forms.

[0054] In the aforementioned network training methods, T can be selected from any data augmentation method, primarily including supervised and unsupervised data augmentation. Supervised data augmentation methods include, but are not limited to, single-sample-based data augmentation methods, such as geometric transformations (e.g., identity transformation, flipping, rotation, cropping, deformation, scaling), color transformations (e.g., noise addition, blurring, shading, erasing, padding), and multi-sample-based data augmentation methods, such as SMOTE, samplepairing, and mixup. Unsupervised data augmentation includes, but is not limited to, autoaugmentation. It should be noted that the aforementioned data augmentation methods are all based on the sample images themselves, meaning they directly perform various transformations on the data. If the data is abbreviated as x, the augmented data is denoted as x' = T(x). These data augmentation methods have limited ability to generate unseen data, meaning that the data distribution of x' is relatively similar to that of x. Consequently, image segmentation models trained on x' are still not very robust to unseen data distributions.

[0055] Figure 2 This is a flow chart of a model generation method provided in an embodiment of the present invention. This embodiment is applicable to generating an image segmentation model, and is particularly applicable to training a robust image segmentation model based on data from a limited data distribution. This method can be executed by the model generation system provided in an embodiment of the present invention. This system can be implemented in software and / or hardware, and can be integrated into an electronic device, which can be various user terminals or servers.

[0056] See also Figure 2 The method of the embodiment of the present invention specifically includes the following steps:

[0057] S110. Obtain a sample image, an enhanced image, and a segmentation label of the sample image, and use the sample image and the segmentation label, as well as the enhanced image and the segmentation label, as a group of training samples, respectively. The enhanced image includes a first enhanced image generated based on the sample image and the image template, and / or a second enhanced image generated based on the target feature map of the sample image and the target feature map template. The image template includes pixel points with pixel values ​​of a first numerical value, and the target feature map template includes pixel points with pixel values ​​of a second numerical value. Both the first numerical value and the second numerical value are smaller than a preset numerical value.

[0058] The sample images can be images used as samples during model training. In terms of image type, they can be medical images or natural images. It should be noted that natural images captured by different imaging devices (such as mobile phones, tablets, and cameras) may have different data distributions. In terms of image processing, they can be unprocessed raw images or processed images, such as those that have undergone at least one of normalization, standardization, windowing, resampling, resizing, cropping, and image block extraction. In terms of image dimensionality, they can be (2D), 2.5D, or 3D images. The segmentation label can be a segmentation label for the sample image. The sample images and segmentation labels are considered a set of training samples. When multiple sample images are acquired, the data distributions of each sample image can be the same or different, without specific limitation.

[0059] The enhanced image includes a first enhanced image and a second enhanced image. Specifically, the first enhanced image may be an enhanced image generated based on a sample image and an image template. The image template may be a pre-generated template related to the sample image and may include pixels having pixel values ​​of a first value, which is less than a preset value. In practical applications, the preset value may optionally be a positive number less than or equal to 1. This means that the first enhanced image generated by applying the image template to the sample image can be understood as a sample image with some image details missing. Due to this missingness, the data distribution of the sample image and the first enhanced image may differ. The second enhanced image may be an enhanced image generated based on a target feature map template and a target feature map of the sample image. The target feature map may be a feature map obtained by feature extraction of the sample image. The target feature map template may be a pre-generated template related to the target feature map and may include pixels having pixel values ​​of a second value, which is less than a preset value. This means that the missing feature map generated by applying the target feature map template to the target feature map can be understood as a target feature map with some feature details missing. Due to this missingness, the data distribution of the second enhanced image generated based on the missing feature map may differ from that of the sample image. It should be noted that when the first value and / or the second value is 0, the above-mentioned deletion can be understood as a complete deletion; when the first value and / or the second value is greater than 0, the above-mentioned deletion can be understood as a partial deletion (ie, suppressed).

[0060] S120 , training the original segmentation model based on multiple groups of training samples to obtain an image segmentation model.

[0061] Among them, the original segmentation model can be a neural network model to be trained that can be used for image segmentation. The original segmentation model is trained based on multiple groups of training samples, that is, based on multiple groups of training samples consisting of sample images and segmentation labels, and multiple groups of training samples consisting of enhanced images and segmentation labels. Since there are differences in the data distribution of sample images and enhanced images, this means that the above-mentioned model training process is based on training samples with multiple data distributions, thereby effectively ensuring the strong robustness of the trained image segmentation model.

[0062] The technical solution of the embodiment of the present invention obtains a sample image, an enhanced image, and a segmentation label of the sample image, and uses the sample image and the segmentation label, and the enhanced image and the segmentation label as a group of training samples respectively, wherein the enhanced image includes a first enhanced image generated according to the sample image and the image template, and / or a second enhanced image generated according to the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of a first numerical value, and the target feature map template includes pixel points with pixel values ​​of a second numerical value, and the first numerical value and the second numerical value are both less than a preset numerical value, which means that the first enhanced image generated after the image template is applied to the sample image can be understood as a sample image with some image details missing, and the missing feature map generated after the target feature map template is applied to the target feature map can be understood as a target feature map with some feature details missing, and it is precisely because of this missing that the data distribution of the sample image and the first enhanced image, as well as the data distribution of the second enhanced image and the sample image generated according to the missing feature map may be different; further, the original segmentation model is trained based on multiple groups of training samples. Since the multiple groups of training samples are samples under multiple data distributions, the image segmentation model obtained by training has strong robustness. The above technical solution generates an enhanced image with a data distribution different from that of the sample image by performing a deletion operation on the sample image or the target feature map of the sample image. Then, the multiple groups of training samples including the sample image and the enhanced image are training samples under multiple data distributions, thereby achieving the effect of training an image segmentation model with strong robustness while only needing to obtain sample images under a limited data distribution.

[0063] In an optional technical solution, when the image template includes a first template and a second template, the first enhanced image includes a first sub-image and a second sub-image. The above-mentioned model generation method may further include: obtaining an initial template of the same size as the sample image, wherein the pixel value of each pixel in the initial template is a first value; setting the pixel values ​​of pixels within a randomly selected region of the initial template to 1 to obtain the first template, and generating a second template complementary to the first template; generating the first sub-image based on the sample image and the first template, and generating the second sub-image based on the sample image and the second template. The initial template may be a template that eliminates every image detail in the sample image. In practical applications, the first value may optionally be 0 or a positive number very close to 0, thereby ensuring effective elimination of image details. The selected region may be a randomly selected region of the initial template, and the pixel values ​​of pixels within this region are set to 1 to obtain the first template. The first sub-image generated by applying this first template to the sample image can be understood as an image in which the image details corresponding to these regions in the sample image are preserved, while the remaining image details are eliminated. The second template is complementary to the first template, that is, the second template can be a template after changing the first numerical value in the first template to 1 and changing 1 to the first numerical value, and the second template is applied to the sample image to generate a second sub-image. It should be noted that the number of first templates and second templates can be one, two or more, that is, the number of first sub-images and second sub-images generated based on each sample image can be one, two or more, and there is no specific limitation here. The above technical solution, through the first enhanced image generated based on the complementary sub-image, can exert a strong constraint on the model during the model training stage, which can improve the model's ability to learn details in the image, thereby effectively improving the model performance of the image segmentation model on data in seen and unseen domains.

[0064] In order to understand the above technical solution more vividly, the following is an exemplary description of it with reference to specific examples. Figure 3a, the first value is 0. In the template, black represents pixel points with a pixel value of 0, and white represents pixel points with a pixel value of 1. Set the side length to S (0 < S < min(H, W)), where H and W are the height and width of the sample image respectively. Generate an initial template M with the same size as the sample image, and the pixel values of all pixel points in it are 0. Randomly select q selection regions of size S×S in M, and assign the pixel values of the pixel points under these selection regions to 1 to form the first template. Since the first sub-image and the second sub-image are complementary, the second template is generated according to the first template, that is, the regions with pixel value 1 in the first template have pixel value 0 in the second template, and the regions with pixel value 0 in the first template have pixel value 1 in the second template. Multiply these two templates with the sample image pixel by pixel to obtain the first sub-image and the second sub-image. Compared with the sample image, both the first sub-image and the second sub-image have some missing image details.

[0065] Another optional technical solution, when the image template includes a third template and a fourth template, the first enhanced image includes a third sub-image and a fourth sub-image. The above model generation method may further include: obtaining at least two intermediate templates having the same size as the sample image, wherein the pixel values of each pixel point in the intermediate template are all the first value; for each intermediate template, setting the pixel values of the pixel points under the target area with a side length corresponding to the intermediate template in the intermediate template to 1, and updating the intermediate template according to the setting result; taking the union of the intermediate templates as the third template, and generating a fourth template complementary to the third template; generating a third sub-image according to the sample image and the third template, and generating a fourth sub-image according to the sample image and the fourth template. Among them, the intermediate template can be a template that makes each image detail in the sample image missing. In practical applications, optionally, the first value can be 0 or a positive number very close to 0, thereby ensuring the effective missing of image details. Each intermediate template corresponds to its own preset side length. For each intermediate template, the target area can be the area with a side length corresponding to the intermediate template in the intermediate template. Set the pixel values of the pixel points in these areas to 1, and use the set template as the intermediate template. Take the union of the intermediate templates as the third template, and generate a fourth template complementary to the third template. The generation method of the fourth template is similar to that of the second template, which will not be elaborated here. Further, generating the third sub-image and the fourth sub-image, and their generation processes are similar to those of the first sub-image and the second sub-image, which will not be elaborated here either. It should be noted that the number of the third template and the fourth template can be one, two or more, that is, the number of the third sub-image and the fourth sub-image generated based on each sample image can be one, two or more, which is not specifically limited here. The above technical solution can generate a strong binding force on the model during the model training stage through the first enhanced image based on complementary sub-images, and can improve the model's learning ability of details in the image, thereby effectively improving the model performance of the image segmentation model on the data in the seen domain and the unseen domain.

[0066] To understand the above technical solution more vividly, the following will give an exemplary illustration with a specific example. Exemplarily, refer to Figure 3b , the first value is 0. The black in the template represents the pixel points with pixel value 0, and the white represents the pixel points with pixel value 1. Set a group of preset side lengths as S1, S2,...(0 < S i < min(H, W)), where H and W are the height and width of the sample image respectively. Initially generate intermediate templates T1, T2,..., the number is the same as the number of the set preset side lengths. The size of each intermediate template is the same as the sample image, and the pixel values of the pixel points inside are all 0. Perform the following operations on each intermediate template: At the center of the intermediate template Ti, 2S i× 2Si Select a side with length S in the area i The target area is assigned a value of 1. The third template is obtained by taking the union of all intermediate templates. Since the third and fourth sub-images are complementary, the fourth template is generated based on the third template. That is, all areas with a pixel value of 1 in the third template have a pixel value of 0 in the fourth template, and all areas with a pixel value of 0 in the third template have a pixel value of 1 in the fourth template. These two templates are multiplied pixel by pixel with the sample image to obtain the third and fourth sub-images. Compared to the sample image, both the third and fourth sub-images lack some image details.

[0067] Figure 4 It is a flowchart of another model generation method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the original segmentation model includes a first segmentation network, the first segmentation network includes an encoder and a transition decoupler connected in sequence, the transition decoupler includes a convolution structure, and the above-mentioned model generation method may also include: inputting the sample image into the first segmentation network, respectively obtaining the encoding feature map output by the encoder and the decoupling feature map output by the transition decoupler, and using the encoding feature map and the decoupling feature map as candidate feature maps of the sample image; obtaining each candidate feature map template, wherein the candidate feature map template includes a randomly generated random feature map template, and / or a targeted feature map template generated according to the gradient matrix corresponding to the candidate feature map; generating a second enhanced image according to the target feature map determined from each candidate feature map, and the target feature map template determined from each candidate feature map template. The explanation of the terms that are the same as or corresponding to the above-mentioned embodiments will not be repeated here.

[0068] See also Figure 4 The method of this embodiment may specifically include the following steps:

[0069] S210. Obtain an original segmentation model, a sample image, and a segmentation label of the sample image, and use the sample image and the segmentation label as a set of first training samples, wherein the original segmentation model includes a first segmentation network including an encoder and a transition decoupler connected in sequence, and the transition decoupler includes a convolutional structure.

[0070] The original segmentation model includes a first segmentation network to be trained for image segmentation, and the first segmentation network may include an encoder and a transition decoupler connected in sequence, specifically, an encoder, a transition decoupler, and a decoder connected in sequence. The encoder may include any form of encoding structure, the decoder may include any form of decoding structure, and the transition decoupler may include any form of convolution structure for connecting the encoder and the decoder, or a stacked structure of convolution structures, such as a stacked structure of ordinary convolution unit structures, a stacked structure of residual network unit structures (residual blocks), a stacked structure of densely connected structures (dense blocks), etc.

[0071] S220. When the original segmentation model is trained based on multiple groups of first training samples, the sample image is input into the first segmentation network to obtain the encoded feature map output by the encoder and the decoupled feature map output by the transition decoupler, and both the encoded feature map and the decoupled feature map are used as candidate feature maps of the sample image.

[0072] When the original segmentation model is trained based on multiple sets of first training samples, each sample image can be input into the first segmentation network. Specifically, for each sample image, it is input into the encoder to obtain an encoded feature map output by the encoder; the encoded feature map is then input into the transition decoupler to obtain a decoupled feature map output by the transition decoupler; and both the encoded feature map and the decoupled feature map are used as candidate feature maps for the sample image. In practical applications, in addition to using the sample image and the segmentation label as a set of first training samples, the first enhanced image and the segmentation label can also be used as a set of first training samples. Thus, when the original segmentation model is trained based on multiple sets of first training samples, for each sample image, it and its corresponding first enhanced image can be input into the first segmentation network. Thus, the encoded feature map output by the encoder for the sample image and the first enhanced image, respectively, and the decoupled feature map output by the transition decoupler for the sample image and the first enhanced image, respectively, can be used as candidate feature maps for the sample image.

[0073] S230. Acquire each candidate feature map template, wherein the candidate feature map template includes a randomly generated random feature map template and / or a targeted feature map template generated according to a gradient matrix corresponding to the candidate feature map.

[0074] Among them, the candidate feature map template can be a template that has been generated and is related to the candidate feature map, which can include a randomly generated random feature map template, a targeted feature map template generated according to the gradient matrix corresponding to the candidate feature map, etc. The targeted feature map template can be a channel-type feature map template, a spatial-type feature map template, a combined-type feature map template that combines the two, etc., which is not specifically limited here. In actual applications, optionally, when the original segmentation model is iteratively trained based on multiple groups of training samples, the candidate feature map template can include a feature map template generated in the current iterative training process, and a feature map template generated in a historical iterative training process before the current iterative training. The historical iterative training process can be the previous iterative training process, the previous two iterative training processes, the previous three iterative training processes, etc. of the current iterative training, which is not specifically limited here.

[0075] S240. Generate a second enhanced image based on the target feature map determined from each candidate feature map and the target feature map template determined from each candidate feature map template, wherein the target feature map template includes pixel points whose pixel values ​​are second numerical values, and the second numerical value is less than a preset numerical value.

[0076] Among them, the target feature map is determined from each candidate feature map, and the target feature map template is determined from each candidate feature map template. In practical applications, optionally, for the case where the original segmentation model is iteratively trained based on multiple sets of training samples, the probability of obtaining the target feature map template from each candidate feature map template generated in the current iterative training process can be greater than the probability of obtaining the target feature map template from each candidate feature map template generated in the historical iterative training process. Further, a second enhanced image is generated based on the target feature map and the target feature map template, such as generating a missing feature map based on the target feature map and the target feature map template, and then inputting the missing feature map into a data generator, using the sample image corresponding to the target feature map as the image to generate supervision information, to obtain a second enhanced image. This data generator can be any form of decoding structure. It should be noted that, as mentioned above, the missing feature map can be understood as a target feature map with some feature details missing. Therefore, after such a feature map is input into the data generator, the data generator supplements the missing feature details. Whether the data distribution of the supplemented feature details is the same as that of the original feature details is not manually controllable, thereby increasing the possibility of generating a second enhanced image with a data distribution different from that of the sample image.

[0077] S250: Use the second enhanced image and the segmentation label as a set of second training samples, and train the original segmentation model based on multiple sets of first training samples and multiple sets of second training samples to obtain an image segmentation model.

[0078] The technical solution of an embodiment of the present invention generates a second enhanced image through a data enhancement method based on network features. The data enhancement method can be to generate candidate feature map templates through random or targeted operations, and then process the target feature map template based on the target feature map template determined from each candidate feature map template to generate a second enhanced image with a different data distribution from the sample image, thereby enhancing the robustness of the image segmentation model trained thereby.

[0079] In order to understand the above technical solution more vividly, the following is an exemplary description of it with reference to specific examples. Figure 5 , with the sample image x, and the two complementary sub-images x of x c1 and x c2 For example, x, x c1 and x c2 Input into encoder E and transition decoupler TD respectively, and the candidate feature maps z can be obtained respectively. 1 k 、z 1 k,c1 、z 1 k,c2 、z 1 m 、z 1 m,c1 and z 1 m,c2 , where the first three are candidate feature maps output by E, and the last three are candidate feature maps output by TD. Then, based on these candidate feature maps, a candidate feature map library related to x is formed (that is, each candidate feature map in the candidate feature map library is a feature map generated during the current iterative training process), and a candidate feature map is selected from the candidate feature map library as the target feature map Z for generating the second enhanced data x'. Assuming that the size of the candidate feature map is C×H×W, where C is the number of channels of the candidate feature map, and H and W are the height and width of the candidate feature map, respectively, then the size of the candidate feature map template is also C×H×W. The candidate feature map templates stored in the candidate feature map template library can include the candidate feature map template generated during the current iterative training process and the candidate feature map template generated during the historical iterative training process, which means that as the number of iterative training increases, the capacity of the candidate feature map template library will continue to increase. In practical applications, optionally, the initial candidate feature map template library can store an identity transformation matrix template IM, and the pixel value of each pixel in IM can all be 1, that is, Z×IM=Z is satisfied. Select the target feature map template M from the candidate feature map template library, multiply M by Z pixel by pixel, and get the missing feature map Z new , then Z new Input into the data generator G to generate x'. The generation process of x' is to use x as the supervision information to generate the reconstruction loss difference Laug 。

[0080] In summary, for generating N x' from one x, it can be obtained by repeating the following four steps N times: (1) randomly select a Z from the candidate feature map library; (2) randomly select a M from the candidate feature map template library; (3) multiply Z and M pixel by pixel to obtain Z new ; (4) use Z new as the input and x as the supervision information, and generate x' by G. For the case of data augmentation for multiple x, such as X = [x1, x2,..., x n , for each x in X, the above operations can be performed sequentially or in parallel, and no specific limitation is made here.

[0081] An optional technical solution, the above model generation method may further include: obtaining a random list, where the random list includes a first number of 1s and a second number of 0s, and the sum of the first number and the second number is the number of channels of the candidate feature map; generating a random feature map template according to the random list, where the pixel values of the pixel points under the channel map corresponding to 1 in the random list in the random feature map template are 1, and the pixel values of the pixel points under the channel map corresponding to 0 in the random list are 0. Among them, there are the number of channels of the candidate feature map values in the random list list, and these values include a first number of 1s and a second number of 0s. The random feature map template RM includes the number of channels of the candidate feature map channels. The pixel values of the pixel points under the channels corresponding to 1 (i.e., the channel map) in these channels are all 1, and the pixel values of the pixel points under the remaining channels (i.e., the channel map) are all 0. Such an RM can better retain some feature details in the target feature map and miss the remaining feature details, which lays a good foundation for the subsequent generation of the second enhanced image. For a more vivid understanding of the above technical solution, the following is an exemplary description of it with a specific example. Exemplarily, see Figure 6a , RM is determined by a randomly generated list with a length of C (C is the number of channels of the candidate feature map), and the list corresponds to the C channels of RM. The list includes c (0 < c < C) 1s and C - c 0s, then the pixel values of the pixel points under the channel map RM 1×H×W corresponding to the position of 1 in the list in RM are all 1, and the values under the channel map RM 1×H×W corresponding to the position of 0 in the list are all 0.

[0082] Another optional technical solution, for the type feature map template, includes a channel type feature map template, the second value includes a pre-set channel value, the above-mentioned model generation method may also include: obtaining a gradient matrix corresponding to the candidate feature map, and a blank feature map template of the same size; for each channel in the gradient matrix, obtaining the channel mean of the channel according to the feature value of each feature point under the channel; setting the pixel value of the pixel point under the channel map corresponding to the channel whose channel mean is greater than or equal to the first response threshold in the blank feature map template to the channel value, and setting the pixel value of the pixel point under the channel map corresponding to the channel whose channel mean is less than the first response threshold to 1, to obtain a channel type feature map template. Among them, the gradient matrix can be a matrix of size C×H×W obtained by backpropagation according to the set loss function. In practical applications, it is optional to compare it with the candidate feature map z 1 k ,z 1 k,c1 ,z 1 k,c2 The corresponding gradient matrix can be selected from the above reconstruction loss L aug The gradient matrix obtained by back propagation, the gradient matrix obtained by back propagation of the segmentation loss, or the gradient matrix obtained by L aug And the sum of the gradient matrices obtained by back propagation of the segmentation loss and the candidate feature map z 1 m ,z 1 m,c1 ,z 1 m,c2 The corresponding gradient matrix can be selected from the gradient matrix obtained by back propagation of the segmentation part loss. The blank feature map template may include a blank feature map template with the same size as the candidate feature map. For each channel in the gradient matrix, the channel mean of the channel is obtained according to the eigenvalues ​​of each feature point under the channel, that is, the mean of each eigenvalue under the channel is used as the channel mean of the channel. After processing each channel, C channel means are obtained. For channel means greater than or equal to the first response threshold T c The channel of the blank feature map template is set to the channel value a under the channel map of the pixel point, and the pixel value of the remaining pixel points under the channel map is set to 1. The channel-type feature map template thus obtained can better retain some feature details in the target feature map and suppress the remaining feature details. For example, see Figure 6b , calculate the channel mean on the H×W plane of each channel in the gradient matrix, and thus obtain C channel means. Set T c , for greater than or equal to T c The channel mean of the channel type feature map template is set to the channel map TM corresponding to the channel mean 1×H×WThe value is a (0 < a < 1); for channel means less than T c of the channel means, the channel map TM 1×H×W corresponding to the channel means on the channel-based feature map template will be set to 1. The above technical solution completes the selection of channels in the channel-based feature map template according to the channel means of each H×W plane in the gradient matrix.

[0083] Another optional technical solution, the type-based feature map template includes a spatial-based feature map template, the second value includes a preset spatial value, and the above model generation method may further include: for each plane position in the gradient matrix, obtaining the position mean of the plane position according to the feature values of the feature points under the plane position; setting the pixel values of the pixel points under the spatial map corresponding to the plane positions in the blank feature map template where the position mean is greater than or equal to the second response threshold to the spatial value, and setting the pixel values of the pixel points under the spatial map corresponding to the plane positions where the position mean is less than the second response threshold to 1, to obtain the spatial-based feature map template. Among them, the plane position can be a certain position on the two-dimensional plane of the gradient matrix, and the entire gradient matrix has H×W plane positions. Since the gradient matrix has C channels, there are C pixel points at any plane position. For each plane position, the position mean of the plane position is obtained according to the feature values of the feature points on it, that is, the mean of the feature values is used as the position mean. After processing all plane positions, H×W position means are obtained. For the plane positions where the position mean is greater than or equal to the second response threshold T s the pixel values of the pixel points under the spatial map corresponding to the plane positions in the blank feature map template are set to the spatial value b, and the pixel values of the remaining pixel points under the spatial map are set to 1. The spatial-based feature map template obtained in this way can better retain some feature details in the target feature map and suppress the remaining feature details. Exemplarily, see Figure 6c calculate the position mean of the C feature values at each plane position in the gradient matrix, and thus obtain H×W position means. Set T s for position means greater than or equal to T s the spatial map TM c×1×1 corresponding to the position mean on the spatial-based feature map template is set to the value b (0 < b < 1); for position means less than T s the spatial map TM c×1×1 corresponding to the position mean on the spatial-based feature map template is set to 1. The above technical solution completes the selection of spatial positions in the spatial-based feature map template according to the position means of the H×W plane positions in the gradient matrix.

[0084] On this basis, the above-mentioned model generation method may further include: obtaining a combined feature map template based on the channel-type feature map template and the spatial-type feature map template, and using the combined feature map template as the combined feature map template. For example, the result of weighted summation of the values ​​at corresponding positions of the channel-type feature map template and the spatial-type feature map template (i.e., the pixel value of a pixel point at a certain plane position on a certain channel) is used as the combined feature map template.

[0085] Figure 7 It is a flowchart of another model generation method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the original segmentation model includes a first segmentation network and a second segmentation network connected in sequence, the enhanced image includes a first enhanced image, and the original segmentation model is trained based on multiple groups of training samples to obtain an image segmentation model, which may include: inputting a sample image set containing multiple sample images into the first segmentation network to obtain a first prediction label set, and inputting the first enhanced image set containing multiple first enhanced images into the first segmentation network to obtain a second prediction label set, and inputting the first prediction label set into the second segmentation network to obtain a third prediction label set. Among them, the explanations of the terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0086] See also Figure 7 The method of this embodiment may specifically include the following steps:

[0087] S310, obtaining an original segmentation model, a sample image, a segmentation label of the sample image, and a first enhanced image generated based on the sample image and the image template, wherein the original segmentation model includes a first segmentation network and a second segmentation network connected in sequence, and the image template includes pixel points whose pixel values ​​are a first value less than a preset value.

[0088] The second segmentation network can be a network structure to be trained for image segmentation. In practical applications, it can optionally include an "encoder-decoder" network structure. The first segmentation network and the second segmentation network are connected in sequence, that is, the sample image and the first enhanced image input into the original segmentation model can be first input into the first segmentation network, and then the output result of the first segmentation network can be input into the second segmentation network.

[0089] S320. Input a sample image set including multiple sample images into a first segmentation network to obtain a first predicted label set, input a first enhanced image set including multiple first enhanced images into the first segmentation network to obtain a second predicted label set, and input the first predicted label set into a second segmentation network to obtain a third predicted label set.

[0090] Among them, each sample image is input into the first segmentation network respectively to obtain its own first prediction label, and the set consisting of these first prediction labels is called the first prediction label set. Similarly, each first enhanced image is input into the first segmentation network respectively to obtain its own second prediction label, and the set consisting of these second prediction labels is called the second prediction label set. Obviously, multiple sample images and multiple first enhanced images are input into the first segmentation network in parallel. Further, the first prediction label set is input into the second segmentation network to obtain the third prediction label set. Obviously, the sample images are trained in series in the first segmentation network and the second segmentation network.

[0091] S330: Pass the first prediction label set, the second prediction label set, the third prediction label set, and the segmentation label set containing multiple segmentation labels to the loss function to obtain the backpropagation gradient, and adjust the network parameters in the original segmentation model according to the backpropagation gradient to obtain the image segmentation model.

[0092] Among them, in practical applications, optionally, the loss function L can be expressed by the following formula: LL seg,1 (Y p ,Y)+L seg,2 (Y1,Y)+L C1 (Y 1,c1 ,Y 1,c2 ,Y1)+L C2 (Y 1,c1 ,Y 1,c2 ,Y1), where Y1 represents the first predicted label, Y 1,c1 and Y 1,c2 Both represent the second predicted label (because the first enhanced image includes two complementary sub-images), Y p represents the third predicted label, Y represents the segmentation label; L seg,1 With L seg,2 are all segmentation loss functions, such as Dice Loss, IoU Loss, cross entropy loss, or a combination of these segmentation loss functions, L seg,1 (Y p ,Y)+L seg,2 The purpose of setting (Y1, Y) is to make Y1 similar to Y and to make Y P Similar to Y; L C1 (Y 1,c1 ,Y 1,c2 ,Y1)+L C2 (Y 1,c1 ,Y 1,c2 ,Y1) is set to make the predicted label Y of the sub-image of X 1,c1 and Y 1,c2 The forecast for Y1 is binding in detail; L C1(r,s,t)=||(a1*r+a2*s)-t||1, a1 and a2 are hyperparameters greater than 0, L C2 (r,s,t)=||(t-b1*r)-b2*s||1, where b1 and b2 are hyperparameters greater than 0.

[0093] The technical solution of the embodiment of the present invention achieves the effect of training a more robust image segmentation model through the mutual cooperation of parallel training of the first enhanced image and the sample image in the first segmentation network, and series training of the sample image in the first segmentation network and the second segmentation network.

[0094] An optional technical solution, the target feature map is a feature map obtained based on the sample image or the first enhanced image, the second enhanced image includes a sample enhanced image corresponding to the sample image and a corresponding enhanced image corresponding to the first enhanced image, the above-mentioned model generation method may also include: inputting a sample enhanced image set consisting of multiple sample enhanced images into the first segmentation network to obtain a fourth predicted label set, inputting a corresponding enhanced image set consisting of multiple corresponding enhanced images into the first segmentation network to obtain a fifth predicted label set, and inputting the fourth predicted label set into the second segmentation network to obtain a sixth predicted label set; obtaining a sample enhanced image set and a corresponding enhanced image set according to the sample enhanced image set and the corresponding enhanced image set. The fifth enhanced image set generated by the set; accordingly, passing the first predicted label set, the second predicted label set, the third predicted label set, and the segmentation label set including multiple segmentation labels to the loss function to obtain the return gradient can include: passing the first predicted label set, the second predicted label set, the third predicted label set, the fourth predicted label set, the fifth predicted label set, the sixth predicted label set, and the segmentation label set including multiple segmentation labels to the segmentation loss function to obtain the segmentation return gradient, passing the fifth enhanced image set and the sample image set to the image reconstruction loss function to obtain the reconstruction return gradient; and obtaining the return gradient based on the segmentation return gradient and the reconstruction return gradient. The sample enhanced image can include a second enhanced image generated based on the target feature map of the sample image, and the corresponding enhanced image can include a sample enhanced sub-image generated by applying the image template generated by the first enhanced image to the sample enhanced image, that is, the first enhanced image and the second enhanced image exist simultaneously. The generation process of the fourth predicted label set, the fifth predicted label set, and the sixth predicted label set is similar to the generation process of the first predicted label set, the second predicted label set, and the third predicted label set. The generation of the fifth enhanced image is similar to the generation of the second enhanced image and will not be repeated here. The sample image and the first enhanced image are trained in parallel. Relative to the whole formed by the sample image and the first enhanced image, the second enhanced image and the whole also constitute a parallel training method, thereby achieving the effect of auxiliary model training through two parallel branches.

[0095] Another optional technical solution, the above-mentioned model generation method may further include: inputting the segmentation label set into the second segmentation network to obtain a seventh reconstructed label set; correspondingly, passing the first prediction label set, the second prediction label set, the third prediction label set, and the segmentation label set containing multiple segmentation labels to the loss function to obtain a backpropagation gradient, which may include: passing the first prediction label set, the second prediction label set, the third prediction label set, the seventh reconstructed label set, and the segmentation label set containing multiple segmentation labels to the loss function to obtain a backpropagation gradient. That is, through this technical solution, the second segmentation network can learn the appearance of the correct segmentation label through self-reconstruction, so as to perform shape correction on the predicted label output by the first segmentation network based on the second segmentation network to improve the segmentation result. This is also a parallel training method.

[0096] In order to better understand the specific implementation process of the above-mentioned series and parallel training methods, the following is an illustrative example. Figure 8 , the original segmentation model includes the first segmentation network S1 and the second segmentation network S2. In each iterative training, S1 and S2 are performed simultaneously, that is, "series" and "parallel" are performed simultaneously. Specifically, (X, Y) is randomly selected from the training sample set, where X=[x1,x2,...,x n ] is a sample image set with a capacity of n in the training sample set, Y=[y1,y2,...,y n ] is each sample image x in this sample image set i The segmentation label set composed of the corresponding segmentation labels. In the "concatenation" part, X is input into the encoder E of S1 to obtain the candidate feature map z k 1 , that is, z k 1 =E(X), then z k 1 Input into the over-decoupler TD to obtain the candidate feature map z m 1 , that is, z m 1 =TD(z k 1 )=TD(E(X)), then z m 1 Input to decoder D to get the first predicted label Y1 output by S1, that is, Y1=SD(z m 1 )=SD(TD(E(X))); further, directly input Y1 into S2 to obtain Y p , that is, Y p= S2(Y1). Since S2's input is only the output of S1, that is, S2's function is only to correct the output of S1, S2 can be a lightweight network, which does not require high hardware requirements. The "parallel" part mainly consists of the following three aspects of auxiliary training: auxiliary training of S1, auxiliary training of S2, and auxiliary training of S1 and S2.

[0097] For auxiliary training of S1, here it refers to the constraints generated by the two complementary sub-image sets generated based on X when training S1. Taking a sample image corresponding to two complementary sub-images as an example, the two complementary sub-image sets generated based on X are X c1 and X c2 , where X c1 =[x 1,c1 ,x 2,c1 ,...,x n,c1 ] and X c2 =[x 1,c2 ,x 2,c2 ,...,x n,c2 ], that is, x i,c1 and x i,c2 Based on x i Generate two complementary sub-images. c1 Input into S1 to get z k,c1 1 , z m,c1 1 and Y 1,c1 ; Similarly, X c2 Input into S1 to get z k,c2 1 , z m,c2 1 and Y 1,c2 This is for S1 X, X c1 and X c2 Parallel training.

[0098] Auxiliary training for S2 means that in each iterative training of S2, in addition to taking Y1 as the input of S2, the segmentation label set Y is also taken as the input of S2 to obtain the corresponding output Y'. This allows S2 to learn the correct segmentation label through self-reconstruction, thereby helping S2 to better correct the shape of S1's output and improve the segmentation results.

[0099] For auxiliary training of S1 and S2, here it means that in each iterative training, in addition to using X, X c1 and X c2 , and will also use the z k 1 , z m 1 , z k,c11 , z m,c1 1 , z k,c2 1 and z m,c2 1 To generate the second enhanced image. For each sample image x in X i For example, a second enhanced image is generated. For the set X'=[x 1’ ,x 2’ ,...,x n’ ], and the corresponding segmentation label set is still Y. Perform all the above operations on (X', Y) (i.e., serial training, auxiliary training for S1, and auxiliary training for S2), and you can get Y1' output by S1 and Y output by S2. p ', Y obtained based on the two complementary sub-images of X' 1,c1 ' and Y 1,c2 ', and X' obtained after data enhancement.

[0100] At this time, the loss function L can be expressed by the following formula: L = L base +L aux , where L base =L seg,1 (Y p ,Y)+L seg,2 (Y1,Y)+L C1 (Y 1,c1 ,Y 1,c2 ,Y1)+L C2 (Y 1,c1 ,Y 1,c2 ,Y1)+L rec (Y',Y),L aux =L aug (X',X)+L aux’ , where L aux’ =L seg,1 (Y p ',Y)+L seg,2 (Y1',Y)+L C1 (Y 1,c1 ',Y 1,c2 ',Y1')+L C2 (Y 1,c1 ',Y 1,c2 ',Y1')+L aug (X”,X).L aug is the image reconstruction loss mentioned above. L aug With L rec They are all reconstruction loss functions, such as L1, L2 and other reconstruction loss functions, or a combination of these reconstruction loss functions. Figure 8L in seg =L seg,1 +L seg,2 Here L aux It is only based on the loss generated by one X'. When there are more than one X', L aux It consists of multiple sets of losses. Backpropagation is performed based on the backpropagated gradients calculated using L to update the network parameters of S1 and S2. All of the above operations are the steps required for one iterative training. Training is performed iteratively until L reaches a certain preset condition, resulting in the final image segmentation models S1 and S2.

[0101] Figure 9 This is a flowchart of an image segmentation method provided in an embodiment of the present invention. This embodiment is applicable to image segmentation, particularly image segmentation based on a robust image segmentation model. This method can be performed by an image segmentation system provided in an embodiment of the present invention. This system can be implemented in software and / or hardware and integrated into an electronic device, such as a user terminal or server.

[0102] See also Figure 9 The method of the embodiment of the present invention specifically includes the following steps:

[0103] S410: Acquire a target image and an image segmentation model generated by the model generation method provided by any embodiment of the present invention.

[0104] The target image may be an image to be segmented, which contains the target to be segmented; the image segmentation model may be a neural network model for image segmentation that is generated based on the above-mentioned model generation method and has strong robustness.

[0105] S420: Input the target image into the image segmentation model, and obtain the object to be segmented in the target image according to the output result of the image segmentation model.

[0106] Among them, since the image segmentation model has strong robustness, after the target image is input into the image segmentation model, the target to be segmented can be accurately segmented from the target image.

[0107] The technical solution in the embodiment of the present invention can accurately segment the target object from the target image by inputting the target image into an image segmentation model with strong robustness, thereby ensuring the accuracy of image segmentation.

[0108] Figure 10This is a structural block diagram of a model generation system provided by an embodiment of the present invention, which is used to execute the model generation method provided by any of the above embodiments. This system and the model generation method of each of the above embodiments belong to the same inventive concept. For details not fully described in the embodiments of the model generation system, please refer to the embodiments of the above model generation method. Figure 10 The system may specifically include: a training sample acquisition module 510 and a model generation module 520.

[0109] A training sample obtaining module 510 is used to obtain a sample image, an enhanced image, and a segmentation label of the sample image, and use the sample image and the segmentation label, and the enhanced image and the segmentation label as a set of training samples respectively;

[0110] A model generation module 520 is used to train the original segmentation model based on multiple sets of training samples to obtain an image segmentation model;

[0111] In which, the enhanced image includes a first enhanced image generated based on the sample image and the image template, and / or a second enhanced image generated based on the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of the first numerical value, and the target feature map template includes pixel points with pixel values ​​of the second numerical value, and both the first numerical value and the second numerical value are smaller than the preset numerical value.

[0112] Optionally, the original segmentation model includes a first segmentation network, the first segmentation network includes an encoder and a transition decoupler connected in sequence, the transition decoupler includes a convolutional structure, and the above-mentioned model generation system may further include:

[0113] A candidate feature map obtaining module is used to input the sample image into the first segmentation network, obtain the encoded feature map output by the encoder and the decoupled feature map output by the transition decoupler, and use both the encoded feature map and the decoupled feature map as candidate feature maps of the sample image;

[0114] A candidate feature map template acquisition module is used to obtain each candidate feature map template, wherein the candidate feature map template includes a randomly generated random feature map template and / or a targeted feature map template generated according to the gradient matrix corresponding to the candidate feature map;

[0115] The second enhanced image generation module is used to generate a second enhanced image based on the target feature map determined from each candidate feature map and the target feature map template determined from each candidate feature map template.

[0116] On this basis, optionally, the above-mentioned model generation system may further include:

[0117] A random list acquisition module, configured to acquire a random list, wherein the random list includes a first number of ones and a second number of zeros, and the sum of the first number and the second number is the number of channels of the candidate feature map;

[0118] The random feature map template generation module is used to generate a random feature map template according to a random list, wherein the pixel value of the pixel point under the channel map corresponding to 1 in the random list in the random feature map template is 1, and the pixel value of the pixel point under the channel map corresponding to 0 in the random list is 0.

[0119] Optionally, the target feature map template may include at least one of a channel feature map template, a spatial feature map template, and a combined feature map template, and the second value includes a preset channel value and a spatial value. The above-mentioned model generation system may further include:

[0120] A blank feature map template acquisition module is used to obtain a gradient matrix corresponding to the candidate feature map and a blank feature map template of the same size;

[0121] The channel mean value obtaining module is used to obtain the channel mean of each channel in the gradient matrix according to the feature values ​​of each feature point under the channel;

[0122] A module for obtaining a channel-type feature map template is used to set the pixel values ​​of the pixel points under the channel map corresponding to the channel whose channel mean is greater than or equal to the first response threshold in the blank feature map template to the channel value, and to set the pixel values ​​of the pixel points under the channel map corresponding to the channel whose channel mean is less than the first response threshold to 1, thereby obtaining a channel-type feature map template;

[0123] and / or,

[0124] The position mean value obtaining module is used to obtain the position mean of the plane position according to the feature values ​​of each feature point under the plane position for each plane position in the gradient matrix;

[0125] A module for obtaining a spatial feature map template is used to set the pixel values ​​of the pixels in the spatial map corresponding to the plane positions whose position mean is greater than or equal to the second response threshold in the blank feature map template to spatial values, and to set the pixel values ​​of the pixels in the spatial map corresponding to the plane positions whose position mean is less than the second response threshold to 1, thereby obtaining a spatial feature map template;

[0126] and / or,

[0127] A module for obtaining a combined feature map template is used to obtain a combined feature map template based on a channel feature map template and a spatial feature map template.

[0128] Optionally, the second enhanced image generation module may be specifically configured to:

[0129] Determine a missing feature map based on a target feature map determined from each candidate feature map and a target feature map template determined from each candidate feature map template;

[0130] The missing feature map is input into a data generator, and its corresponding sample image is used as supervision information to obtain a second enhanced image, wherein the data generator includes a decoding structure.

[0131] Alternatively, when the original segmentation model is iteratively trained based on multiple groups of training samples, the candidate feature map templates include feature map templates generated in the current iterative training process and feature map templates generated in the historical iterative training process before the current iterative training, and the probability of obtaining the target feature map template from the candidate feature map templates generated in the current iterative training process is greater than the probability of obtaining the target feature map template from the candidate feature map templates generated in the historical iterative training process.

[0132] Optionally, when the image template includes a first template and a second template, the first enhanced image includes a first sub-image and a second sub-image, and the model generation system may further include:

[0133] An initial template acquisition module, configured to acquire an initial template having the same size as the sample image, wherein the pixel value of each pixel in the initial template is a first value;

[0134] A second template generation module is used to set the pixel values ​​of the pixels in the randomly selected area of ​​the initial template to 1 to obtain the first template, and generate a second template complementary to the first template;

[0135] A second sub-image generating module, configured to generate a first sub-image based on the sample image and the first template, and to generate a second sub-image based on the sample image and the second template;

[0136] and / or,

[0137] When the image template includes a third template and a fourth template, and the first enhanced image includes a third sub-image and a fourth sub-image, the above-mentioned model generation system may further include:

[0138] an intermediate template acquisition module, configured to acquire at least two intermediate templates of the same size as the sample image, wherein the pixel value of each pixel in the intermediate template is a first value;

[0139] An intermediate template updating module, configured to set, for each intermediate template, pixel values ​​of pixels in a target area of ​​the intermediate template whose side length is a preset side length corresponding to the intermediate template to 1, and update the intermediate template according to the setting result;

[0140] a fourth template generating module, configured to use the union of the intermediate templates as a third template to generate a fourth template complementary to the third template;

[0141] The fourth sub-image generating module is configured to generate a third sub-image according to the sample image and the third template, and to generate a fourth sub-image according to the sample image and the fourth template.

[0142] Optionally, the original segmentation model may include a first segmentation network and a second segmentation network connected in sequence, the enhanced image includes a first enhanced image, and the model generation module 520 may include:

[0143] a third predicted label set obtaining unit, configured to input a sample image set comprising a plurality of sample images into the first segmentation network to obtain a first predicted label set, input a first enhanced image set comprising a plurality of first enhanced images into the first segmentation network to obtain a second predicted label set, and input the first predicted label set into the second segmentation network to obtain a third predicted label set;

[0144] The image segmentation model obtaining unit is used to pass the first prediction label set, the second prediction label set, the third prediction label set, and the segmentation label set containing multiple segmentation labels to the loss function, obtain the backpropagation gradient, and adjust the network parameters in the original segmentation model according to the backpropagation gradient to obtain the image segmentation model.

[0145] On this basis, optionally, the target feature map is a feature map obtained based on the sample image or the first enhanced image, the second enhanced image includes a sample enhanced image corresponding to the sample image and a corresponding enhanced image corresponding to the first enhanced image, and the model generation module 520 may further include:

[0146] a sixth predicted label set obtaining unit, configured to input a sample enhanced image set comprising a plurality of sample enhanced images into the first segmentation network to obtain a fourth predicted label set, input a corresponding enhanced image set comprising a plurality of corresponding enhanced images into the first segmentation network to obtain a fifth predicted label set, and input the fourth predicted label set into the second segmentation network to obtain a sixth predicted label set;

[0147] a fifth enhanced image set acquiring unit, configured to acquire a fifth enhanced image set generated according to the sample enhanced image set and the corresponding enhanced image set;

[0148] The image segmentation model obtains units, which may include:

[0149] A reconstruction return gradient obtaining subunit is used to pass the first prediction label set, the second prediction label set, the third prediction label set, the fourth prediction label set, the fifth prediction label set, the sixth prediction label set, and the segmentation label set including multiple segmentation labels to the segmentation loss function to obtain the segmentation return gradient, and pass the fifth enhanced image set and the sample image set to the image reconstruction loss function to obtain the reconstruction return gradient;

[0150] The first subunit of the return gradient is used to obtain the return gradient according to the segmentation return gradient and the reconstruction return gradient.

[0151] Optionally, the model generation module 520 may include:

[0152] a seventh reconstruction label set obtaining unit, configured to input the segmentation label set into the second segmentation network to obtain a seventh reconstruction label set;

[0153] The image segmentation model obtains units, which may include:

[0154] The second backpropagation gradient obtaining subunit is used to pass the first prediction label set, the second prediction label set, the third prediction label set, the seventh reconstruction label set, and the segmentation label set including multiple segmentation labels into the loss function to obtain the backpropagation gradient.

[0155] The model generation system provided by an embodiment of the present invention obtains a sample image, an enhanced image, and a segmentation label of the sample image through a training sample acquisition module, and uses the sample image and the segmentation label, and the enhanced image and the segmentation label as a group of training samples, respectively, wherein the enhanced image includes a first enhanced image generated based on the sample image and the image template, and / or a second enhanced image generated based on the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of a first numerical value, and the target feature map template includes pixel points with pixel values ​​of a second numerical value, and both the first numerical value and the second numerical value are less than a preset numerical value, which means that the first enhanced image generated after the image template is applied to the sample image can be understood as a sample image with some image details missing, and the missing feature map generated after the target feature map template is applied to the target feature map can be understood as a target feature map with some feature details missing. It is precisely because of this missing, the data distribution of the sample image and the first enhanced image, and the data distribution of the second enhanced image and the sample image generated based on the missing feature map may be different; further, the original segmentation model is trained based on multiple sets of training samples by the model generation module. Since the multiple sets of training samples are samples under multiple data distributions, the image segmentation model obtained by such training has strong robustness. The above system generates an enhanced image with a data distribution different from that of the sample image by performing a deletion operation on the sample image or the target feature map of the sample image. Then, the multiple groups of training samples including the sample image and the enhanced image are training samples under multiple data distributions, thereby achieving the effect of training an image segmentation model with strong robustness while only needing to obtain sample images under a limited data distribution.

[0156] The model generation system provided by the embodiment of the present invention can execute the model generation method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0157] It is worth noting that in the embodiment of the above-mentioned model generation system, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0158] Figure 11 This is a structural block diagram of an image segmentation system provided by an embodiment of the present invention, which is used to execute the image segmentation method provided by any of the above embodiments. This system and the image segmentation method of the above embodiments belong to the same inventive concept. For details not fully described in the embodiments of the image segmentation system, please refer to the embodiments of the above image segmentation method. Figure 11 The system may specifically include: an image segmentation model acquisition module 610 and an image segmentation module 620.

[0159] The image segmentation model acquisition module 610 is used to acquire a target image and an image segmentation model generated by the model generation method provided by any embodiment of the present invention;

[0160] The image segmentation module 620 is used to input the target image into the image segmentation model and obtain the target to be segmented in the target image according to the output result of the image segmentation model.

[0161] The image segmentation system provided by the embodiment of the present invention inputs the target image into the image segmentation model with strong robustness through the cooperation between the image segmentation model acquisition module and the image segmentation module, thereby accurately segmenting the target to be segmented from the target image, thereby ensuring the accuracy of image segmentation.

[0162] The image segmentation system provided by the embodiment of the present invention can execute the image segmentation method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0163] It is worth noting that in the embodiment of the above-mentioned image segmentation system, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0164] Figure 12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 12 As shown, the electronic device includes a memory 710, a processor 720, an input system 730, and an output system 740. The number of processors 720 in the electronic device can be one or more. Figure 12 In the figure, a processor 720 is used as an example; the memory 710, the processor 720, the input system 730 and the output system 740 in the electronic device can be connected via a bus or other means. Figure 12 The connection via bus 750 is taken as an example.

[0165] The memory 710, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the model generation method in the embodiment of the present invention (for example, the training sample acquisition module 510 and the model generation module 520 in the model generation system), or the program instructions / modules corresponding to the image segmentation method in the embodiment of the present invention (for example, the image segmentation model acquisition module 610 and the image segmentation module 620 in the image segmentation system). The processor 720 executes the software programs, instructions, and modules stored in the memory 710 to execute various functional applications and data processing of the electronic device, that is, to implement the above-mentioned model generation method or image segmentation method.

[0166] The memory 710 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 710 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 710 may further include a memory remotely located relative to the processor 720, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The input system 730 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the system. The output system 740 may include a display device such as a display screen.

[0168] An embodiment of the present invention provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform a model generation method. The method includes:

[0169] Obtain a sample image, an enhanced image, and a segmentation label of the sample image, and use the sample image and the segmentation label, and the enhanced image and the segmentation label as a set of training samples respectively;

[0170] The original segmentation model is trained based on multiple sets of training samples to obtain an image segmentation model;

[0171] In which, the enhanced image includes a first enhanced image generated based on the sample image and the image template, and / or a second enhanced image generated based on the target feature map of the sample image and the target feature map template, the image template includes pixel points with pixel values ​​of the first numerical value, and the target feature map template includes pixel points with pixel values ​​of the second numerical value, and both the first numerical value and the second numerical value are smaller than the preset numerical value.

[0172] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, and can also execute related operations in the model generation method provided by any embodiment of the present invention.

[0173] An embodiment of the present invention provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform an image segmentation method. The method includes:

[0174] Acquire a target image and an image segmentation model generated according to the model generation method provided by any embodiment of the present invention;

[0175] The target image is input into the image segmentation model, and the object to be segmented in the target image is obtained according to the output result of the image segmentation model.

[0176] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, but can also execute related operations in the image segmentation method provided by any embodiment of the present invention.

[0177] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer's floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0178] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A model generation method, characterized in that: include: Obtaining a sample image, an enhanced image, and a segmentation label of the sample image, and using the sample image and the segmentation label, and the enhanced image and the segmentation label as a group of training samples respectively; Training the original segmentation model based on the multiple groups of training samples to obtain an image segmentation model; The enhanced image includes a first enhanced image generated according to the sample image and an image template, and / or a second enhanced image generated according to a target feature map of the sample image and a target feature map template, the image template includes pixel points whose pixel values ​​are a first value, the target feature map template includes pixel points whose pixel values ​​are a second value, the first value and the second value are both less than a preset value, and the preset value is a positive number less than or equal to 1; The original segmentation model includes a first segmentation network, the first segmentation network includes an encoder and a transition decoupler connected in sequence, the transition decoupler includes a convolution structure, and the method further includes: Inputting the sample image into the first segmentation network, respectively obtaining an encoding feature map output by the encoder and a decoupling feature map output by the transition decoupler, and using both the encoding feature map and the decoupling feature map as candidate feature maps of the sample image; Acquire each candidate feature map template, wherein the candidate feature map template includes a randomly generated random feature map template and / or a targeted feature map template generated according to a gradient matrix corresponding to the candidate feature map; The second enhanced image is generated according to the target feature map determined from each of the candidate feature maps and the target feature map template determined from each of the candidate feature map templates.

2. The method according to claim 1, characterized in that Also includes: Obtain a random list, wherein the random list includes a first number of 1s and a second number of 0s, and the sum of the first number and the second number is the number of channels of the candidate feature map; The random feature map template is generated according to the random list, wherein the pixel value of the pixel point under the channel map corresponding to 1 in the random list in the random feature map template is 1, and the pixel value of the pixel point under the channel map corresponding to 0 in the random list is 0.

3. The method according to claim 1, characterized in that The targeted feature map template includes at least one of a targeted channel feature map template, a targeted space feature map template, and a targeted combined feature map template, the second value includes a preset channel value and a preset space value, and the method further includes: Obtain the gradient matrix and a blank feature map template of the same size; For each channel in the gradient matrix, obtaining a channel mean of the channel according to the feature values ​​of each feature point under the channel; The pixel values ​​of the pixels under the channel map corresponding to the channel whose channel mean is greater than or equal to the first response threshold in the blank feature map template are set to the channel value, and the pixel values ​​of the pixels under the channel map corresponding to the channel whose channel mean is less than the first response threshold are set to 1, to obtain the channel-type feature map template; and / or, For each plane position in the gradient matrix, obtaining a position mean value of the plane position according to the feature values ​​of each feature point at the plane position; The pixel values ​​of the pixels in the space map corresponding to the plane position whose position mean is greater than or equal to the second response threshold in the blank feature map template are set to the spatial value, and the pixel values ​​of the pixels in the space map corresponding to the plane position whose position mean is less than the second response threshold are set to 1, to obtain the space-type feature map template; and / or, According to the channel-type feature map template and the space-type feature map template, the combined-type feature map template is obtained.

4. The method according to claim 1, characterized in that The step of generating the second enhanced image according to the target feature map determined from each of the candidate feature maps and the target feature map template determined from each of the candidate feature map templates comprises: Determine a missing feature map according to the target feature map determined from each of the candidate feature maps and the target feature map template determined from each of the candidate feature map templates; The missing feature map is input into a data generator, and the sample image is used as supervisory information to obtain the second enhanced image, wherein the data generator includes a decoding structure.

5. The method according to claim 1, characterized in that When the original segmentation model is iteratively trained based on multiple groups of training samples, the candidate feature map templates include feature map templates generated in the current iterative training process and feature map templates generated in the historical iterative training process before the current iterative training, and the probability of obtaining the target feature map template from the candidate feature map templates generated in the current iterative training process is greater than the probability of obtaining the target feature map template from the candidate feature map templates generated in the historical iterative training process.

6. The method according to claim 1, characterized in that When the image template includes a first template and a second template, the first enhanced image includes a first sub-image and a second sub-image, and the method further includes: Acquire an initial template having the same size as the sample image, wherein the pixel value of each pixel point in the initial template is the first value; The pixel values ​​of the pixels in the randomly selected area in the initial template are set to 1 to obtain the first template, and the second template complementary to the first template is generated; generating the first sub-image according to the sample image and the first template, and generating the second sub-image according to the sample image and the second template; and / or, When the image template includes a third template and a fourth template, the first enhanced image includes a third sub-image and a fourth sub-image, and the method further includes: Acquire at least two intermediate templates having sizes consistent with the sample image, wherein the pixel value of each pixel point in the intermediate template is the first value; For each of the intermediate templates, setting the pixel values ​​of the pixels in the target area of ​​the intermediate template whose side length is a preset side length corresponding to the intermediate template to 1, and updating the intermediate template according to the setting result; Taking the union of the intermediate templates as the third template, and generating the fourth template complementary to the third template; The third sub-image is generated according to the sample image and the third template, and the fourth sub-image is generated according to the sample image and the fourth template.

7. The method according to claim 1, characterized in that The original segmentation model includes a first segmentation network and a second segmentation network connected in sequence, the enhanced image includes the first enhanced image, and the image segmentation model is obtained by training the original segmentation model based on multiple groups of training samples, including: Inputting a sample image set including a plurality of the sample images into the first segmentation network to obtain a first predicted label set, inputting a first enhanced image set including a plurality of the first enhanced images into the first segmentation network to obtain a second predicted label set, and inputting the first predicted label set into the second segmentation network to obtain a third predicted label set; The first prediction label set, the second prediction label set, the third prediction label set, and the segmentation label set including multiple segmentation labels are passed to the loss function to obtain the backpropagation gradient, and the network parameters in the original segmentation model are adjusted according to the backpropagation gradient to obtain the image segmentation model.

8. The method according to claim 7, characterized in that The target feature map is a feature map obtained according to the sample image or the first enhanced image, and the second enhanced image includes a sample enhanced image corresponding to the sample image and a corresponding enhanced image corresponding to the first enhanced image; The training of the original segmentation model based on the multiple groups of training samples to obtain the image segmentation model also includes: Inputting a sample enhanced image set including a plurality of the sample enhanced images into the first segmentation network to obtain a fourth predicted label set, inputting a corresponding enhanced image set including a plurality of the corresponding enhanced images into the first segmentation network to obtain a fifth predicted label set, and inputting the fourth predicted label set into the second segmentation network to obtain a sixth predicted label set; Acquire a fifth enhanced image set generated according to the sample enhanced image set and the corresponding enhanced image set; Accordingly, the first prediction label set, the second prediction label set, the third prediction label set, and the segmentation label set including the plurality of segmentation labels are passed to the loss function to obtain the back-transferred gradient, including: Passing the first prediction label set, the second prediction label set, the third prediction label set, the fourth prediction label set, the fifth prediction label set, the sixth prediction label set, and a segmentation label set including a plurality of the segmentation labels to a segmentation loss function to obtain a segmentation return gradient, and passing the fifth enhanced image set and the sample image set to an image reconstruction loss function to obtain a reconstruction return gradient; A return gradient is obtained according to the segmentation return gradient and the reconstruction return gradient.

9. The method according to claim 7, characterized in that: The training of the original segmentation model based on the multiple groups of training samples to obtain the image segmentation model also includes: Inputting the segmentation label set into the second segmentation network to obtain a seventh reconstruction label set; Accordingly, the first prediction label set, the second prediction label set, the third prediction label set, and the segmentation label set including the plurality of segmentation labels are passed to the loss function to obtain the back-transferred gradient, including: The first prediction label set, the second prediction label set, the third prediction label set, the seventh reconstruction label set, and a segmentation label set including a plurality of the segmentation labels are passed to a loss function to obtain a back-transferred gradient.

10. An image segmentation method, characterized in that: include: Acquire a target image and an image segmentation model generated according to any one of the methods of claims 1-9; The target image is input into the image segmentation model, and the object to be segmented in the target image is obtained according to the output result of the image segmentation model.

11. A model generation system, characterized in that: include: A training sample obtaining module, used for obtaining a sample image, an enhanced image and a segmentation label of the sample image, and taking the sample image and the segmentation label, and the enhanced image and the segmentation label as a group of training samples respectively; A model generation module, used for training the original segmentation model based on the multiple groups of training samples to obtain an image segmentation model; The enhanced image includes a first enhanced image generated according to the sample image and an image template, and / or a second enhanced image generated according to a target feature map of the sample image and a target feature map template, the image template includes pixel points whose pixel values ​​are a first value, the target feature map template includes pixel points whose pixel values ​​are a second value, the first value and the second value are both less than a preset value, and the preset value is a positive number less than or equal to 1; The original segmentation model includes a first segmentation network, the first segmentation network includes an encoder and a transition decoupler connected in sequence, the transition decoupler includes a convolution structure, and the system also includes: a candidate feature map acquisition module, a candidate feature map template acquisition module, and a second enhanced image generation module; The candidate feature map obtaining module is used to input the sample image into the first segmentation network, obtain the encoding feature map output by the encoder and the decoupling feature map output by the transition decoupler, and use both the encoding feature map and the decoupling feature map as candidate feature maps of the sample image; The candidate feature map template acquisition module is used to acquire each candidate feature map template, wherein the candidate feature map template includes a randomly generated random feature map template and / or a targeted feature map template generated according to a gradient matrix corresponding to the candidate feature map; The second enhanced image generation module is used to generate the second enhanced image based on the target feature map determined from each of the candidate feature maps and the target feature map template determined from each of the candidate feature map templates.

12. An image segmentation system, characterized in that: include: An image segmentation model acquisition module, used to acquire a target image and an image segmentation model generated according to any one of the methods of claims 1 to 9; The image segmentation module is used to input the target image into the image segmentation model and obtain the target to be segmented in the target image according to the output result of the image segmentation model.

13. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the model generation method as described in any one of claims 1 to 9, or the image segmentation method as described in claim 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model generation method as described in any one of claims 1 to 9 or the image segmentation method as described in claim 10 is implemented.

Images