Image Segmentation Method, System, Electronic Device and Medium Based on a Segmentation Model

Abstract

The present invention discloses an image segmentation method, system, electronic device and storage medium based on a segmentation model. The segmentation model includes an image inpainting model and an image synthesizer. The method includes: obtaining a sample image set; generating a curve structure library corresponding to the sample images based on a preset space colonization algorithm; performing skeleton annotation on the sample images, and dividing the sample image set after skeleton annotation to obtain an annotated sample subset and an original sample subset; inputting the annotated sample subset into the image inpainting model to extract the background and output an image background library; inputting the curve structure library and the image background library into the image synthesizer to output a synthesized image set; inputting the synthesized image set and the original sample subset into the segmentation model to obtain a target segmentation model; and inputting the obtained sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result. In the embodiments of the present invention, the annotation accuracy can be improved, the annotation cost can be reduced, and the image segmentation efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular, to an image segmentation method, system, electronic device, and storage medium based on a segmentation model. Background Art

[0002] With the rapid development of computer vision technology, computer vision technology has been applied in various fields of real life, including: the fields of logistics and transportation, medical devices, catering services, etc. In the biomedical field, research shows that the morphology and topology of specific curve anatomical structures are highly correlated with the presence or severity of various diseases, such as hypertension, keratitis, diabetic retinopathy, etc. Among them, the early symptoms of many ophthalmic diseases are reflected in the abnormalities of microvessels and capillaries. Therefore, the accurate segmentation of various curve structures is of great significance for computer-aided diagnosis, quantitative analysis, and early screening. In recent years, benefiting from the development of deep learning, many curve structure segmentation algorithms based on deep learning have been proposed, and compared with traditional methods, these algorithms have shown overwhelming performance. However, the above methods are usually fully supervised and require a large-scale well-annotated dataset, resulting in an increase in the financial cost and time cost required for annotating the dataset, and due to the fact that curve structures are slender, multi-scale, complex in shape, and fine in detail, the difficulty of annotating curve structures is increased, reducing the efficiency of image segmentation. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein.

[0004] Embodiments of the present invention provide an image segmentation method, system, electronic device, and storage medium based on a segmentation model, which can improve the annotation accuracy, reduce the annotation cost, and improve the image segmentation efficiency.

[0005] In a first aspect, an embodiment of the present invention provides an image segmentation method based on a segmentation model, where the segmentation model includes an image repair model and an image synthesizer, and the method includes:

[0006] Obtain a sample image set, where the sample image set includes multiple sample images;

[0007] Simulate the foreground of the sample image based on a preset space colonization algorithm to generate a curve structure library corresponding to the sample image;

[0008] Perform skeleton annotation on the sample image, and divide the sample image set according to the sample image after skeleton annotation to obtain an annotated sample subset and an original sample subset;

[0009] Input the annotated sample subset into the image repair model for background extraction, and output an image background library;

[0010] Input the curve structure library and the image background library into the image synthesizer for image synthesis, and output a set of synthesized images;

[0011] Input the set of synthesized images and the original sample subset into the segmentation model for training to obtain a target segmentation model;

[0012] Obtain a sample image to be detected, and input the sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result.

[0013] The image segmentation method based on a segmentation model provided by an embodiment of the present invention has at least the following beneficial effects: First, obtain a set of sample images including multiple sample images, and generate a curve structure library corresponding to the foreground of the sample images based on a preset space colonization algorithm, which is convenient for subsequent training of the image synthesizer and the segmentation model, improves the performance of the segmentation model. Then, perform noise skeleton annotation on the sample images, and divide the set of sample images according to the annotated sample images to obtain an annotated sample subset and an original sample subset, which is convenient for subsequent extraction of the image background, can improve the fully supervised performance of the segmentation model, and reduce the annotation cost. Then, input the annotated sample subset into an image restoration model for background extraction, and output an image background library, which realizes the removal of the foreground of the images in the annotated sample subset and improves the ability to extract the global background. Input the curve structure library and the image background library into the image synthesizer for image synthesis, and output a set of synthesized images, which is convenient for subsequent training of the segmentation model. Finally, input the set of synthesized images and the original sample subset into the segmentation model for training to obtain a target segmentation model, which realizes the training of the segmentation model, can transform the weakly supervised problem into a fully supervised problem, improves the performance of the segmentation model, and input the obtained sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result, thereby improving the accuracy of image segmentation.

[0014] In some embodiments, generating the curve structure library corresponding to the sample image based on the preset space colonization algorithm includes:

[0015] Perform foreground simulation on the sample image according to the space colonization algorithm to obtain an attractor set and a node set, where the attractor set includes multiple attractors and the node set includes multiple nodes;

[0016] Connect the attractors and the nodes according to a preset attraction distance to obtain connection information, where the connection information is used to characterize the influence ability of the nodes on the attractors;

[0017] For each node, determine multiple attractors affecting the node according to the connection information, and calculate the directions of the multiple attractors to obtain at least one average direction information;

[0018] Perform a node generation operation according to the average direction information, preset segment length information, and preset stop range to determine the number of target nodes;

[0019] Adjust the segment length information, the stop range, and the attraction distance based on preset boundary information and obstacle information to obtain various curve structures, where the stop range is used to represent the stop distance for setting the attractor;

[0020] Perform a diameter simulation on the branches of the curve structure to generate the curve structure library.

[0021] In some embodiments, the performing a node generation operation according to the average direction information, preset segment length information, and preset stop range to determine the number of target nodes includes:

[0022] Normalize the average direction information to obtain a unit vector;

[0023] Calculate the unit vector and the segment length information to determine position information, where the position information is used to represent the position for placing a new node;

[0024] Generate a target node according to the position information, and compare the stop range with the position information of the target node to obtain a comparison result;

[0025] When it is determined that the comparison result is that the target node is within the stop range, delete the attractor corresponding to the stop range, update the node set according to the target node, and update the attractor set;

[0026] For each updated node, determine multiple influencing attractors that affect the updated node according to the connection information, and calculate the influencing attractors until the current number of nodes meets the preset number of nodes, and determine the number of target nodes according to the preset number of nodes.

[0027] In some embodiments, the labeled sample subset includes multiple mask images, where the mask image is obtained by dilating the annotation of the sample image by a noise skeleton, and the image inpainting model includes an inpainting network, a discriminator, and an inpainter; the inputting the labeled sample subset into the image inpainting model for background extraction and outputting an image background library includes:

[0028] Input the labeled sample subset into the image inpainting model, so that the inpainting network in the image inpainting model performs image completion on the mask image and outputs a repaired image;

[0029] Evaluate the distance between the repaired image and the sample image based on a trained residual network to obtain a perceptual loss function;

[0030] Input the repaired image and the sample image into the discriminator for discrimination, and output discrimination parameters, where the discrimination parameters are used to characterize the feature definition performance and perceptual loss performance of the discriminator;

[0031] Generate the objective function of the repairer based on a preset gradient penalty function, preset hyperparameters, the perceptual loss function, and the discrimination parameters;

[0032] During a preset training period, train the repairer according to the objective function to obtain a target repairer;

[0033] Input the mask image into the target repairer, so that the target repairer removes the foreground in the mask image according to the noise skeleton, and extracts the background of the removed mask image to obtain multiple image backgrounds;

[0034] Construct the image background library according to multiple said image backgrounds.

[0035] In some embodiments, the inputting the curve structure library and the image background library into the image synthesizer for image synthesis to output a synthesized image set includes:

[0036] Generate an intermediate data set according to the curve structure library and the image background library, where the intermediate data set includes multiple temporary samples;

[0037] Input the intermediate data set and the original sample subset into the image synthesizer for multi-layer piece-by-piece contrast learning to obtain a synthesis loss function;

[0038] During a preset training period, train the image synthesizer according to the synthesis loss function;

[0039] Input the curve structure library and the image background library into the trained image synthesizer for image synthesis to output the synthesized image set.

[0040] In some embodiments, the image synthesizer includes an encoder and a multi-layer perceptron; the inputting the intermediate data set and the original sample subset into the image synthesizer for multi-layer piece-by-piece contrast learning to obtain a synthesis loss function includes:

[0041] Input the intermediate data set into the encoder for image downsampling to obtain a high-level feature map, and obtain patch information corresponding to the sample image according to the high-level feature map, where the patch information corresponds to the pixels in the high-level feature map;

[0042] Extract feature information of multiple scales from the patch information through the encoder;

[0043] Input the multi-scale feature information into the multi-layer perceptron for feature mapping, and output a feature stack;

[0044] Determine patch feature information and spatial location information according to the feature stack;

[0045] Calculate the patch feature information and the spatial location information based on a preset noise contrast estimation method to obtain a synthetic objective function;

[0046] Input the original sample subset into the encoder for sample selection to obtain patch features, negative samples, and positive samples;

[0047] Obtain a regularization loss value according to the patch features, the negative samples, and the positive samples;

[0048] Obtain the synthetic loss function according to the synthetic objective function, the regularization loss value, and a preset adversarial loss function.

[0049] In some embodiments, the training of the synthetic image set and the original sample subset by inputting them into the segmentation model to obtain a target segmentation model includes:

[0050] Input the synthetic image set into the segmentation model for training to obtain a rough segmenter, and input the original sample subset into the rough segmenter for segmentation prediction to obtain pseudo-label information;

[0051] Determine the rough segmentation loss value of the rough segmenter according to a preset segmentation loss function, and determine the pseudo-label loss value according to the pseudo-label information;

[0052] Determine the target loss function according to the rough segmentation loss value and the pseudo-label loss value;

[0053] Train the segmentation model based on the target loss function, the synthetic image set, and the original sample subset to obtain a fine segmenter;

[0054] Obtain the target segmentation model according to the rough segmenter and the fine segmenter.

[0055] In a second aspect, an image segmentation system based on a segmentation model provided by an embodiment of the present invention includes:

[0056] A sample acquisition module, configured to acquire a sample image set, where the sample image set includes multiple sample images;

[0057] A curve generation module, configured to simulate the foreground of the sample image based on a preset spatial colonization algorithm to generate a curve structure library corresponding to the sample image;

[0058] A skeleton annotation module, which is used to perform skeleton annotation on the sample image, and divide the sample image set according to the sample image after skeleton annotation to obtain an annotated sample subset and an original sample subset;

[0059] A background extraction module, which is used to input the annotated sample subset into the image inpainting model for background extraction, and output an image background library;

[0060] An image synthesis module, which is used to input the curve structure library and the image background library into the image synthesizer for image synthesis, and output a synthesized image set;

[0061] A model training module, which is used to input the synthesized image set and the original sample subset into the segmentation model for training to obtain a target segmentation model;

[0062] An image segmentation module, which is used to obtain a sample image to be detected, and input the sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result.

[0063] In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the image segmentation method based on a segmentation model as described in the first aspect.

[0064] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute the image segmentation method based on a segmentation model as described in the first aspect.

[0065] Other features and advantages of the present invention will be described in the following description, and some of them will become obvious from the description, or be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the description and the drawings. Description of the Drawings

[0066] The drawings are used to provide a further understanding of the technical solutions of the present invention, and constitute a part of the description. They are used together with the examples of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation to the technical solutions of the present invention.

[0067] Figure 1 is the overall flowchart of the image segmentation method based on a segmentation model provided by an embodiment of the present invention;

[0068] Figure 2 is Figure 1 the specific method flowchart of step S102 in

[0069] Figure 3 is Figure 2 the specific method flowchart of step S204 in

[0070] Figure 4 is Figure 1 the specific method flowchart of step S104 in

[0071] Figure 5 is Figure 1 the specific method flowchart of step S105 in

[0072] Figure 6 is Figure 5 the specific method flowchart of step S502 in

[0073] Figure 7 is Figure 1 the specific method flowchart of step S106 in

[0074] Figure 8 the structural schematic diagram of an image segmentation system based on a segmentation model provided by an embodiment of the present invention;

[0075] Figure 9 the hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0076] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0077] An embodiment of the present invention provides an image segmentation method based on a segmentation model. First, a sample image set including multiple sample images is obtained, and a curve structure library corresponding to the foreground of the sample images is generated based on a preset space colonization algorithm, which facilitates the subsequent training of the image synthesizer and the segmentation model and improves the performance of the segmentation model. Then, the sample images are subjected to noise skeleton annotation, and the annotated sample image set is divided into an annotated sample subset and an original sample subset, which facilitates the subsequent extraction of the image background, can improve the fully supervised performance of the segmentation model, and reduce the annotation cost. Then, the annotated sample subset is input into an image inpainting model for background extraction, and an image background library is output, realizing the removal of the foreground of the images in the annotated sample subset and improving the ability to extract the global background. The curve structure library and the image background library are input into an image synthesizer for image synthesis, and a synthesized image set is output, which facilitates the subsequent training of the segmentation model. Finally, the synthesized image set and the original sample subset are input into the segmentation model for training to obtain a target segmentation model, realizing the training of the segmentation model, which can convert the weakly supervised problem into a fully supervised problem, improve the performance of the segmentation model, and input the obtained sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result, thereby improving the accuracy of image segmentation.

[0078] The following further elaborates on the embodiments of the present invention with reference to the accompanying drawings.

[0079] It should be noted that the segmentation model in this embodiment is a new weakly supervised framework for single-sample skeleton / graffiti supervised curve structure segmentation, called YoloCurvSeg. Among them, the segmentation model includes a curve generator for generating a binary curve mask similar to the foreground of the corresponding sample image, a background generator for extracting the background from the labeled samples, an image synthesizer for generating synthesized images, and a two-stage segmenter that needs to be trained.

[0080] Refer to Figure 1 , an embodiment of the present invention provides an image segmentation method based on a segmentation model. The image segmentation method based on the segmentation model includes but is not limited to the following steps S101 to step S107.

[0081] Step S101, obtain a sample image set;

[0082] It should be noted that the sample image set includes multiple sample images.

[0083] In some embodiments, a sample image set is obtained, where the sample image set is obtained by collecting blood vessels, nerve fibers, cells, etc. The specific type of the sample image set can be collected by the user according to needs, and this embodiment does not make specific limitations.

[0084] It can be understood that the sample image set can be obtained in the following ways: obtained from an open-source database; collected by taking photos by oneself; recorded by microscopic observation, etc.

[0085] Step S102, simulate the foreground of the sample image based on a preset space colonization algorithm to generate a curve structure library corresponding to the sample image;

[0086] In some embodiments, a curve structure library corresponding to the foreground of the sample image is generated based on a preset space colonization algorithm.

[0087] It can be understood that the space colonization algorithm is a procedural modeling algorithm in computer graphics, which can be used to simulate the growth of branched networks or tree-like structures, including blood vessels, leaf veins, roots, etc. In this embodiment, it is used to simulate the iterative growth of curve structures.

[0088] It should be noted that the segmentation model includes a curve generator, and step S102 occurs in the curve generator.

[0089] Step S103, perform skeleton annotation on the sample image, and divide the sample image set according to the sample image after skeleton annotation to obtain an annotated sample subset and an original sample subset;

[0090] In some embodiments, perform skeleton annotation on the sample image, where the dilated noisy skeleton can fully cover the foreground of the sample image, and divide the sample image set according to the sample image after skeleton annotation to obtain an annotated sample subset and an original sample subset, which is convenient for image repair and background extraction.

[0091] It can be understood that the original sample subset is the unannotated sample subset.

[0092] Step S104, input the annotated sample subset into an image repair model for background extraction, and output an image background library;

[0093] In some embodiments, input the annotated sample subset into an image repair model for image repair and background extraction, and output an image background library. Among them, the image background library includes the foregrounds of multiple annotated images, realizing the extraction of the image background, which is convenient for subsequent image synthesis.

[0094] Step S105, input the curve structure library and the image background library into an image synthesizer for image synthesis, and output a synthesized image set;

[0095] In some embodiments, input the curve structure library and the image background library into an image synthesizer, so that the image synthesizer extracts the curve structures in the curve structure library and extracts the image backgrounds in the image background library, and performs image synthesis on the extracted curve structures and image backgrounds, and outputs a synthesized image set, which is convenient for subsequent training of the segmentation model.

[0096] Step S106: Input the synthetic image set and the original sample subset into the segmentation model for training to obtain the target segmentation model;

[0097] In some embodiments, input the synthetic image set and the original sample subset into the segmentation model for training to obtain a two-stage segmenter, realize the coarse-to-fine segmentation of the segmentation model, obtain the target segmentation model, and improve the segmentation ability of the segmentation model.

[0098] Step S107: Obtain the sample image to be detected, and input the sample image to be detected into the target segmentation model for image segmentation to obtain the segmentation result.

[0099] In some embodiments, obtain the sample image to be detected, and input the sample image to be detected into the target segmentation model for image segmentation, so as to realize the segmentation of the sample image to be detected and obtain the final segmentation result.

[0100] Refer to Figure 2 , in some embodiments, in order to generate a curve structure library corresponding to the sample image, a space colonization algorithm can be used to generate a binary curve mask similar to the image foreground of the sample image, so as to facilitate the subsequent generation of the synthetic image set. Among them, step S102 may include but is not limited to steps S201 to S206:

[0101] Step S201: Perform foreground simulation on the sample image according to the space colonization algorithm to obtain an attractor set and a node set;

[0102] It should be noted that the attractor set includes multiple attractors, and the node set includes multiple nodes.

[0103] In some embodiments, perform foreground simulation on the sample image according to the space colonization algorithm to obtain an attractor set and a node set, which is convenient for subsequent generation of the corresponding curve structure.

[0104] Step S202: Connect the attractors and nodes according to a preset attraction distance to obtain connection information;

[0105] It should be noted that the connection information is used to characterize the influence ability of the node on the attractor.

[0106] In some embodiments, connect the attractors and nodes according to a preset attraction distance to obtain connection information, so as to accurately determine the distance between the attractors and nodes.

[0107] It can be understood that a group of attractors is randomly or selected from the attractor set according to a predetermined pattern, and the nodes are connected to the nearby attractors according to a preset attraction distance D a to obtain connection information.

[0108] Step S203: For each node, determine multiple attractors affecting the node according to the connection information, and calculate the directions of the multiple attractors to obtain at least one average direction information.

[0109] In some embodiments, for each node, determine multiple attractors affecting the node according to the connection information, and calculate the directions of the multiple attractors to obtain at least one average direction information, which is convenient for subsequently determining the positions of new nodes.

[0110] Step S204: Perform node generation operations according to the average direction information, the preset segment length information, and the preset stop range to determine the target number of nodes.

[0111] In some embodiments, perform node generation operations according to the average direction information, the preset segment length information L s and the preset stop range D k to determine the target number of nodes, where the target number of nodes is the maximum number of nodes that the nodes can reach, thereby increasing the accuracy of the simulated curve structure.

[0112] Step S205: Adjust the segment length information, the stop range, and the attraction distance based on the preset boundary information and obstacle information to obtain various curve structures.

[0113] It should be noted that the stop range is used to represent the stop distance set by the attractor.

[0114] In some embodiments, within the preset boundary information and obstacle information, adjust the segment length information L s , the stop range D k and the attraction distance D a to obtain various curve structures, thereby improving the accuracy of the simulated curve.

[0115] It should be noted that during the parameter adjustment process, it is also necessary to adjust the root node coordinates C r to simulate various curve structures, thereby making the simulated structure more accurate.

[0116] Step S206: Simulate the diameters of the branches of the curve structure to generate a curve structure library.

[0117] In some embodiments, in addition to simulating the curve shape, it is also necessary to simulate the diameters of the branches of the curve structure to generate a curve structure library.

[0118] It should be noted that the calculation method of the diameter simulation is shown in the following formula (1):

[0119]

[0120] where R is the branch radius of the parent branch, and are the branch radii of the two sub - branches respectively. In this embodiment, the calculation is performed recursively from the branch tip (whose radius is set to 1) towards the tree base. By pre - defining parameter settings for the random grid attractor and the root node, multiple curve structure libraries are constructed, and then they are used to train the segmentation model.

[0121] Referring to Figure 3 , in some embodiments, in order to accurately simulate the foreground of the sample image, nodes can be generated according to the average direction information, the preset segment length information, and the stop range, so as to increase the accuracy of the simulated curve structure. Among them, step S204 may include but is not limited to steps S301 to S305:

[0122] Step S301: Normalize the average direction information to obtain a unit vector;

[0123] Step S302: Calculate the unit vector and the segment length information to determine the position information;

[0124] It should be noted that the position information is used to represent the position where the new node is placed.

[0125] Step S303: Generate a target node according to the position information, and compare the stop range with the position information of the target node to obtain a comparison result;

[0126] Step S304: When it is determined that the comparison result is that the target node is within the stop range, delete the attractor corresponding to the stop range, update the node set according to the target node, and update the attractor set;

[0127] Step S305: For each updated node, determine multiple influencing attractors that affect the updated node according to the connection information, and calculate the influencing attractors until the current number of nodes meets the preset number of nodes, and determine the number of target nodes according to the preset number of nodes.

[0128] In steps S301 to S305 of some embodiments, during the process of determining the number of target nodes, it is necessary to first normalize the average direction information to obtain a unit vector, then calculate according to the unit vector, and scale the position of the new node according to the preset segment length information L s to determine the position information of the new node. Then, place the target node at the calculated position information, and compare the stop range D k with the position information of the target node, so as to be able to check whether there is a node within the stop range of the attractor. When it is determined that the target node is within the stop range D k, the pruning operation is performed on the attractors, the attractors located within the stop range are deleted, and the node set and the attractor set are updated. Finally, for each updated node, the multiple influencing attractors affecting the updated node are determined again according to the connection information, and the influencing attractors are calculated to obtain the average direction of the influencing attractors, that is, steps S301 to S305 are repeated until the current number of nodes meets the preset number of nodes. The target nodes are determined according to the preset number of nodes, where the preset number of nodes is the maximum number of nodes that the nodes can reach, so as to accurately calculate the number of nodes and facilitate the simulation of the curve structure.

[0129] Referring to Figure 4 , in some embodiments, in order to extract the background of the labeled sample subset, the labeled sample subset can be input into an image inpainting model for background extraction, so that an image background library can be generated through image completion operations, which is convenient for subsequent image synthesis. Among them, step S104 may include but is not limited to steps S401 to S407:

[0130] It should be noted that the labeled sample subset includes multiple mask images, where the mask images are obtained by dilating the sample images labeled by the noise skeletons. The image inpainting model includes a repair network, a discriminator, and a restorer.

[0131] It can be understood that the repair network is a repair network based on Fast Fourier Convolution (FFC).

[0132] Step S401, input the labeled sample subset into the image inpainting model, so that the repair network in the image inpainting model performs image completion on the mask image and outputs a repaired image;

[0133] In some embodiments, the labeled sample subset is input into the image inpainting model, so that the repair network in the image inpainting model performs image completion on the mask image and outputs a repaired image, thereby realizing the repair of the image.

[0134] It should be noted that the mask image is represented as I⊙(1 - m), where I represents the sample image and m represents the binary mask of the repair area. The feedforward repair network f θ (·) repairs the four-channel input I′ = concat(I⊙(1 - m), m), and thus the output repaired image

[0135] It should be noted that FFC is based on the channel-scale Fast Fourier Transform (FFT) and has a receptive field covering the entire image. It divides the channels into two parallel branches: a local branch using conventional convolution and a global branch using the real-valued FFT to capture global context. The real-valued FFT is only applicable to real-valued signals, and the inverse real-valued FFT ensures that the output is real-valued. Compared with the FFT, the real FFT only uses half of the spectrum. In FFC, the real FFT is first applied to the input tensor, and the complex-to-real operation is performed by concatenating the real and imaginary parts. Then, convolution is applied in the frequency domain. After the real-to-complex operation, the inverse transform is applied to restore the spatial structure. Finally, the local and global branches are fused.

[0136] Step S402: Based on the trained residual network, evaluate the distance between the restored image and the sample image to obtain the perceptual loss function.

[0137] In some embodiments, based on the trained residual network, evaluate the distance between the restored image and the sample image to obtain the perceptual loss function, so as to accurately evaluate the distance between the restored image and the sample image.

[0138] It should be noted that, compared with the supervision loss that may lead to blurred prediction, the perceptual loss evaluates the distance between the restored image and the sample image through the pre-trained network φ(·), and it does not require precise reconstruction, allowing the reconstructed image to have diversity. Given that restoration focuses on understanding the global structure, in this embodiment, the perceptual loss with a large receptive field is introduced through the pre-trained Residual Network 50 (ResNet50) φ HRF (·), where the specific perceptual loss function is shown in the following formula (2):

[0139]

[0140] where is a continuous two-stage averaging operation, that is, obtaining the inter-layer average of the in-layer average. In addition, the adversarial loss is used to encourage the authenticity of the restored image, and the unsaturated adversarial loss is defined as shown in the following formulas (3)-(5):

[0141]

[0142]

[0143]

[0144] It is understandable that for the repaired output, sg var represents the stop gradient with respect to var, E I，m is a defined mathematical parameter.

[0145] Step S403: Input the repaired image and the sample image into the discriminator for discrimination, and output discrimination parameters;

[0146] It should be noted that the discrimination parameters are used to characterize the feature definition performance and perceptual loss performance of the discriminator.

[0147] In some embodiments, the repaired image and the sample image are input into the discriminator for discrimination, and the image intersecting with the mask image is marked as false, and other images are marked as true, and discrimination parameters are output so as to obtain the feature definition performance and perceptual loss performance of the discriminator.

[0148] Step S404: Generate the objective function of the repairer based on a preset gradient penalty function, preset hyperparameters, perceptual loss function, and discrimination parameters;

[0149] In some embodiments, target parameters are generated based on a preset gradient penalty function, preset hyperparameters, perceptual loss function, and discrimination parameters, so as to ensure the local details and authenticity during the process of the repairer repairing the image, and improve the performance of image repair.

[0150] It should be noted that the objective function of the repairer is shown in the following formula (6):

[0151]

[0152] Among them, λ adv , λ DP and λ GP are hyperparameters for balancing the contribution degrees of different losses, is the perceptual loss function, which is used to be responsible for the consistency of the supervision signal and the global structure, and are respectively responsible for local details and authenticity, is the gradient penalty function, and the gradient penalty function is expressed as the following formula (7):

[0153]

[0154] It should be noted that the hyperparameters in this embodiment are set as λ adv =3, λ DP =10, λ GP =10 -4 .

[0155] Step S405: During a preset training period, train the restorer according to the objective function to obtain the target restorer.

[0156] In some embodiments, during a preset training period, when training the restorer according to the objective function, since the training data of the restorer does not require annotation and it has learned the general ability to restore missing regions through context understanding, a unified restorer is used in this embodiment. This restorer was pre-trained on the Places Challenge dataset and fine-tuned on the combination of all the datasets used to obtain the target restorer.

[0157] It should be noted that during the process of training the restorer in this embodiment, training is carried out with a batch size of 8, and the Adam optimizer is used with a learning rate of 10 -3 , the training period is 50 epochs, and data augmentation techniques such as random flipping, rotation, and color jitter are used to enhance the training data and improve the training efficiency of the restorer.

[0158] Step S406: Input the mask image into the target restorer, so that the target restorer removes the foreground in the mask image according to the noise skeleton and extracts the background of the mask image after removal to obtain multiple image backgrounds.

[0159] In some embodiments, after obtaining the target restorer, input the mask image into the target restorer, so that the target restorer removes the foreground of the mask image marked by the noise skeleton, and uses the dilated noise skeleton as a mask to extract the background of the mask image after removal to obtain multiple image backgrounds, thereby realizing the extraction of the image background.

[0160] It should be noted that the process of extracting the background of the mask image in this embodiment includes but is not limited to operations such as random flipping and rotation, and no specific limitations are made in this embodiment.

[0161] Step S407: Construct an image background library based on the multiple image backgrounds.

[0162] In some embodiments, constructing an image background library based on the multiple image backgrounds facilitates the subsequent training of the segmentation model.

[0163] Refer to Figure 5 , in some embodiments, in order to obtain a synthetic image, image synthesis can be performed using the curve structure library obtained in step S102 and the image background library obtained in step S104 to generate a synthetic image set and improve the authenticity of the image synthesis. Among them, step S105 may include but is not limited to steps S501 to S504:

[0164] Step S501: Generate an intermediate dataset according to the curve structure library and the image background library.

[0165] It should be noted that the intermediate dataset includes multiple temporary samples;

[0166] In some embodiments, one curve structure c curv ={c 1 , …, c N} and one image background b bg ={b 1 , …, b N} are randomly selected from the curve structure library B i and the image background library B i , and then they are concatenated to form a temporary sample x i =concat(b i , c i ), thereby constructing an intermediate dataset X inter ={x 1 , …, x N}, which can transform the weakly supervised problem into a conversion task from an unpaired intermediate dataset to a subset of the original samples, that is, design a synthesizer to learn the mapping from the intermediate dataset to the corresponding subset of the original samples, and the specific steps will be described in detail later.

[0167] Step S502, input the intermediate dataset and the subset of the original samples into the image synthesizer for multi-layer slice-by-slice contrastive learning to obtain a synthetic loss function;

[0168] In some embodiments, the intermediate dataset X inter ={x 1 , …, x N} and the subset of the original samples Y are input into the image synthesizer for multi-layer slice-by-slice contrastive learning to obtain a synthetic loss function, which is convenient for subsequent training of the image synthesizer, thereby increasing the authenticity of image synthesis.

[0169] Step S503, within a preset training period, train the image synthesizer according to the synthetic loss function;

[0170] In some embodiments, within a preset training period, train the image synthesizer according to the synthetic loss function, where the training period can be 100, 200, or 300, etc., and in this embodiment, it is 300.

[0171] It should be noted that during the training of the image synthesizer, the training of the image synthesizer uses an Adaptive Moment Estimation (Adam) optimizer with a learning rate of 10 -4 and uses a cosine decay strategy, and the batch size is 1.

[0172] Step S504: Input the curve structure library and the image background library into the trained image synthesizer for image synthesis, and output a set of synthesized images.

[0173] In some embodiments, input the curve structure library B curv ={c 1 , …, c N} and the image background library into the trained image synthesizer for image synthesis, and output a set of synthesized images, thereby improving the authenticity of the synthesized images.

[0174] Referring to Figure 6 , in some embodiments, in order to improve the synthesis efficiency and synthesis accuracy of the image synthesizer, the intermediate data set and the original sample subset can be input into the image synthesizer for multi-layer piece-by-piece contrast learning, thereby realizing the training of the image synthesizer and increasing the authenticity of image synthesis. Among them, step S502 may include but is not limited to steps S601 to S608:

[0175] It should be noted that the image synthesizer includes an encoder and a multi-layer perceptron.

[0176] It is worth noting that for unpaired image translation, most existing methods use a generative adversarial network (GAN) with a cyclic structure, relying on cycle consistency to ensure the correspondence of high-level semantics. Although the GAN with a cyclic structure is effective, the underlying bijective behind cycle consistency is sometimes too restrictive / strict, which may reduce the diversity of the generated samples. More importantly, cycle consistency is not suitable for the synthesis task of this application because it cannot guarantee any explicit or implicit spatial constraints. In this case, this application introduces a synthesizer based on multi-layer piece-by-piece contrast learning to learn the mapping from the intermediate data set to the original data subset, which will be specifically described below.

[0177] Step S601: Input the intermediate data set into the encoder for image downsampling to obtain a high-level feature map, and obtain patch information corresponding to the sample image according to the high-level feature map;

[0178] It should be noted that the patch information corresponds to the pixels in the high-level feature map.

[0179] In some embodiments, input the intermediate data set X inter ={x 1 , …, x N} into the encoder for image downsampling operation, and input the intermediate data set X inter ={x 1 , …, x N} Downsample to high-level features to obtain a high-level feature map. Among them, the encoder has 3 residual blocks equipped with instance normalization and ReLU activation. Therefore, the patch information corresponding to the sample image can be directly obtained according to the high-level feature map, that is, each pixel in the high-level feature map represents the embedded feature vector of the patch in the sample image.

[0180] Step S602, extract features from the patch information through the encoder to obtain multi-scale feature information;

[0181] In some embodiments, through several interested layers E l∈L (x) in the encoder, extract features from the patch information to obtain multi-scale feature information.

[0182] Step S603, input the multi-scale feature information into a multi-layer perceptron for feature mapping and output a feature stack;

[0183] In some embodiments, input the scale feature information into a two-layer multi-layer perceptron for feature mapping to obtain a feature stack {v l∈L = H l∈L [E l∈L (x)]}.

[0184] Step S604, determine the patch feature information and spatial position information according to the feature stack;

[0185] In some embodiments, according to the feature stack {v l∈L = H l∈L [E l∈L (x)]}, determine the patch feature information v l and the corresponding pair And according to the pair Determine the spatial position information s1 and s2.

[0186] Step S605, calculate the patch feature information and the spatial position information based on the preset noise contrast estimation method to obtain a synthesis objective function;

[0187] In some embodiments, calculate the patch feature information and the spatial position information based on the preset Noise Contrastive Estimation (NCE) to obtain a synthesis objective function, so as to be able to maintain local information at the same spatial position. Among them, the specific process of obtaining the synthesis objective function is shown in the following formula (8):

[0188]

[0189] Among them, v + and v are patches with the same position, Represents the nth of n patches at different positions, and τ is the temperature hyperparameter.

[0190] Step S606: Input the original sample subset into the encoder for sample selection to obtain patch features, negative samples, and positive samples.

[0191] Step S607: Obtain the regularization loss value based on the patch features, negative samples, and positive samples.

[0192] In steps S606 to S607 of some embodiments, input the original sample subset into the encoder for sample selection to obtain patch feature v * , negative sample v *- and positive sample Then obtain the regularization loss value according to the above information, and the specific process is shown in the following formula (9):

[0193]

[0194] Step S608: Obtain the synthesis loss function according to the synthesis objective function, the regularization loss value, and a preset adversarial loss function.

[0195] In some embodiments, according to the synthesis objective function regularization loss value and a preset adversarial loss function obtain the synthesis loss function to facilitate the subsequent training of the image synthesizer and improve the authenticity of the synthesized image. The specific process of obtaining the synthesis loss function is shown in the following formula (10):

[0196]

[0197] where λ adv , λ c and λ id are trade-off parameters.

[0198] Referring to Figure 7 , in some embodiments, in order to achieve accurate segmentation of the image, the synthesized image set and the original sample subset can be input into the segmentation model for training, so as to improve the segmentation ability of the segmentation model and achieve a coarse-to-fine segmentation of the segmentation model. Among them, step S106 may include but is not limited to steps S701 to S705:

[0199] It should be noted that when training the segmentation model using the combination of the synthesized image set and the original image subset, the weakly supervised task is converted into a fully supervised or semi-supervised task. In this embodiment, a two-stage coarse-to-fine segmentation technical route is used to process this task.

[0200] Step S701: Input the synthetic image set into the segmentation model for training to obtain a rough segmenter, and input the original sample subset into the rough segmenter for segmentation prediction to obtain pseudo-label information;

[0201] In some embodiments, first input the synthetic image set into the segmentation model for training, and train a specific segmentation network on the synthetic image set to obtain a rough segmenter S coarse , and input the original sample subset into the rough segmenter S coarse for segmentation prediction, and use the prediction of the rough segmenter S coarse on the original sample subset as the label to obtain pseudo-label information.

[0202] It should be noted that the performance of the rough segmenter S coarse is mainly limited by two problems; one is that there is still a certain morphological gap between the curve structure generated by the curve generator and the foreground of the real image, and the other is that there is also a slight but inevitable intensity gap between the synthetic image set generated by the image synthesizer and the real data. Therefore, it is necessary to use the synthetic image set to further improve the segmentation performance.

[0203] Step S702: Determine the rough segmentation loss value of the rough segmenter according to the preset segmentation loss function, and determine the pseudo-label loss value according to the pseudo-label information;

[0204] In some embodiments, determine the rough segmentation loss value of the rough segmenter according to the preset segmentation loss function and determine the pseudo-label loss value according to the pseudo-label information

[0205] It should be noted that the rough segmenter is trained on the original data subset, and the formula for determining the rough segmentation loss value according to the segmentation loss function is shown in the following formula (11):

[0206]

[0207] where and represent the cross-entropy loss and the Dice loss respectively.

[0208] Step S703: Determine the target loss function according to the rough segmentation loss value and the pseudo-label loss value;

[0209] In some embodiments, determine the target loss function according to the rough segmentation loss value and the pseudo-label loss value, where the specific process of obtaining the target loss function is shown in the following formula (12):

[0210]

[0211] Among them, λ psd is a trade-off parameter, and the trade-off parameter is set to 1.

[0212] Step S704: Train the segmentation model based on the target loss function, the synthetic image set, and the original sample subset to obtain a refined segmenter;

[0213] In some embodiments, the refined segmenter is obtained by training with the target loss function on the combination of the synthetic image set and the original sample subset, thereby improving the segmentation accuracy of the image.

[0214] Step S705: Obtain the target segmentation model according to the coarse segmenter and the refined segmenter.

[0215] In some embodiments, the present embodiment uses the original U-Net with 16, 32, 64, 128, and 256 feature channels as the coarse segmenter S coarse and the refined segmenter S fine structure. And the SGD optimizer (weight decay = 10-4, momentum = 0.9) is used to train the coarse segmenter S coarse and the refined segmenter S fine , where the batch size is 12 and the initial learning rate is 10 -2 . And finally, the target segmentation model is obtained according to the coarse segmenter S coarse and the refined segmenter S fine to improve the segmentation accuracy of the segmentation model and reduce the cost of labeled samples.

[0216] Please refer to Figure 8 , the embodiment of the present application also provides an image segmentation system based on a segmentation model, which can implement the above-mentioned image segmentation method based on a segmentation model. The system includes:

[0217] A sample acquisition module 801, configured to acquire a sample image set, where the sample image set includes multiple sample images;

[0218] A curve generation module 802, configured to simulate the foreground of the sample image based on a preset space colonization algorithm to generate a curve structure library corresponding to the sample image;

[0219] A skeleton annotation module 803, configured to perform skeleton annotation on the sample image and divide the sample image set according to the skeleton-annotated sample image to obtain a labeled sample subset and an original sample subset;

[0220] A background extraction module 804, configured to input the labeled sample subset into an image inpainting model for background extraction and output an image background library;

[0221] The image synthesis module 805 is configured to input a curve structure library and an image background library into an image synthesizer for image synthesis, and output a set of synthesized images;

[0222] The model training module 806 is configured to input the set of synthesized images and the original sample subset into a segmentation model for training to obtain a target segmentation model;

[0223] The image segmentation module 807 is configured to obtain a sample image to be detected, and input the sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result.

[0224] The specific implementation manner of the image segmentation system based on the segmentation model is basically the same as the specific embodiments of the above-mentioned image segmentation method based on the segmentation model, and will not be elaborated here.

[0225] An embodiment of this application further provides an electronic device. The electronic device includes: a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. When the program is executed by the processor, the above-mentioned image segmentation method based on the segmentation model is realized. The electronic device may be any intelligent terminal including a tablet computer, an in-vehicle computer, etc.

[0226] Please refer to Figure 9 , Figure 9 which schematically shows the hardware structure of an electronic device according to another embodiment. The electronic device includes:

[0227] The processor 901 may be implemented by using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of this application;

[0228] The memory 902 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 902 may store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 902 and are called by the processor 901 to execute the image segmentation method based on the segmentation model of the embodiments of this application;

[0229] The input / output interface 903 is configured to implement information input and output;

[0230] A communication interface 904 for implementing communication interaction between this device and other devices, which can achieve communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0231] A bus 905 for transmitting information between various components of the device (such as a processor 901, a memory 902, an input / output interface 903, and a communication interface 904);

[0232] Among them, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904 achieve communication connections with each other inside the device through the bus 905.

[0233] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned image segmentation method based on a segmentation model.

[0234] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0235] The embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more control processors, for example, by Figure 9 one of the processors 901, it can cause the above-mentioned one or more control processors to execute the image segmentation method based on a segmentation model in the above method embodiment.

[0236] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0237] Those of ordinary skill in the art will understand that all or some of the steps and systems disclosed above in the methods can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or can be implemented as hardware, or can be implemented as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage systems, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0238] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the present invention.

Claims

1. An image segmentation method based on a segmentation model, characterized in that The segmentation model includes an image inpainting model and an image synthesizer, and the method includes: Obtain a sample image set, where the sample image set includes multiple sample images; Simulate the foreground of the sample images based on a preset space colonization algorithm to generate a curve structure library corresponding to the sample images; Perform skeleton annotation on the sample images, and divide the sample image set according to the skeleton-annotated sample images to obtain an annotated sample subset and an original sample subset; Input the annotated sample subset into the image inpainting model for background extraction, and output an image background library; Input the curve structure library and the image background library into the image synthesizer for image synthesis, and output a synthesized image set; Input the synthesized image set and the original sample subset into the segmentation model for training to obtain a target segmentation model; Obtain a sample image to be detected, and input the sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result.

2. The image segmentation method based on a segmentation model according to claim 1, wherein The simulating the foreground of the sample images based on a preset space colonization algorithm to generate a curve structure library corresponding to the sample images includes: Perform foreground simulation on the sample images according to the space colonization algorithm to obtain an attractor set and a node set, where the attractor set includes multiple attractors and the node set includes multiple nodes; Connect the attractors and the nodes according to a preset attraction distance to obtain connection information, where the connection information is used to characterize the influence ability of the nodes on the attractors; For each node, determine multiple attractors affecting the node according to the connection information, and calculate the directions of the multiple attractors to obtain at least one average direction information; Perform a node generation operation according to the average direction information, a preset segment length information, and a preset stop range to determine the number of target nodes; Adjust the segment length information, the stop range, and the attraction distance based on preset boundary information and obstacle information to obtain multiple curve structures, where the stop range is used to characterize the stop distance set by the attractors; Perform diameter simulation on the branches of the curve structures to generate the curve structure library.

3. The image segmentation method based on a segmentation model according to claim 2, wherein The performing a node generation operation according to the average direction information, a preset segment length information, and a preset stop range to determine the number of target nodes includes: Normalize the average direction information to obtain a unit vector; Calculate the unit vector and the segment length information to determine position information, where the position information is used to characterize the position for placing a new node; Generate a target node according to the position information, and compare the stop range with the position information of the target node to obtain a comparison result; When it is determined that the comparison result is that the target node is within the stop range, delete the attractor corresponding to the stop range, update the node set according to the target node, and update the attractor set; For each updated node, determine multiple influencing attractors that affect the updated node according to the connection information, and calculate the influencing attractors until the current number of nodes meets a preset number of nodes, and determine the target number of nodes according to the preset number of nodes.

4. The image segmentation method based on a segmentation model according to claim 1, wherein The labeled sample subset includes multiple mask images, where the mask images are obtained by dilating the sample images labeled by the noise skeleton, and the image inpainting model includes an inpainting network, a discriminator, and an inpainter; the inputting the labeled sample subset into the image inpainting model for background extraction and outputting an image background library includes: Input the labeled sample subset into the image inpainting model, so that the inpainting network in the image inpainting model completes image inpainting on the mask image and outputs an inpainted image; Based on the trained residual network, evaluate the distance between the inpainted image and the sample image to obtain a perceptual loss function; Input the inpainted image and the sample image into the discriminator for discrimination and output discrimination parameters, where the discrimination parameters are used to characterize the feature definition performance and perceptual loss performance of the discriminator; Generate an objective function for the inpainter based on a preset gradient penalty function, preset hyperparameters, the perceptual loss function, and the discrimination parameters; During a preset training period, train the inpainter according to the objective function to obtain a target inpainter; Input the mask image into the target inpainter, so that the target inpainter removes the foreground in the mask image according to the noise skeleton and performs background extraction on the mask image after removal to obtain multiple image backgrounds; Construct the image background library according to the multiple image backgrounds.

5. The image segmentation method based on a segmentation model according to claim 1, characterized in that The inputting the curve structure library and the image background library into the image synthesizer for image synthesis and outputting a synthesized image set includes: Generate an intermediate data set according to the curve structure library and the image background library, where the intermediate data set includes multiple temporary samples; Input the intermediate data set and the original sample subset into the image synthesizer for multi-layer piece-by-piece contrast learning to obtain a synthesis loss function; During a preset training period, train the image synthesizer according to the synthesis loss function; Input the curve structure library and the image background library into the trained image synthesizer for image synthesis and output the synthesized image set.

6. The image segmentation method based on a segmentation model according to claim 5, wherein The image synthesizer includes an encoder and a multi-layer perceptron; the inputting the intermediate data set and the original sample subset into the image synthesizer for multi-layer piece-by-piece contrast learning to obtain a synthesis loss function includes: Input the intermediate data set into the encoder for image downsampling to obtain a high-level feature map, and obtain patch information corresponding to the sample image according to the high-level feature map, where the patch information corresponds to the pixels in the high-level feature map; Extract features from the patch information through the encoder to obtain multi-scale feature information; Input the multi-scale feature information into the multi-layer perceptron for feature mapping and output a feature stack; Determine patch feature information and spatial position information according to the feature stack; Calculate the patch feature information and the spatial position information based on the preset noise contrast estimation method to obtain a synthetic objective function; Input the original sample subset into the encoder for sample selection to obtain patch features, negative samples, and positive samples; Obtain a regularization loss value according to the patch features, the negative samples, and the positive samples; Obtain the synthetic loss function according to the synthetic objective function, the regularization loss value, and a preset adversarial loss function.

7. The image segmentation method based on a segmentation model according to claim 1, wherein, The step of inputting the synthetic image set and the original sample subset into the segmentation model for training to obtain a target segmentation model includes: Input the synthetic image set into the segmentation model for training to obtain a rough segmenter, and input the original sample subset into the rough segmenter for segmentation prediction to obtain pseudo-label information; Determine the rough segmentation loss value of the rough segmenter according to a preset segmentation loss function, and determine the pseudo-label loss value according to the pseudo-label information; Determine a target loss function according to the rough segmentation loss value and the pseudo-label loss value; Train the segmentation model based on the target loss function, the synthetic image set, and the original sample subset to obtain a fine segmenter; Obtain the target segmentation model according to the rough segmenter and the fine segmenter.

8. An image segmentation system based on a segmentation model, characterized in that, The segmentation model includes an image inpainting model and an image synthesizer, and the system includes: A sample acquisition module for acquiring a sample image set, where the sample image set includes multiple sample images; A curve generation module for simulating the foreground of the sample image based on a preset space colonization algorithm to generate a curve structure library corresponding to the sample image; A skeleton annotation module for performing skeleton annotation on the sample image and dividing the sample image set according to the sample image after skeleton annotation to obtain an annotated sample subset and an original sample subset; A background extraction module for inputting the annotated sample subset into the image inpainting model for background extraction and outputting an image background library; An image synthesis module for inputting the curve structure library and the image background library into the image synthesizer for image synthesis and outputting a synthetic image set; A model training module for inputting the synthetic image set and the original sample subset into the segmentation model for training to obtain a target segmentation model; An image segmentation module for acquiring a sample image to be detected and inputting the sample image to be detected into the target segmentation model for image segmentation to obtain a segmentation result.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the image segmentation method based on a segmentation model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to execute the image segmentation method based on a segmentation model according to any one of claims 1 to 7.

Images