Training method and device of image segmentation model, electronic equipment and storage medium

Abstract

The application discloses a kind of training method, device, electronic equipment and storage medium of image segmentation model.The method comprises: obtaining multiple groups of training sample data, and training sample data includes sample image;In the iterative training process of image segmentation model, sample image is input to the encoder in image segmentation model, and the encoding feature image is obtained, the encoding feature image is input to the decoder in image segmentation model, and the first reconstruction image is obtained, the encoding feature image is input to the variational autoencoder in image segmentation model, and the second reconstruction image is obtained;Determine segmentation model loss based on the first reconstruction image, the second reconstruction image and sample image, adjust the parameter of current image segmentation model based on segmentation model loss, and obtain target image segmentation model.The above technical solution realizes self-supervised learning by increasing variational autoencoder, realizes unlabelled data training, reduces dependence on labeled data, and reduces data labeling cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a training method and device of an image segmentation model, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, automatic segmentation of lesions using medical image AI models is one of the important directions of intelligent medical development. An artificial intelligence scheme based on original images and labeled images for training and inference can save doctors' annotation time and reduce doctors' workload.

[0003] However, the training of a supervised learning AI model based on a neural network relies on a large amount of medical label data. Although a large amount of image data is generated in the medical field every day, the image data labeling cost is extremely high, and experienced imaging experts are needed to label a large number of medical images with high quality.

[0004] In the process of implementing the present application, the present application has found that the prior art has at least the following technical problems: the prior art has the problem of strong dependence of model training on labeled data. SUMMARY

[0005] The present application provides a training method and device of an image segmentation model, an electronic device and a storage medium, which reduces the dependence on labeled data.

[0006] According to an aspect of the present application, a training method of an image segmentation model is provided, comprising:

[0007] obtaining a plurality of sets of training sample data, wherein the training sample data comprises sample images;

[0008] In the iterative training process of the image segmentation model, the sample images are input into an encoder in the image segmentation model to obtain encoded feature images, the encoded feature images are input into a decoder in the image segmentation model to obtain first reconstructed images, and the encoded feature images are input into a variational autoencoder in the image segmentation model to obtain second reconstructed images;

[0009] determining a segmentation model loss based on the first reconstructed images, the second reconstructed images and the sample images, adjusting parameters of the current image segmentation model based on the segmentation model loss, and until the iterative training is completed to obtain a target image segmentation model.

[0010] According to another aspect of the present application, a training device of an image segmentation model is provided, comprising:

[0011] a sample data acquisition module configured to obtain a plurality of sets of training sample data, wherein the training sample data comprises sample images;

[0012] The reconstructed image generation module is configured to, in the iterative training process of the image segmentation model, input the sample image into an encoder in the image segmentation model to obtain an encoded feature image, input the encoded feature image into a decoder in the image segmentation model to obtain a first reconstructed image, and input the encoded feature image into a variational autoencoder in the image segmentation model to obtain a second reconstructed image.

[0013] The segmentation model loss determination module is configured to determine a segmentation model loss based on the first reconstructed image, the second reconstructed image, and the sample image, and adjust parameters of the current image segmentation model based on the segmentation model loss until the iterative training is completed to obtain a target image segmentation model.

[0014] According to an aspect of the present application, there is provided an image segmentation method, comprising:

[0015] obtaining at least one image to be segmented;

[0016] inputting the image to be segmented into the target image segmentation model according to any one of the embodiments of the present application to obtain a target segmented image corresponding to the image to be segmented.

[0017] According to another aspect of the present application, there is provided an electronic device, comprising:

[0018] at least one processor; and

[0019] a memory in communication connection with the at least one processor; wherein

[0020] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the training method of the image segmentation model or the image segmentation method according to any one of the embodiments of the present application.

[0021] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to implement the training method of the image segmentation model or the image segmentation method according to any one of the embodiments of the present application when executed by the processor.

[0022] The technical scheme of the embodiment of the present application comprises the following steps: obtaining multiple sets of training sample data, wherein the training sample data comprises sample images; in the iterative training process of the image segmentation model, inputting the sample images into an encoder in the image segmentation model to obtain encoded feature images, inputting the encoded feature images into a decoder in the image segmentation model to obtain first reconstructed images, and inputting the encoded feature images into a variational autoencoder in the image segmentation model to obtain second reconstructed images; determining a segmentation model loss based on the first reconstructed images, the second reconstructed images and the sample images, adjusting parameters of the current image segmentation model based on the segmentation model loss to obtain a target image segmentation model. The above technical scheme realizes self-supervised learning by adding a variational autoencoder, realizes training without labeled data, reduces the dependence on labeled data, and further reduces the data labeling cost.

[0023] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is a flowchart of a training method of an image segmentation model according to the first embodiment of the present application;

[0026] Figure 2 is a flowchart of a training method of an image segmentation model according to the second embodiment of the present application;

[0027] Figure 3 is a network structure diagram of an image segmentation model according to the second embodiment of the present application;

[0028] Figure 4 is a flowchart of an image segmentation method according to the third embodiment of the present application;

[0029] Figure 5 is a structural schematic diagram of a training device of an image segmentation model according to the fourth embodiment of the present application;

[0030] Figure 6 is a structural schematic diagram of an electronic device for implementing the training method of the image segmentation model of the present application. DETAILED DESCRIPTION

[0031] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort should belong to the scope of the present application.

[0032] It should be noted that the terms "first", "second", and the like in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0033] Embodiment one

[0034] Figure 1 A flowchart of a training method of an image segmentation model provided by the first embodiment of the present application is shown in the figure. The present embodiment can be applied to the case of training an image segmentation model with a small amount of training sample data. The method can be performed by a training device of an image segmentation model, which can be realized in the form of hardware and / or software, and can be configured in a computer terminal. As shown in the figure, the method comprises: Figure 1

[0035] S110, obtaining a plurality of sets of training sample data, wherein the training sample data comprises sample images.

[0036] The training sample data can be understood as a set of sample images used for model training.

[0037] For example, the training sample data can be pre-stored in the local electronic device, other storage devices associated with the electronic device, or the cloud, and can be directly obtained from the corresponding storage location when needed.

[0038] In an optional implementation of the present embodiment, obtaining a plurality of sets of training sample data comprises: obtaining at least one set of original sample data, wherein the original sample data comprises sample images; and performing data augmentation on the original sample data to obtain a plurality of sets of training sample data. ​

[0039] The original sample data refers to an unprocessed original image.

[0040] It can be understood that, by performing data augmentation on the original sample data, the number of training samples can be increased, providing a variety of training samples for model training, thereby improving the generalization ability of the model. For example, the data augmentation method can be offline augmentation, which can be achieved by image processing methods such as flipping, rotating, cropping, scaling, shifting, and masking. For example, the data augmentation method can be Random Erasing, Mixup, etc.

[0041] S120, in the iterative training process of the image segmentation model, input the sample image into the encoder in the image segmentation model to obtain an encoded feature image, input the encoded feature image into the decoder in the image segmentation model to obtain a first reconstructed image, and input the encoded feature image into the variational autoencoder in the image segmentation model to obtain a second reconstructed image.

[0042] In this embodiment, the image segmentation model to be trained includes an encoder, a decoder, and a variational autoencoder.

[0043] Specifically, the sample image is input as input data into the encoder in the image segmentation model to be trained to obtain an encoded feature image. After obtaining the encoded feature image, the encoded feature image can be input into the decoder and the variational autoencoder, respectively, to obtain a first reconstructed image and a second reconstructed image. The image segmentation model can be trained in a self-supervised manner through the first reconstructed image, the second reconstructed image, and the sample image, which can realize training without labeled data, reduce the dependence on labeled data, and reduce the cost of data labeling.

[0044] In an optional embodiment of this embodiment, the encoder and the decoder in the image segmentation model are U-shaped network structures.

[0045] The U-shaped network structure refers to a network structure based on U-Net. In other words, the encoder in the image segmentation model is composed of a convolution module and a down-sampling module, and the decoder is composed of a convolution module, a skip connection module, and an up-sampling module.

[0046] S130, determining a segmentation model loss based on the first reconstructed image, the second reconstructed image, and the sample image, adjusting the parameters of the current image segmentation model based on the segmentation model loss, until the iterative training is completed to obtain a target image segmentation model.

[0047] The segmentation model loss refers to the loss of the current image segmentation model.

[0048] Specifically, the image segmentation model can be obtained by training a plurality of sets of training sample data in advance. In the iterative training process of the image segmentation model, the sample image is pre-encoded, decoded, and the like to obtain a first reconstructed image and a second reconstructed image, the segmentation model loss is determined according to the first reconstructed image, the second reconstructed image and the sample image, and then the parameters of the current image segmentation model are adjusted according to the segmentation model loss, so as to realize the self-supervised training of the image segmentation model.

[0049] In an optional implementation of the embodiment, after obtaining the target image segmentation model, the method further includes: removing the variational autoencoder from the target image segmentation model to obtain an image segmentation removal model; obtaining a plurality of sets of labeled training samples, wherein the labeled training samples include sample images and sample target segmentation images corresponding to the sample images; training the image segmentation removal model based on the plurality of sets of labeled training samples to obtain a fine-tuned target image segmentation model.

[0050] It should be noted that the variational autoencoder is removed from the target image segmentation model in the embodiment to obtain the image segmentation removal model, but the image segmentation removal model retains the model parameters of the target image segmentation model. In other words, the image segmentation removal model refers to a simplified model that removes the variational autoencoder and retains the model parameters. Further, the image segmentation removal model is supervised trained by the labeled training samples to adjust the model parameters, and a target image segmentation model with fine-tuned model parameters is obtained, so that the target image segmentation model has higher precision.

[0051] The technical scheme of the embodiment of the application includes the following steps: obtaining a plurality of sets of training sample data, the training sample data including sample images; in the iterative training process of the image segmentation model, inputting the sample image into the encoder in the image segmentation model to obtain an encoded feature image, inputting the encoded feature image into the decoder in the image segmentation model to obtain a first reconstructed image, and inputting the encoded feature image into the variational autoencoder in the image segmentation model to obtain a second reconstructed image; determining a segmentation model loss based on the first reconstructed image, the second reconstructed image and the sample image, adjusting the parameters of the current image segmentation model based on the segmentation model loss, and obtaining a target image segmentation model. The above technical scheme realizes self-supervised learning by adding a variational autoencoder, realizes training without labeled data, reduces the dependence on labeled data, and further reduces the data labeling cost.

[0052] Embodiment two

[0053] Figure 2A flowchart of a training method of an image segmentation model is provided for the second embodiment of the present application. The method of the present embodiment can be combined with the various optional schemes of the training method of the image segmentation model provided in the above embodiments. The training method of the image segmentation model provided in the present embodiment is further optimized. Optionally, determining the segmentation model loss based on the first reconstructed image, the second reconstructed image and the sample image comprises: inputting the first reconstructed image, the second reconstructed image and the sample image into a loss function to obtain the segmentation model loss; wherein the loss function comprises a mean square error loss term, a divergence loss term and a contrast loss term.

[0054] As shown in Figure 2 , the method comprises:

[0055] S210, obtaining a plurality of sets of training sample data, wherein the training sample data comprises a sample image.

[0056] S220, in the iterative training process of the image segmentation model, inputting the sample image into an encoder in the image segmentation model to obtain an encoded feature image, inputting the encoded feature image into a decoder in the image segmentation model to obtain a first reconstructed image, and inputting the encoded feature image into a variational autoencoder in the image segmentation model to obtain a second reconstructed image.

[0057] S230, inputting the first reconstructed image, the second reconstructed image and the sample image into a loss function to obtain a segmentation model loss, wherein the loss function comprises a mean square error loss term, a divergence loss term and a contrast loss term.

[0058] The mean square error loss term refers to the mean square error loss function of the first reconstructed image, the second reconstructed image and the sample image. The divergence loss term refers to the loss function for calculating the Kullback-Leibler divergence, wherein the Kullback-Leibler divergence is also known as relative entropy. The contrast loss term refers to a loss function that can perform difficult negative sample self-discovery, and can be used to improve the self-supervised learning quality of the model. In the present embodiment, the image segmentation model is trained through the above multiple loss functions, which can effectively improve the accuracy of model training.

[0059] In an optional implementation of the present embodiment, the loss function comprises:

[0060] Loss=αL i,j +βL KL +(1-α-β)L contrastive

[0061] L i,j =L1(X,X i )+L2(X,Xj )

[0062]

[0063]

[0064] wherein, L i,j represents a mean square error loss term, L KL represents a divergence loss term, L contrastive represents a contrast loss term, and α, β represent weight coefficients of each loss term; L1 represents a first mean square error loss term, L2 represents a second mean square error loss term, X represents a sample image, X i represents an i-th first reconstructed image, X j represents a j-th second reconstructed image; L KL is a Kullback-Leibler divergence, as a penalty term of the variational autoencoder, so that the hidden layer features of the image segmentation model are close to a normal distribution, n represents a total number of voxels of the sample image, N(σ, μ) represents a distribution of the encoding feature image, N std represents a standard normal distribution; z i represents a feature vector of X i , z j represents a feature vector of X j , z k represents a feature vector other than z i , cos<z i ,z j > represents a cosine similarity of z i and z j , Ι [k≠i] represents an index function with a value range of {0, 1}, τ represents a temperature coefficient, and N represents a number of sample images.

[0065] S240, adjusting parameters of the current image segmentation model based on the segmentation model loss until an iterative training is completed to obtain a target image segmentation model.

[0066] Exemplarily, Figure 3is a network structure diagram of an image segmentation model provided by the embodiment. The sample image 1 can be a breast cancer MRI (magnetic resonance imaging) image. After obtaining the breast cancer MRI image, data enhancement can be performed on the breast cancer MRI image, and the embodiment can adopt a random padding patch mask manner to perform data enhancement on the same breast cancer MRI image to obtain two different images. The enhanced breast cancer MRI image 2 is sent to a neural network with a U-Net architecture of a variational autoencoder (VAE) branch. The sample image 1 is subjected to multiple times of downsampling to obtain an encoded feature image 3; the encoder branch finally outputs a first reconstructed image 4; the beginning of the VAE branch accesses a hidden layer with high-level semantics of the U-Net subjected to multiple times of downsampling, and the distribution expectation after convolution finally approximates a prior normal distribution to obtain a second reconstructed image 5. The embodiment utilizes the original sample image 1 and the output results of the two branches, the first reconstructed image 4 and the second reconstructed image 5, to construct a reconstruction proxy task and a contrast learning task and perform model training of the neural network, and the loss function is:

[0067] Loss=αL i,j +βL KL +(1-α-β)L contrastive .

[0068] Further, the VAE branch is removed from the target image segmentation model to obtain an image segmentation removal model, and the model parameters are reused; further, a plurality of sets of labeled training samples are obtained, the image segmentation removal model is trained based on supervised learning according to the plurality of sets of labeled training samples, and the model is fine-tuned by taking Dice Loss as a target function of breast cancer lesion segmentation to obtain a fine-tuned target image segmentation model, so as to further improve the training accuracy of the model. In the prediction stage, the model parameters of the fine-tuned target image segmentation model are kept unchanged, the breast cancer MRI image is sent to the fine-tuned target image segmentation model, and a breast cancer lesion segmentation image is obtained.

[0069] The technical scheme of the embodiment of the application, by obtaining a plurality of sets of training sample data, the training sample data including a sample image; in the iterative training process of the image segmentation model, the sample image is input into an encoder in the image segmentation model to obtain an encoded feature image, the encoded feature image is input into a decoder in the image segmentation model to obtain a first reconstructed image, and the encoded feature image is input into a variational autoencoder in the image segmentation model to obtain a second reconstructed image; the first reconstructed image, the second reconstructed image and the sample image are input into a loss function to obtain a segmentation model loss, wherein the loss function includes a mean square error loss term, a divergence loss term and a contrast loss term, the parameters of the current image segmentation model are adjusted based on the segmentation model loss to obtain a target image segmentation model. The above technical scheme can effectively improve the accuracy of model training by training the image segmentation model through the above loss function.

[0070] Embodiment three

[0071] Figure 4 A flowchart of an image segmentation method provided in Embodiment three of the present application. The method of the present embodiment can be combined with the various optional schemes in the training method of the image segmentation model provided in the above embodiments. As shown in the flowchart, the method comprises the following steps. Figure 4

[0072] S310, obtaining at least one image to be segmented.

[0073] S320, inputting the image to be segmented into a target image segmentation model to obtain a target segmentation image corresponding to the image to be segmented.

[0074] The image to be segmented refers to an image to be subjected to image segmentation processing. For example, the image to be segmented can be a breast cancer MRI image. The target segmentation image refers to a segmented image. For example, the target segmentation image can be a breast cancer lesion segmentation image.

[0075] The target image segmentation model in the present embodiment can be obtained by the training method of the image segmentation model in the above embodiments, and the specific training process will not be repeated here.

[0076] For example, in the prediction stage, the breast cancer MRI image is input into the fine-tuned target image segmentation model to obtain a breast cancer lesion segmentation image. It can be understood that the fine-tuned target image segmentation model has better feature extraction capability, thereby improving the segmentation accuracy of the breast cancer lesion segmentation image.

[0077] The technical solution of the present embodiment inputs the image to be segmented into a target image segmentation model with better feature extraction capability to obtain a more accurate target segmentation image.

[0078] Embodiment four

[0079] Figure 5 A structural schematic diagram of a training device of an image segmentation model provided in Embodiment four of the present application. As shown in the structural schematic diagram, the device comprises the following modules. Figure 5

[0080] A sample data acquisition module 410 is configured to acquire a plurality of groups of training sample data, wherein the training sample data comprises sample images.

[0081] ​​The reconstruction image generation module 420 is configured to, in the iterative training process of the image segmentation model, input the sample image into an encoder in the image segmentation model to obtain an encoded feature image, input the encoded feature image into a decoder in the image segmentation model to obtain a first reconstruction image, and input the encoded feature image into a variational autoencoder in the image segmentation model to obtain a second reconstruction image.

[0082] The segmentation model loss determination module 430 is configured to determine a segmentation model loss based on the first reconstruction image, the second reconstruction image and the sample image, and adjust parameters of a current image segmentation model based on the segmentation model loss until an iterative training is completed to obtain a target image segmentation model.

[0083] The technical scheme of the embodiment of the application comprises the following steps: obtaining a plurality of groups of training sample data, the training sample data comprising sample images; in an iterative training process of an image segmentation model, inputting a sample image into an encoder in the image segmentation model to obtain an encoded feature image, inputting the encoded feature image into a decoder in the image segmentation model to obtain a first reconstruction image, and inputting the encoded feature image into a variational autoencoder in the image segmentation model to obtain a second reconstruction image; determining a segmentation model loss based on the first reconstruction image, the second reconstruction image and the sample image, and adjusting parameters of a current image segmentation model based on the segmentation model loss to obtain a target image segmentation model. The above technical scheme realizes self-supervised learning by adding a variational autoencoder, realizes training without labeled data, reduces the dependence on labeled data, and further reduces the data labeling cost.

[0084] In some optional embodiments, the sample data acquisition module 410 is specifically configured to:

[0085] acquire at least one group of original sample data, wherein the original sample data comprises sample images;

[0086] perform data enhancement on the original sample data to obtain a plurality of groups of training sample data.

[0087] In some optional embodiments, the segmentation model loss determination module 430 comprises:

[0088] a segmentation model loss determination unit configured to input the first reconstruction image, the second reconstruction image and the sample image into a loss function to obtain a segmentation model loss;

[0089] wherein the loss function comprises a mean square error loss term, a divergence loss term and a contrastive loss term.

[0090] In some optional embodiments, the loss function comprises:

[0091] Loss = αLi,j + βL KL + (1 - α - β)L contrastive

[0092] L i,j = L1(X, X i ) + L2(X, X j )

[0093]

[0094]

[0095] wherein, L i,j represents a mean square error loss term, L KL represents a divergence loss term, L contrastive represents a contrast loss term, and α and β represent weight coefficients of each loss term; L1 represents a first mean square error loss term, L2 represents a second mean square error loss term, X represents a sample image, X i represents an i-th first reconstructed image, and X j represents an j-th second reconstructed image; L KL is a Kullback-Leibler divergence, which is a penalty term of the variational autoencoder, so that the hidden layer features of the image segmentation model are close to a normal distribution, n represents a total number of voxels of the sample image, N(σ, μ) represents a distribution of the encoded feature image, and N std represents a standard normal distribution; z i represents a feature vector of X i , z j represents a feature vector of X j , z k represents a feature vector other than z i , and cos<z i ,z j > represents a cosine similarity of z i and z j , Ι [k≠i] represents an index function with a value range of {0, 1}, τ represents a temperature coefficient, and N represents a number of sample images.

[0096] In some optional embodiments, the apparatus further includes:

[0097] a model pruning module configured to prune the variational autoencoder from the target image segmentation model to obtain an image segmentation pruning model;

[0098] a labeled sample acquisition module configured to acquire a plurality of groups of labeled training samples, wherein the labeled training samples include sample images and sample target segmentation images corresponding to the sample images;

[0099] The labeling sample training module is configured to train the image segmentation pruning model based on the multiple sets of labeled training samples, and obtain a fine-tuned target image segmentation model.

[0100] In some optional embodiments, the encoder and the decoder in the image segmentation model are in a U-shaped network structure.

[0101] The training device of the image segmentation model provided in the embodiments of the present application can perform the training method of the image segmentation model provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0102] Embodiment five

[0103] Figure 6 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0104] As shown in Figure 6 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0105] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a loudspeaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0106] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as a training method of an image segmentation model, which includes:

[0107] obtaining a plurality of sets of training sample data, wherein the training sample data includes sample images;

[0108] In an iterative training process of the image segmentation model, inputting the sample images into an encoder in the image segmentation model to obtain encoded feature images, inputting the encoded feature images into a decoder in the image segmentation model to obtain first reconstructed images, inputting the encoded feature images into a variational autoencoder in the image segmentation model to obtain second reconstructed images;

[0109] determining a segmentation model loss based on the first reconstructed images, the second reconstructed images, and the sample images, adjusting parameters of a current image segmentation model based on the segmentation model loss until an iterative training is completed to obtain a target image segmentation model.

[0110] In some embodiments, the training method of the image segmentation model can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the training method of the image segmentation model described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the training method of the image segmentation model by any other appropriate means, such as by means of firmware.

[0111] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0112] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented on the computer or other programmable apparatus. The computer programs can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0113] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0114] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0115] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0116] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0117] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0118] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and principles of the disclosure. Accordingly, the disclosure is not limited to the specific embodiments described above, but only by the scope of the appended claims.

Claims

1. A method for training an image segmentation model, characterized in that, The method comprises: obtaining a plurality of sets of training sample data, wherein the training sample data comprises sample images; in the iterative training process of the image segmentation model, inputting the sample images into an encoder in the image segmentation model to obtain encoded feature images, inputting the encoded feature images into a decoder in the image segmentation model to obtain first reconstructed images, and inputting the encoded feature images into a variational autoencoder in the image segmentation model to obtain second reconstructed images; determining a segmentation model loss based on the first reconstructed images, the second reconstructed images and the sample images, adjusting parameters of a current image segmentation model based on the segmentation model loss, and until iterative training is completed to obtain a target image segmentation model; the determination of the segmentation model loss based on the first reconstructed images, the second reconstructed images and the sample images comprises: inputting the first reconstructed images, the second reconstructed images and the sample images into a loss function to obtain a segmentation model loss; wherein the loss function comprises a mean square error loss term, a divergence loss term and a contrast loss term; and the mean square error loss term is a mean square error loss function of the first reconstructed images, the second reconstructed images and the sample images.

2. The method of claim 1, wherein, The obtaining of the plurality of sets of training sample data comprises: obtaining at least one set of original sample data, wherein the original sample data comprises sample images; performing data enhancement on the original sample data to obtain a plurality of sets of training sample data.

3. The method of claim 1, wherein, The loss function comprises: Loss = aL i,j + βL KL + (1 - a - β)L contrastive L i,j = L1(X, X i ) + L2(X, X j ) wherein, L i,j represents a mean square error loss term, L KL represents a divergence loss term, L contrastive represents a contrast loss term, and α and β represent weight coefficients of each loss term; L1 represents a first mean square error loss term, L2 represents a second mean square error loss term, X represents a sample image, X i represents an i-th first reconstructed image, X j represents a j-th second reconstructed image; L KL is a Kullback-Leibler divergence, as a penalty term of the variational autoencoder, so that the hidden layer features of the image segmentation model are close to a normal distribution, n represents a total number of voxels of the sample image, N(σ, μ) represents a distribution of the encoding feature image, N std represents a standard normal distribution; z i represents a feature vector of X i , z j represents a feature vector of X j , z k represents a feature vector other than z i , cos<z i ,z j > represents a cosine similarity of z i and z j , I [k≠i] represents an index function with a value range of {0, 1}, τ represents a temperature coefficient, and N represents a number of sample images.

4. The method of claim 1, wherein, after obtaining the target image segmentation model, the method further comprises: eliminating the variational autoencoder from the target image segmentation model to obtain an image segmentation elimination model; obtaining a plurality of sets of labeled training samples, wherein the labeled training samples comprise sample images and sample target segmentation images corresponding to the sample images; training the image segmentation elimination model based on the plurality of sets of labeled training samples to obtain a fine-tuned target image segmentation model.

5. The method of claim 1, wherein, The encoder and the decoder in the image segmentation model are U-shaped network structures.

6. An image segmentation method characterized by, The method comprises: obtaining at least one image to be segmented; inputting the image to be segmented into the target image segmentation model of any one of claims 1-5 to obtain a target segmentation image corresponding to the image to be segmented.

7. A training device for an image segmentation model, characterized in that, The method comprises: a sample data obtaining module configured to obtain a plurality of sets of training sample data, wherein the training sample data comprises sample images; a reconstructed image generating module configured to, in an iterative training process of an image segmentation model, input the sample images into an encoder in the image segmentation model to obtain encoded feature images, input the encoded feature images into a decoder in the image segmentation model to obtain first reconstructed images, and input the encoded feature images into a variational autoencoder in the image segmentation model to obtain second reconstructed images; a segmentation model loss determining module configured to determine a segmentation model loss based on the first reconstructed images, the second reconstructed images and the sample images, adjust parameters of a current image segmentation model based on the segmentation model loss, and until iterative training is completed to obtain a target image segmentation model; The segmentation model loss determination module comprises: a segmentation model loss determination unit configured to input the first reconstructed image, the second reconstructed image and the sample image into a loss function to obtain a segmentation model loss; wherein the loss function comprises a mean square error loss term, a divergence loss term and a contrast loss term; and the mean square error loss term is a mean square error loss function of the first reconstructed image, the second reconstructed image and the sample image.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image segmentation model training method of any one of claims 1-5 or the image segmentation method of claim 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the image segmentation model training method of any one of claims 1-5 or the image segmentation method of claim 6 when executed.

Images