Image segmentation method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide an image segmentation method and device, electronic equipment and storage medium, belonging to the field of image processing and artificial intelligence. The method comprises: acquiring an object labeled image with sample label information; based on a preset initial image segmentation model, performing image coding processing on the object labeled image to obtain a target feature image; based on the object labeled image, performing random noise injection on the target feature image to obtain a noise image; performing self-supervised image denoising on the noise image to obtain a predicted segmentation image; based on the predicted segmentation image, the sample segmentation object and the sample label information, adjusting parameters of the initial image segmentation model to obtain a target image segmentation model; and based on the target image segmentation model, performing image segmentation on a preset target image to obtain a target segmentation image. The embodiments of the present application can improve the clarity of the segmentation image.

Description

Technical Field

[0001] The present application relates to the field of image processing and artificial intelligence technology, and in particular to an image segmentation method and device, an electronic device, and a storage medium. Background Art

[0002] Image segmentation refers to separating a specified area in an image from the background and other areas. For example, in medical scenarios, by performing image segmentation on the area where the organs are located in a medical image, organ images for disease diagnosis and treatment can be obtained.

[0003] Currently, the most common image segmentation method is to use convolutional neural networks to segment the target image. However, the target image is easily affected by factors such as noise, artifacts, low contrast, and uneven lighting, resulting in poor image quality and blurry segmented images. Therefore, how to improve the clarity of the segmented image has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide an image segmentation method and device, an electronic device, and a storage medium, aiming to improve the clarity of the segmented image.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides an image segmentation method, the method comprising:

[0006] Acquire an object annotated image having sample annotation information, wherein the object annotated image includes a sample object region containing the sample segmented object and a sample background region not containing the sample segmented object, and the sample annotation information is used to annotate the sample object region;

[0007] Based on a preset initial image segmentation model, image coding processing is performed on the object annotation image to obtain a target feature image;

[0008] Based on the object annotation image, random noise is injected into the target feature image to obtain a noise image;

[0009] Performing self-supervised image denoising on the noisy image to obtain a predicted segmented image;

[0010] Based on the predicted segmented image, the sample segmentation object and the sample annotation information, adjusting parameters of the initial image segmentation model to obtain a target image segmentation model;

[0011] Image segmentation is performed on a preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0012] In some embodiments, performing self-supervised image denoising on the noisy image to obtain a predicted segmented image includes:

[0013] Performing feature extraction on the noise image to obtain a noise feature vector;

[0014] Performing contextual understanding on the noise feature vector to obtain a noise attention vector;

[0015] Performing feature modification on the noise attention vector to obtain a predicted latent vector;

[0016] The predicted segmented image is predicted based on the predicted latent vector and the noise feature vector.

[0017] In some embodiments, performing feature modification on the noise attention vector to obtain a predicted latent vector includes:

[0018] Performing feature reconstruction on the noise attention vector to obtain a reconstructed feature vector;

[0019] Feature amplification is performed on the reconstructed feature vector to obtain the predicted latent vector.

[0020] In some embodiments, predicting the predicted segmented image based on the predicted latent vector and the noise feature vector includes:

[0021] Performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature;

[0022] Feature activation is performed on the fused features to obtain the predicted segmentation image.

[0023] In some embodiments, injecting random noise into the target feature image based on the object annotation image to obtain a noise image includes:

[0024] Based on the object annotated image, a target noise sequence is screened out from a preset noise data pool, wherein the target noise sequence includes target noise;

[0025] Scaling the target noise to obtain random noise;

[0026] Based on the random noise, the target feature image is subjected to noise addition processing to obtain the noise image.

[0027] In some embodiments, adjusting parameters of the initial image segmentation model based on the predicted segmented image, the sample segmentation object, and the sample annotation information to obtain a target image segmentation model includes:

[0028] Determining the sample segmentation area of ​​the object annotation image based on the sample segmentation object and the sample annotation information;

[0029] Based on the sample segmentation area and the predicted segmentation image, performing loss calculation on the initial image segmentation model to obtain object segmentation loss data;

[0030] Based on the object segmentation loss data, the parameters of the initial image segmentation model are updated to obtain the target image segmentation model.

[0031] In some embodiments, performing image encoding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image includes:

[0032] performing standardization processing on the object annotated image to obtain a standard image;

[0033] Based on the initial image segmentation model, convolution processing is performed on the object annotation image to obtain an initial feature image;

[0034] Pooling is performed on the initial feature image to obtain the target feature image.

[0035] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an image segmentation device, comprising:

[0036] a labeled image acquisition module, configured to acquire an object labeled image having sample labeling information, wherein the object labeled image includes a sample object region containing the sample segmented object and a sample background region not containing the sample segmented object, and the sample labeling information is used to label the sample object region;

[0037] Annotated image coding module, used for performing image coding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image;

[0038] A feature image denoising module, configured to inject random noise into the target feature image based on the object annotation image to obtain a noise image;

[0039] a noise image denoising module, configured to perform self-supervised image denoising on the noise image to obtain a predicted segmented image;

[0040] An initial model optimization module is used to adjust the parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model;

[0041] A target image segmentation module is used to perform image segmentation on a preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image contains the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0042] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0043] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect.

[0044] The image segmentation method and device, electronic device and storage medium proposed in the present application obtain an object labeled image by acquiring an image with sample annotation information, and then using a preset initial image segmentation model to perform image encoding processing on the object labeled image to obtain a target feature image, and based on the object labeled image, inject random noise into the target feature image, so that the object labeled image can be closer to the image information under the influence of various noises in real life, thereby improving the accuracy of image segmentation. Secondly, self-supervised image denoising is performed on the noise image, and the parameters of the initial image segmentation model are adjusted according to the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model, thereby improving the accuracy of the model in processing image segmentation tasks. Finally, the target image segmentation model is used to perform image segmentation on the preset target image to obtain a target segmented image, thereby improving the clarity of the segmented image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of the image segmentation method provided in an embodiment of the present application;

[0046] Figure 2 yes Figure 1 Flowchart of step S102 in FIG.

[0047] Figure 3 yes Figure 1 Flowchart of step S103 in FIG.

[0048] Figure 4 yes Figure 1 Flowchart of step S104 in FIG.

[0049] Figure 5 yes Figure 4 Flowchart of step S403 in FIG.

[0050] Figure 6 yes Figure 4 Flowchart of step S404 in FIG.

[0051] Figure 7 yes Figure 1 Flowchart of step S105 in FIG.

[0052] Figure 8 Schematic diagram of the structure of the image segmentation device provided in an embodiment of the present application;

[0053] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0057] First, let’s analyze some of the terms used in this application:

[0058] Diffusion model: Diffusion model is a generative model that learns the data distribution by simulating the process of gradually adding and removing noise from the data. During the training phase, the model gradually transforms the real data into pure noise and denoises it step by step through the inverse process to generate new samples. In the generation phase, starting from random noise, the model applies the inverse diffusion process to gradually recover the data structure, thus generating new samples similar to the training data. Diffusion models perform well in image generation, image inpainting, and super-resolution tasks.

[0059] Attention mechanism: Attention mechanism is a computational method that mimics human visual attention, aiming to enable the model to dynamically focus on the most relevant information when processing input data, thereby improving performance. Its benefits include effectively capturing long-range dependencies and important features, significantly improving the accuracy and efficiency of natural language processing, image recognition, and translation tasks. By adaptively adjusting the focus, attention mechanism enhances the model's expressive power, making it perform better in complex tasks.

[0060] Multi-head attention mechanism: Multi-head attention mechanism is a deep learning technique used to simultaneously focus on different parts of the input when processing sequential data. It divides the input features into multiple subspaces, each of which independently calculates attention weights to capture different contextual information. Finally, the outputs of these subspaces are concatenated or weighted combined to form a rich feature representation. Multi-head attention mechanism is widely used in natural language processing and computer vision, enhancing the model's understanding of complex dependencies.

[0061] Image segmentation refers to the process of separating a specified region in an image from the background and other regions. For example, in the medical field, by performing image segmentation on the region where the organ is located in the medical image, an organ image for disease diagnosis and treatment can be obtained.

[0062] Currently, common image segmentation methods usually use convolutional neural networks to segment target images. However, target images are easily affected by noise, artifacts, low contrast, uneven lighting, and other factors, resulting in low image quality and blurred segmented images. Therefore, how to improve the clarity of segmented images has become a technical problem to be solved.

[0063] Therefore, the embodiments of the present application provide an image segmentation method and device, electronic equipment and storage medium, which aims to improve the clarity of the segmented image.

[0064] The image segmentation method and device, electronic equipment and storage medium provided by the embodiments of the present application are specifically explained by the following embodiments. First, the image segmentation method in the embodiments of the present application is described.

[0065] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0066] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0067] The image segmentation method provided in the embodiment of the present application relates to the field of image processing and artificial intelligence technology. The image segmentation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the image segmentation method, etc., but is not limited to the above forms.

[0068] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0069] Figure 1 This is an optional flowchart of the image segmentation method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0070] Step S101, obtaining an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing the sample segmentation object, and the sample annotation information is used to annotate the sample object region;

[0071] Step S102: performing image coding processing on the object annotation image based on a preset initial image segmentation model to obtain a target feature image;

[0072] Step S103, based on the object annotation image, random noise is injected into the target feature image to obtain a noise image;

[0073] Step S104, performing self-supervised image denoising on the noisy image to obtain a predicted segmented image;

[0074] Step S105, adjusting the parameters of the initial image segmentation model based on the predicted segmented image, the sample segmentation object, and the sample annotation information to obtain a target image segmentation model;

[0075] Step S106, performing image segmentation on the preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing a target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0076] In steps S101 to S108 illustrated in the embodiment of the present application, an object annotated image is obtained by acquiring an image including a sample object area containing a sample segmentation object and a sample background area not containing the sample segmentation object, wherein the sample annotation information is used to annotate the sample object area, and then the object annotated image is subjected to image encoding processing using a preset initial image segmentation model to obtain a target feature image. Furthermore, random noise is injected into the target feature image based on the object annotated image to obtain a noise image. Secondly, self-supervised image denoising is performed on the noise image to obtain a predicted segmentation image, and parameters of the initial image segmentation model are adjusted based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model. Finally, image segmentation is performed on a preset target image including a target area containing a target object and a target background area not containing a target object according to the target image segmentation model to obtain a target segmentation image containing the target object and target annotation information, wherein the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target annotation information is used to annotate the target object. Therefore, the present application obtains an object annotation image by acquiring an image with sample annotation information, and then uses a preset initial image segmentation model to perform image encoding processing on the object annotation image to obtain a target feature image, and injects random noise into the target feature image based on the object annotation image, so that the object annotation image can be closer to the image information under the influence of various noises in real life, thereby improving the accuracy of image segmentation. Secondly, self-supervised image denoising is performed on the noise image, and the parameters of the initial image segmentation model are adjusted according to the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model, thereby improving the accuracy of the model in processing image segmentation tasks. Finally, the target image segmentation model is used to perform image segmentation on the preset target image to obtain a target segmented image, thereby improving the clarity of the segmented image.

[0077] In step S101 of some embodiments, the sample segmentation object may be an object to be segmented. For example, if an organ in a medical image needs to be segmented, the sample segmentation object may be a lung, kidney, heart, etc. in the medical image.

[0078] It should be noted that object-annotated images may include but are not limited to image data such as medical images and product images.

[0079] The present application can obtain an object-annotated image by performing target annotation on an unannotated initial image. Specifically, it is first necessary to perform target recognition on the initial image to identify object areas in the initial image that may be target objects, namely, suspected object areas. Secondly, the suspected object areas are screened to obtain sample object areas. Then, the non-sample object areas in the initial image are annotated with sample background areas, and information is annotated on the sample object areas to obtain sample segmented objects. Finally, the initial image with sample annotation information is called an object-annotated image.

[0080] In step S102 of some embodiments, the initial image segmentation model may be a diffusion model with set initial parameters and network structure, and the initial image segmentation model includes an encoder and a decoder.

[0081] The present application can use the encoder in the initial image segmentation model to encode the object annotation image obtained above to obtain a target feature image containing image features. Specifically, the encoding processing of the object annotation image by the initial image segmentation model includes but is not limited to image scaling, image cropping, image normalization, image data enhancement, and convolution pooling operations.

[0082] For details, see Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0083] Step S201, performing standardization processing on the object annotated image to obtain a standard image;

[0084] Step S202: performing convolution processing on the object annotation image based on the initial image segmentation model to obtain an initial feature image;

[0085] Step S203: performing pooling processing on the initial feature image to obtain a target feature image.

[0086] In step S201 of some embodiments, the object-annotated image may be standardized by performing a series of processes such as image scaling, image cropping, image normalization, and image data enhancement on the object-annotated image to obtain a standard image.

[0087] In step S202 of some embodiments, the object annotation image is convolved using the convolutional layer of the convolutional neural network of the encoding layer in the initial image segmentation model to obtain an initial feature image containing feature information.

[0088] In step S203 of some embodiments, the initial feature image is pooled using the pooling layer of the convolutional neural network of the encoding layer to obtain a reduced-dimensional initial feature image, that is, a target feature image.

[0089] In steps S201 to S203 shown in this embodiment, the object annotated image is standardized to obtain a standard image, and then the object annotated image is convolved according to the initial image segmentation model to obtain an initial feature image. The initial feature image is pooled, which can extract image feature information from the object annotated image and reduce the difficulty of the model in calculating the image feature information in the object annotated image, thereby improving the efficiency of image segmentation.

[0090] In step S103 of some embodiments, after obtaining the target feature image, it is necessary to determine the type of noise to be added based on the object annotation image, and then construct a target noise sequence based on the selected noise type. Furthermore, the target feature image is denoised using the noise in the target noise sequence to obtain a noise image containing noise data.

[0091] For details, see Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S303:

[0092] Step S301: based on the object annotated image, a target noise sequence is screened from a preset noise data pool, wherein the target noise sequence includes target noise;

[0093] Step S302, scaling the target noise to obtain random noise;

[0094] Step S303 : performing noise processing on the target feature image based on random noise to obtain a noise image.

[0095] In step S301 of some embodiments, the noise data pool is a pre-built data warehouse for storing noise data of various types, such as Gaussian noise, white noise, etc. The target noise sequence is composed of multiple target noises.

[0096] This application can annotate an image according to an object, clarify the noise type of noise data that needs to be added, and then filter the target noise from a pre-built noise data pool according to the noise type, and sort the filtered target noise to obtain a target noise sequence.

[0097] It is important to note that the amount of target noise is the same as the number of times noise is added to the target feature image.

[0098] In step S302 of some embodiments, before adding noise to the target feature image using the target noise, in order to improve robustness, the target noise can be scaled to obtain random noise. Specifically, a random vector that obeys a standard normal distribution is randomly generated, and then the random vector is multiplied by the square of the variance of the target noise to obtain random noise.

[0099] In step S303 of some embodiments, a noise image can be obtained by performing additive merging processing on the random noise and the target feature image.

[0100] It should be noted that the noise addition process is a multi-cycle process, and the number of cycles is the same as the amount of target noise minus one. Therefore, the present application can use the following formula to express the process of noise addition on the target feature image:

[0101] x t+1 =(1-z) 2 x t +(z) 2 s

[0102] Among them, x represents the state of the target feature image after noise processing, t+1 represents the next moment after time t, and x t+1 represents the state of the target feature image at time t+1, z represents the variance of the target noise, x t Represents the state of the target feature image at time t, and s represents the above random vector.

[0103] In steps S301 to S303 shown in this embodiment, first, a target noise sequence is screened from a preset noise data pool based on the sample segmentation object, which can ensure the accuracy of the added target noise. Secondly, the target noise is scaled to obtain random noise, which can improve the randomness of the noise image. Finally, based on the random noise, the target feature image is noised to obtain a noise image, so that the noise image can express the state of the initial image that can be encountered in practice, reducing the number of initial image samples required.

[0104] In step S104 of some embodiments, the decoder in the initial image segmentation model may be used to perform denoising on the noisy image to obtain a predicted segmented image.

[0105] For details, see Figure 4 In some embodiments, step S104 may include but is not limited to steps S401 to S404:

[0106] Step S401, extracting features from the noise image to obtain a noise feature vector;

[0107] Step S402: performing context understanding on the noise feature vector to obtain a noise attention vector;

[0108] Step S403: modify the features of the noise attention vector to obtain a predicted latent vector;

[0109] Step S404: predicting a segmented image based on the predicted latent vector and the noise feature vector.

[0110] In step S401 of some embodiments, the structure of the decoder may be a transposed convolutional neural network structure. Therefore, the noise feature vector of the noise image may be obtained by performing convolution processing on the noise image.

[0111] In step S402 of some embodiments, the multi-head attention module in the above-mentioned decoder can be used to perform contextual understanding of the feature map, thereby obtaining a contextual representation of the noise feature vector with respect to the neighboring vector, wherein the multi-head attention module is a plurality of parallel attention mechanism modules, and the neighboring vector refers to a vector adjacent to the noise feature vector.

[0112] In step S403 of some embodiments, by performing feature reconstruction and amplification processing on the noise attention vector, the features of the predicted object annotation image, that is, the predicted latent vector, can be obtained.

[0113] For details, see Figure 5 In some embodiments, step S403 may include but is not limited to steps S501 to S502:

[0114] Step S501, reconstructing the noise attention vector to obtain a reconstructed feature vector;

[0115] Step S502: perform feature amplification on the reconstructed feature vector to obtain a predicted latent vector.

[0116] In step S501 of some embodiments, the convolution layer in the above-mentioned transposed convolutional neural network can be used to perform convolution processing on the noise attention vector to achieve feature reconstruction of the noise attention vector and obtain a reconstructed feature vector.

[0117] In step S502 of some embodiments, the reconstructed feature vector may be deconvolved using the upsampling layer in the transposed convolutional neural network, thereby amplifying the reconstructed feature vector to obtain the predicted latent vector.

[0118] In steps S501 to S502 shown in this embodiment, the noise attention vector is feature reconstructed to obtain a reconstructed feature vector, and the reconstructed feature vector is feature amplified to obtain a predicted latent vector, so that the size of the predicted latent vector is the same as the size of the input target feature image, thereby improving the clarity of the image segmentation.

[0119] In step S404 of some embodiments, the predicted latent vector is jump-connected with the noise feature vector to enrich the feature content of the predicted latent vector, and the fused features after the jump connection are activated to obtain a predicted segmented image.

[0120] For details, see Figure 6 In some embodiments, step S404 may include but is not limited to steps S601 to S602:

[0121] Step S601, performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature;

[0122] Step S602: Activate the fused features to obtain a predicted segmentation image.

[0123] In step S601 of some embodiments, by performing a jump connection between the noise feature vector that retains the detail information of the noise image and the predicted latent vector after the feature amplification, the output fusion feature can retain more detail information of the noise image, thereby improving the accuracy of the predicted segmented image.

[0124] In step S602 of some embodiments, linear or nonlinear operations are performed on the fused features using the activation function in the initial image segmentation model to obtain the probability that the fused features represent each object in the object annotation image, and based on the probability, the objects represented by the fused features in the object annotation image are segmented to obtain a predicted segmented image.

[0125] In steps S601 to S602 shown in this embodiment, by performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature, and then performing feature activation on the fused feature to obtain a predicted segmented image, the accuracy and detail information of the fused feature can be increased, thereby improving the accuracy of the predicted segmented image.

[0126] In steps S401 to S404 shown in this embodiment, features are extracted from the noise image to obtain a noise feature vector, and then the noise feature vector is contextually understood to obtain a noise attention vector. Secondly, features are modified on the noise attention vector to obtain a predicted latent vector. Finally, based on the predicted latent vector and the noise feature vector, a predicted segmented image is predicted, which can achieve denoising processing of the noise image after noise addition, thereby improving the clarity of the predicted segmented image.

[0127] In step S105 of some embodiments, after obtaining the predicted segmentation image calculated by the initial image segmentation model, the area where the sample segmentation object is located in the object annotation image, that is, the above-mentioned sample segmentation area, can be determined based on the sample annotation information. Then, based on the difference between the sample segmentation area and the predicted segmentation image, the parameters in the initial image segmentation model are updated and adjusted to obtain the trained target image segmentation model.

[0128] For details, see Figure 7 In some embodiments, step S105 may include but is not limited to steps S701 to S703:

[0129] Step S701, determining a sample segmentation area of ​​an object annotation image based on the sample segmentation object and the sample annotation information;

[0130] Step S702 , based on the sample segmentation region and the predicted segmentation image, performing loss calculation on the initial image segmentation model to obtain object segmentation loss data;

[0131] Step S703: Based on the object segmentation loss data, update the parameters of the initial image segmentation model to obtain a target image segmentation model.

[0132] In step S701 of some embodiments, based on the sample annotation information on the object annotation image, the location of the region where the sample segmentation object to be segmented in the object annotation image is located, ie, the above-mentioned sample segmentation region, can be determined.

[0133] In step S702 of some embodiments, the object segmentation loss function preset in the initial image segmentation model is used to calculate the difference between the predicted segmented image and the sample segmented object in the sample segmented region, thereby obtaining object segmentation loss data.

[0134] It should be noted that the predicted segmentation image in this application is obtained after the fusion features are activated, and the initial image segmentation model needs to perform multiple feature fusions to obtain multiple fusion features. Therefore, there are also multiple predicted segmentation images, so it is necessary to calculate the difference between each predicted segmentation image and the sample segmentation object in the sample segmentation area, and then add and merge the multiple loss data representing the difference between the predicted segmentation image and the sample segmentation object in the sample segmentation area to obtain the above-mentioned object segmentation loss data.

[0135] In step S703 of some embodiments, when the object segmentation loss data is greater than a pre-set loss threshold, it indicates that the difference between the predicted segmentation image and the sample segmentation object in the sample segmentation area is too large, and the predicted segmented image needs to be re-predicted, and the relevant parameters in the initial image segmentation model need to be adjusted and optimized, so that the initial image segmentation model is transformed into the target image segmentation model.

[0136] In steps S701 to S703 shown in this embodiment, the sample segmentation area of ​​the object annotation image is determined based on the sample segmentation object and the sample annotation information, and the loss of the initial image segmentation model is calculated based on the sample segmentation area and the predicted segmentation image to obtain object segmentation loss data. Then, based on the object segmentation loss data, the parameters of the initial image segmentation model are updated to obtain the target image segmentation model, thereby improving the accuracy of the model in image segmentation.

[0137] In step S106 of some embodiments, the target image including the target area containing the target object and the target background area not containing the target object can be input into the above-mentioned trained target image segmentation model. The target image segmentation model can obtain the target segmentation image of the target image through the methods of steps S102 to S104.

[0138] It should be noted that when the target image contains multiple target objects, the number of target segmentation images should be the same as the number of target images.

[0139] In an application scenario, taking organ segmentation of medical images as an example, by labeling the area where the organ is located in the medical image, sample labeling information of the medical image is obtained, and then the initial medical image organ segmentation model is used to perform image encoding on the medical image containing the sample labeling information, an image representation of the feature information in the medical image, that is, a target feature medical image, can be obtained. Furthermore, the initial medical image organ segmentation model is used to perform random noise addition processing on the target feature medical image, and a target feature medical image containing noise, that is, a medical noise image, can be obtained. The initial medical image organ segmentation model is then used to perform self-supervised denoising processing on the medical noise image, and the organ image to be segmented, that is, the predicted segmented image, can be predicted. Secondly, according to the organ image and the sample labeling information of the above-mentioned medical image, the parameters in the initial medical image organ segmentation model are adjusted, so that the initial medical image organ segmentation model is converted into a trained target medical image organ segmentation model. Finally, the target medical image organ segmentation model is used to perform organ segmentation on the medical image without labeling information, and the target organ image can be obtained.

[0140] This application obtains an object annotation image by acquiring an image with sample annotation information, and then uses a preset initial image segmentation model to perform image encoding processing on the object annotation image to obtain a target feature image, and injects random noise into the target feature image based on the object annotation image, so that the object annotation image can be closer to the image information under the influence of various noises in real life, thereby improving the accuracy of image segmentation. Secondly, self-supervised image denoising is performed on the noise image, and the parameters of the initial image segmentation model are adjusted according to the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model, thereby improving the accuracy of the model in processing image segmentation tasks. Finally, the target image segmentation model is used to perform image segmentation on the preset target image to obtain a target segmented image, thereby improving the clarity of the segmented image.

[0141] See also Figure 8 The present application also provides an image segmentation device that can implement the above-mentioned image segmentation method. The device includes:

[0142] Annotated image acquisition module 801 is used to acquire an object annotated image with sample annotation information, where the object annotated image includes a sample object region containing the sample segmented object and a sample background region not containing the sample segmented object. The sample annotation information is used to annotate the sample object region.

[0143] The annotated image encoding module 802 is used to perform image encoding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image;

[0144] The feature image noise adding module 803 is used to inject random noise into the target feature image based on the object annotation image to obtain a noise image;

[0145] Noise image denoising module 804, used to perform self-supervised image denoising on the noisy image to obtain a predicted segmented image;

[0146] The initial model optimization module 805 is used to adjust the parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain the target image segmentation model;

[0147] The target image segmentation module 806 is used to perform image segmentation on a preset target image based on a target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing a target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0148] The specific implementation of the image segmentation device is basically the same as the specific embodiment of the image segmentation method described above, and will not be repeated here.

[0149] The embodiment of the present application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to realize the image segmentation method described above. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0150] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0151] The processor 901 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to realize the technical solutions provided by the embodiments of the present application.

[0152] The memory 902 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 902 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 902 and called and executed by the processor 901 to realize the image segmentation method of the embodiments of the present application.

[0153] The input / output interface 903 is used to realize information input and output.

[0154] The communication interface 904 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0155] The bus 905 is used to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0156] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between them inside the device.

[0157] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image segmentation method is implemented.

[0158] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The image segmentation method, image segmentation device, electronic device and storage medium provided by the embodiments of the present application obtain an object annotated image by acquiring an image including a sample object area containing a sample segmentation object and a sample background area not containing the sample segmentation object, wherein the sample annotation information is used to annotate the sample object area, and then using a preset initial image segmentation model, image encoding processing is performed on the object annotated image to obtain a target feature image. Furthermore, based on the object annotated image, random noise is injected into the target feature image to obtain a noise image. Secondly, self-supervised image denoising is performed on the noise image to obtain a predicted segmented image, and the parameters of the initial image segmentation model are adjusted based on the predicted segmented image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model. Finally, according to the target image segmentation model, image segmentation is performed on a preset target image including a target area containing a target object and a target background area not containing a target object to obtain a target segmented image containing the target object and target annotation information, wherein the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target annotation information is used to annotate the target object.

[0160] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0161] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0163] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0164] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0165] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0170] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. An image segmentation method, characterized in that: The method comprises: Acquire an object annotated image having sample annotation information, wherein the object annotated image includes a sample object region containing the sample segmented object and a sample background region not containing the sample segmented object, and the sample annotation information is used to annotate the sample object region; Based on a preset initial image segmentation model, image coding processing is performed on the object annotation image to obtain a target feature image; Based on the object annotation image, random noise is injected into the target feature image to obtain a noise image; Performing self-supervised image denoising on the noisy image to obtain a predicted segmented image; Based on the predicted segmented image, the sample segmentation object and the sample annotation information, adjusting parameters of the initial image segmentation model to obtain a target image segmentation model; Image segmentation is performed on a preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

2. The method according to claim 1, characterized in that The performing self-supervised image denoising on the noisy image to obtain a predicted segmented image includes: Performing feature extraction on the noise image to obtain a noise feature vector; Performing contextual understanding on the noise feature vector to obtain a noise attention vector; Performing feature modification on the noise attention vector to obtain a predicted latent vector; The predicted segmented image is predicted based on the predicted latent vector and the noise feature vector.

3. The method according to claim 2, characterized in that The step of modifying the noise attention vector to obtain a predicted latent vector includes: Performing feature reconstruction on the noise attention vector to obtain a reconstructed feature vector; Feature amplification is performed on the reconstructed feature vector to obtain the predicted latent vector.

4. The method according to claim 2, characterized in that The predicting the predicted segmented image based on the predicted latent vector and the noise feature vector includes: Performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature; Feature activation is performed on the fused features to obtain the predicted segmentation image.

5. The method according to claim 1, wherein The step of injecting random noise into the target feature image based on the object annotation image to obtain a noise image includes: Based on the object annotated image, a target noise sequence is screened out from a preset noise data pool, wherein the target noise sequence includes target noise; Scaling the target noise to obtain random noise; Based on the random noise, the target feature image is subjected to noise addition processing to obtain the noise image.

6. The method according to any one of claims 1 to 5, characterized in that The step of adjusting parameters of the initial image segmentation model based on the predicted segmented image, the sample segmentation object, and the sample annotation information to obtain a target image segmentation model includes: Determining a sample segmentation area of ​​the object annotation image based on the sample segmentation object and the sample annotation information; Based on the sample segmentation area and the predicted segmentation image, performing loss calculation on the initial image segmentation model to obtain object segmentation loss data; Based on the object segmentation loss data, the parameters of the initial image segmentation model are updated to obtain the target image segmentation model.

7. The method according to any one of claims 1 to 5, characterized in that The step of performing image coding processing on the object annotation image based on a preset initial image segmentation model to obtain a target feature image includes: performing standardization processing on the object annotated image to obtain a standard image; Based on the initial image segmentation model, convolution processing is performed on the object annotation image to obtain an initial feature image; Pooling is performed on the initial feature image to obtain the target feature image.

8. An image segmentation device, characterized in that: The device comprises: a labeled image acquisition module, configured to acquire an object labeled image having sample labeling information, wherein the object labeled image includes a sample object region containing the sample segmented object and a sample background region not containing the sample segmented object, and the sample labeling information is used to label the sample object region; Annotated image coding module, used for performing image coding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image; A feature image denoising module, configured to inject random noise into the target feature image based on the object annotation image to obtain a noise image; a noise image denoising module, configured to perform self-supervised image denoising on the noise image to obtain a predicted segmented image; An initial model optimization module is used to adjust the parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model; A target image segmentation module is used to perform image segmentation on a preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image contains the target object and target annotation information, and the target annotation information is used to annotate the target object.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the image segmentation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image segmentation method according to any one of claims 1 to 7 is implemented.

Images