Image segmentation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image segmentation method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing and artificial intelligence. The method comprises the following steps: acquiring an object annotation image with sample annotation information; based on a preset initial image segmentation model, carrying out image coding processing on the object annotation image to obtain a target feature image; performing random noise injection on the target feature image based on the object annotation image to obtain a noise image; performing self-supervised image denoising on the noise image to obtain a predicted segmented image; based on the prediction segmentation image, the sample segmentation object and the sample labeling information, performing parameter adjustment on the initial image segmentation model to obtain a target image segmentation model; and performing image segmentation on a preset target image based on the target image segmentation model to obtain a target segmented image. According to the embodiment of the invention, the definition of segmented images can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing and artificial intelligence technology, and in particular to an image segmentation method and device, an electronic device and a storage medium. Background Art

[0002] Image segmentation refers to separating a specified area in an image from the background and other areas. For example, in the medical field, by performing image segmentation on the area where the organs are located in the medical image, organ images for disease diagnosis and treatment can be obtained.

[0003] Currently, the common image segmentation method usually uses convolutional neural networks to segment the target image. However, the target image is easily affected by factors such as noise, artifacts, low contrast, and uneven lighting, resulting in poor image quality and making the segmented image blurred. Therefore, how to improve the clarity of the segmented image has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an image segmentation method and device, an electronic device and a storage medium, aiming to improve the clarity of the segmented image.

[0005] To achieve the above object, a first aspect of an embodiment of the present application provides an image segmentation method, the method comprising:

[0006] Acquire an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing the sample segmentation object, and the sample annotation information is used to annotate the sample object region;

[0007] Based on a preset initial image segmentation model, image encoding processing is performed on the object annotation image to obtain a target feature image;

[0008] Based on the object annotation image, injecting random noise into the target feature image to obtain a noise image;

[0009] Performing self-supervised image denoising on the noisy image to obtain a predicted segmented image;

[0010] Based on the predicted segmented image, the sample segmentation object and the sample annotation information, adjusting parameters of the initial image segmentation model to obtain a target image segmentation model;

[0011] Based on the target image segmentation model, image segmentation is performed on a preset target image to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0012] In some embodiments, performing self-supervised image denoising on the noisy image to obtain a predicted segmented image includes:

[0013] Extracting features from the noise image to obtain a noise feature vector;

[0014] Performing context understanding on the noise feature vector to obtain a noise attention vector;

[0015] Performing feature modification on the noise attention vector to obtain a predicted latent vector;

[0016] The predicted segmented image is predicted based on the predicted latent vector and the noise feature vector.

[0017] In some embodiments, the step of performing feature modification on the noise attention vector to obtain a predicted latent vector includes:

[0018] Performing feature reconstruction on the noise attention vector to obtain a reconstructed feature vector;

[0019] Feature amplification is performed on the reconstructed feature vector to obtain the predicted latent vector.

[0020] In some embodiments, predicting the predicted segmented image based on the predicted latent vector and the noise feature vector includes:

[0021] Performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature;

[0022] Feature activation is performed on the fused features to obtain the predicted segmentation image.

[0023] In some embodiments, injecting random noise into the target feature image based on the object annotation image to obtain a noise image includes:

[0024] Based on the object annotated image, a target noise sequence is screened out from a preset noise data pool, wherein the target noise sequence includes target noise;

[0025] Scaling the target noise to obtain random noise;

[0026] Based on the random noise, the target feature image is subjected to noise addition processing to obtain the noise image.

[0027] In some embodiments, adjusting parameters of the initial image segmentation model based on the predicted segmented image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model includes:

[0028] Determining the sample segmentation area of ​​the object annotation image based on the sample segmentation object and the sample annotation information;

[0029] Based on the sample segmentation area and the predicted segmentation image, performing loss calculation on the initial image segmentation model to obtain object segmentation loss data;

[0030] Based on the object segmentation loss data, the parameters of the initial image segmentation model are updated to obtain the target image segmentation model.

[0031] In some embodiments, performing image encoding processing on the object annotation image based on a preset initial image segmentation model to obtain a target feature image includes:

[0032] Performing standardization processing on the object annotated image to obtain a standard image;

[0033] Based on the initial image segmentation model, convolution processing is performed on the object annotation image to obtain an initial feature image;

[0034] The initial feature image is pooled to obtain the target feature image.

[0035] To achieve the above objective, a second aspect of an embodiment of the present application provides an image segmentation device, the device comprising:

[0036] an annotated image acquisition module, used to acquire an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing the sample segmentation object, and the sample annotation information is used to annotate the sample object region;

[0037] The annotated image encoding module is used to perform image encoding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image;

[0038] A feature image denoising module, used for injecting random noise into the target feature image based on the object annotation image to obtain a noise image;

[0039] A noise image denoising module, used for performing self-supervised image denoising on the noise image to obtain a predicted segmented image;

[0040] An initial model optimization module, used to adjust parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model;

[0041] A target image segmentation module is used to perform image segmentation on a preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0042] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0043] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0044] The image segmentation method and device, electronic device and storage medium proposed in the present application obtain an object annotated image by acquiring an image with sample annotation information, and then use a preset initial image segmentation model to perform image encoding processing on the object annotated image to obtain a target feature image, and inject random noise into the target feature image based on the object annotated image, so that the object annotated image can be closer to the image information under the influence of various noises in real life, thereby improving the accuracy of image segmentation. Secondly, self-supervised image denoising is performed on the noise image, and the parameters of the initial image segmentation model are adjusted according to the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model, thereby improving the accuracy of the model in processing image segmentation tasks. Finally, the target image segmentation model is used to perform image segmentation on the preset target image to obtain a target segmented image, thereby improving the clarity of the segmented image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the image segmentation method provided in an embodiment of the present application;

[0046] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0047] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0048] Figure 4 yes Figure 1 Flow chart of step S104 in FIG.

[0049] Figure 5 yes Figure 4 Flow chart of step S403 in FIG.

[0050] Figure 6 yes Figure 4 Flow chart of step S404 in FIG.

[0051] Figure 7 yes Figure 1 Flow chart of step S105 in FIG.

[0052] Figure 8 is a schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application;

[0053] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] First, some nouns involved in this application are analyzed:

[0058] Diffusion model: Diffusion model is a generative model that learns the data distribution by simulating the process of gradually adding and removing noise from the data. In the training phase, the model gradually transforms the real data into pure noise and gradually removes the noise through the inverse process to generate new samples. In the generation phase, starting from random noise, the model applies the inverse diffusion process to gradually restore the data structure to generate new samples similar to the training data. Diffusion models perform well in tasks such as image generation, image restoration, and super-resolution.

[0059] Attention mechanism: The attention mechanism is a computational method that mimics human visual attention, which aims to enable the model to dynamically focus on the most relevant information when processing input data, thereby improving performance. Its benefit is that it can effectively capture long-distance dependencies and important features, significantly improving the accuracy and efficiency of tasks such as natural language processing, image recognition, and translation. By adaptively adjusting the focus of attention, the attention mechanism enhances the model's expressiveness and enables it to perform better in complex tasks.

[0060] Multi-head attention mechanism: Multi-head attention mechanism is a deep learning technique used to focus on different parts of the input simultaneously when processing sequence data. It captures different contextual information by dividing the input features into multiple subspaces, each of which calculates attention weights independently. Finally, the outputs of these subspaces are concatenated or weighted to form a rich feature representation. Multi-head attention mechanism is widely used in fields such as natural language processing and computer vision, enhancing the model's ability to understand complex dependencies.

[0061] Image segmentation refers to separating a specified area in an image from the background and other areas. For example, in the medical field, by performing image segmentation on the area where the organs are located in the medical image, organ images for disease diagnosis and treatment can be obtained.

[0062] Currently, the common image segmentation method usually uses convolutional neural networks to segment the target image. However, the target image is easily affected by factors such as noise, artifacts, low contrast, and uneven lighting, resulting in poor image quality and making the segmented image blurred. Therefore, how to improve the clarity of the segmented image has become a technical problem that needs to be solved urgently.

[0063] Based on this, the embodiments of the present application provide an image segmentation method and device, an electronic device and a storage medium, aiming to improve the clarity of the segmented image.

[0064] The image segmentation method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the image segmentation method in the embodiments of the present application is described.

[0065] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0066] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0067] The image segmentation method provided in the embodiment of the present application relates to the field of image processing and artificial intelligence technology. The image segmentation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the image segmentation method, etc., but is not limited to the above forms.

[0068] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0069] Figure 1 is an optional flowchart of the image segmentation method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0070] Step S101, obtaining an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing the sample segmentation object, and the sample annotation information is used to annotate the sample object region;

[0071] Step S102, based on a preset initial image segmentation model, performing image coding processing on the object annotation image to obtain a target feature image;

[0072] Step S103, based on the object annotation image, random noise is injected into the target feature image to obtain a noise image;

[0073] Step S104, performing self-supervised image denoising on the noisy image to obtain a predicted segmented image;

[0074] Step S105, adjusting the parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model;

[0075] Step S106, performing image segmentation on a preset target image based on a target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing a target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0076] In steps S101 to S108 shown in the embodiment of the present application, an object annotated image is obtained by acquiring an image including a sample object area containing a sample segmentation object and a sample background area not containing the sample segmentation object, wherein the sample annotation information is used to annotate the sample object area, and then a preset initial image segmentation model is used to perform image encoding processing on the object annotated image to obtain a target feature image. Furthermore, based on the object annotated image, random noise is injected into the target feature image to obtain a noise image. Secondly, self-supervised image denoising is performed on the noise image to obtain a predicted segmentation image, and parameters of the initial image segmentation model are adjusted based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model. Finally, according to the target image segmentation model, an image segmentation is performed on a preset target image including a target area containing a target object and a target background area not containing a target object to obtain a target segmentation image containing the target object and the target annotation information, wherein the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target annotation information is used to annotate the target object. Therefore, the present application obtains an object annotated image by acquiring an image with sample annotation information, and then uses a preset initial image segmentation model to perform image encoding processing on the object annotated image to obtain a target feature image, and injects random noise into the target feature image based on the object annotated image, so that the object annotated image can be closer to the image information under the influence of various noises in real life, thereby improving the accuracy of image segmentation. Secondly, self-supervised image denoising is performed on the noise image, and the parameters of the initial image segmentation model are adjusted according to the predicted segmented image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model, thereby improving the accuracy of the model in processing image segmentation tasks. Finally, the target image segmentation model is used to perform image segmentation on the preset target image to obtain a target segmented image, thereby improving the clarity of the segmented image.

[0077] In step S101 of some embodiments, the sample segmentation object may be an object to be segmented. For example, if an organ in a medical image needs to be segmented, the sample segmentation object may be a lung, a kidney, a heart, etc. in the medical image.

[0078] It should be noted that the object annotated image may include but is not limited to image data such as medical images and product images.

[0079] The present application can obtain an object-annotated image by performing target annotation on an unlabeled initial image. Specifically, it is first necessary to perform target recognition on the initial image to identify object areas in the initial image that may be target objects, namely, suspected object areas. Secondly, the suspected object areas are screened to obtain sample object areas. The non-sample object areas in the initial image are then annotated with sample background areas. Information is annotated on the sample object areas to obtain sample segmented objects. Finally, the initial image with sample annotation information is called an object-annotated image.

[0080] In step S102 of some embodiments, the initial image segmentation model may be a diffusion model with set initial parameters and network structure, and the initial image segmentation model includes an encoder and a decoder.

[0081] The present application can utilize the encoder in the initial image segmentation model to encode the acquired object annotated image to obtain a target feature image containing image features. Specifically, the encoding processing of the object annotated image by the initial image segmentation model includes but is not limited to operations such as image scaling, image cropping, image normalization, image data enhancement, and convolution pooling.

[0082] For details, see Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0083] Step S201, standardizing the object annotated image to obtain a standard image;

[0084] Step S202, based on the initial image segmentation model, convolution processing is performed on the object annotation image to obtain an initial feature image;

[0085] Step S203, performing pooling processing on the initial feature image to obtain a target feature image.

[0086] In step S201 of some embodiments, the object annotated image may be standardized by performing a series of processes such as image scaling, image cropping, image normalization, and image data enhancement on the object annotated image to obtain a standard image.

[0087] In step S202 of some embodiments, the object annotation image is convolved using the convolutional layer of the convolutional neural network of the encoding layer in the initial image segmentation model to obtain an initial feature image containing feature information.

[0088] In step S203 of some embodiments, the initial feature image is pooled by using the pooling layer of the convolutional neural network of the encoding layer to obtain a reduced-dimensional initial feature image, that is, a target feature image.

[0089] In steps S201 to S203 shown in this embodiment, the object annotated image is standardized to obtain a standard image, and then the object annotated image is convolved according to the initial image segmentation model to obtain an initial feature image. The initial feature image is pooled, which can extract image feature information in the object annotated image and reduce the difficulty of calculating the image feature information in the object annotated image by the model, thereby improving the efficiency of image segmentation.

[0090] In step S103 of some embodiments, after obtaining the target feature image, it is necessary to determine the type of noise to be added based on the object annotation image, and then construct a target noise sequence based on the selected noise type. Furthermore, the target feature image is denoised using the noise in the target noise sequence to obtain a noise image containing noise data.

[0091] For details, see Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S303:

[0092] Step S301, based on the object annotated image, screening out a target noise sequence from a preset noise data pool, wherein the target noise sequence includes target noise;

[0093] Step S302, scaling the target noise to obtain random noise;

[0094] Step S303: performing noise processing on the target feature image based on random noise to obtain a noise image.

[0095] In step S301 of some embodiments, the noise data pool is a pre-built data warehouse for storing noise data of various styles, such as Gaussian noise, white noise, etc. The target noise sequence is composed of multiple target noises.

[0096] The present application can annotate an image according to an object, clarify the noise type of noise data that needs to be added, and then filter the target noise from a pre-built noise data pool according to the noise type, and sort the filtered target noise to obtain a target noise sequence.

[0097] It is important to note that the amount of target noise is the same as the number of times noise is added to the target feature image.

[0098] In step S302 of some embodiments, before adding noise to the target feature image using the target noise, in order to improve robustness, the target noise can be scaled to obtain random noise. Specifically, a random vector that obeys a standard normal distribution is randomly generated, and then the random vector is multiplied by the square of the variance of the target noise to obtain the random noise.

[0099] In step S303 of some embodiments, the random noise and the target feature image may be added and combined to obtain a noise image.

[0100] It should be noted that the process of adding noise is a multi-cycle process, and the number of cycles is the same as the number of target noise minus one. Therefore, the present application can use the following formula to represent the process of adding noise to the target feature image:

[0101] x t+1 =(1-z) 2 x t +(z) 2 s

[0102] Among them, x represents the state of the target feature image after noise processing, t+1 represents the next moment after time t, and x t+1 represents the state of the target feature image at time t+1, z represents the variance of the target noise, x t represents the state of the target feature image at time t, and s represents the above random vector.

[0103] In steps S301 to S303 shown in this embodiment, first, according to the sample segmentation object, a target noise sequence is screened out from a preset noise data pool to ensure the accuracy of the added target noise. Secondly, the target noise is scaled to obtain random noise, which can improve the randomness of the noise image. Finally, according to the random noise, the target feature image is noised to obtain a noise image, so that the noise image can express the state of the initial image that can be encountered in practice, reducing the number of initial image samples required.

[0104] In step S104 of some embodiments, the decoder in the initial image segmentation model may be used to perform denoising on the noisy image to obtain a predicted segmented image.

[0105] For details, see Figure 4 In some embodiments, step S104 may include but is not limited to steps S401 to S404:

[0106] Step S401, extracting features from the noise image to obtain a noise feature vector;

[0107] Step S402, performing context understanding on the noise feature vector to obtain a noise attention vector;

[0108] Step S403, modifying the features of the noise attention vector to obtain a predicted latent vector;

[0109] Step S404: predicting a segmented image based on the predicted latent vector and the noise feature vector.

[0110] In step S401 of some embodiments, the structure of the decoder may be a transposed convolutional neural network structure. Therefore, the noise feature vector of the noise image may be obtained by performing convolution processing on the noise image.

[0111] In step S402 of some embodiments, the multi-head attention module in the above-mentioned decoder can be used to perform contextual understanding of the feature map, so as to obtain a contextual representation of the noise feature vector with respect to the neighboring vector, wherein the multi-head attention module is a plurality of parallel attention mechanism modules, and the neighboring vector refers to a vector adjacent to the noise feature vector.

[0112] In step S403 of some embodiments, the features of the predicted object annotation image, ie, the predicted latent vector, can be obtained by performing feature reconstruction and amplification processing on the noise attention vector.

[0113] For details, see Figure 5 In some embodiments, step S403 may include but is not limited to steps S501 to S502:

[0114] Step S501, reconstructing the noise attention vector to obtain a reconstructed feature vector;

[0115] Step S502: perform feature amplification on the reconstructed feature vector to obtain a predicted latent vector.

[0116] In step S501 of some embodiments, the convolution layer in the above-mentioned transposed convolutional neural network can be used to perform convolution processing on the noise attention vector to achieve feature reconstruction of the noise attention vector and obtain a reconstructed feature vector.

[0117] In step S502 of some embodiments, the upsampling layer in the transposed convolutional neural network may be used to perform inverse convolution processing on the reconstructed feature vector, so as to amplify the reconstructed feature vector and obtain the predicted latent vector.

[0118] In steps S501 to S502 shown in this embodiment, the noise attention vector is feature reconstructed to obtain a reconstructed feature vector, and then the reconstructed feature vector is feature amplified to obtain a predicted latent vector, so that the size of the predicted latent vector is the same as the size of the input target feature image, thereby improving the clarity of image segmentation.

[0119] In step S404 of some embodiments, the predicted latent vector is jump-connected with the noise feature vector to enrich the feature content of the predicted latent vector, and then the fused features after the jump connection are activated to obtain a predicted segmented image.

[0120] For details, see Figure 6 In some embodiments, step S404 may include but is not limited to steps S601 to S602:

[0121] Step S601, performing feature fusion on the predicted latent vector and the noise feature vector to obtain fused features;

[0122] Step S602: Activate the fused features to obtain a predicted segmentation image.

[0123] In step S601 of some embodiments, by jump-connecting the noise feature vector retaining the detail information of the noise image and the predicted latent vector after the feature amplification, the output fusion feature can retain more detail information of the noise image, thereby improving the accuracy of the predicted segmented image.

[0124] In step S602 of some embodiments, linear or nonlinear operations are performed on the fused features using the activation function in the initial image segmentation model to obtain the probability that the fused features represent each object in the object annotated image, and based on the probability, the objects represented by the fused features in the object annotated image are segmented to obtain a predicted segmented image.

[0125] In steps S601 to S602 shown in this embodiment, by performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature, and then performing feature activation on the fused feature to obtain a predicted segmented image, the accuracy detail information of the fused feature can be increased, thereby improving the accuracy of the predicted segmented image.

[0126] In steps S401 to S404 shown in this embodiment, features are extracted from the noise image to obtain a noise feature vector, and then the noise feature vector is contextually understood to obtain a noise attention vector. Next, features of the noise attention vector are modified to obtain a predicted latent vector. Finally, a predicted segmented image is predicted based on the predicted latent vector and the noise feature vector, which can achieve denoising of the noise image after noise addition, thereby improving the clarity of the predicted segmented image.

[0127] In step S105 of some embodiments, after obtaining the predicted segmentation image calculated by the initial image segmentation model, the area where the sample segmentation object is located in the object annotation image, that is, the above-mentioned sample segmentation area, can be determined based on the sample annotation information, and then based on the difference between the sample segmentation area and the predicted segmentation image, the parameters in the initial image segmentation model are updated and adjusted to obtain a trained target image segmentation model.

[0128] For details, see Figure 7 In some embodiments, step S105 may include but is not limited to steps S701 to S703:

[0129] Step S701, determining a sample segmentation region of an object annotation image based on a sample segmentation object and sample annotation information;

[0130] Step S702, based on the sample segmentation area and the predicted segmentation image, the initial image segmentation model is subjected to loss calculation to obtain object segmentation loss data;

[0131] Step S703: based on the object segmentation loss data, update the parameters of the initial image segmentation model to obtain a target image segmentation model.

[0132] In step S701 of some embodiments, based on the sample annotation information on the object annotation image, the location of the region where the sample segmentation object to be segmented in the object annotation image is located, ie, the above-mentioned sample segmentation region, can be determined.

[0133] In step S702 of some embodiments, the object segmentation loss function preset in the initial image segmentation model is used to calculate the difference between the predicted segmented image and the sample segmented object in the sample segmentation region, thereby obtaining object segmentation loss data.

[0134] It should be noted that in the present application, the predicted segmentation image is obtained after the fusion features are activated, and the initial image segmentation model needs to perform multiple feature fusions to obtain multiple fusion features. Therefore, there are also multiple predicted segmentation images. Therefore, it is necessary to calculate the difference between each predicted segmentation image and the sample segmentation object in the sample segmentation area, and then add and merge the multiple loss data representing the difference between the predicted segmentation image and the sample segmentation object in the sample segmentation area to obtain the above-mentioned object segmentation loss data.

[0135] In step S703 of some embodiments, when the object segmentation loss data is greater than a preset loss threshold, it indicates that the difference between the predicted segmented image and the sample segmented object in the sample segmentation area is too large, and the image obtained by the prediction segmentation needs to be re-performed. Then, the relevant parameters in the initial image segmentation model need to be adjusted and optimized, so that the initial image segmentation model is transformed into the target image segmentation model.

[0136] In steps S701 to S703 shown in the present embodiment, the sample segmentation region of the object annotation image is determined according to the sample segmentation object and the sample annotation information, and the loss of the initial image segmentation model is calculated according to the sample segmentation region and the predicted segmentation image to obtain the object segmentation loss data, and then the parameters of the initial image segmentation model are updated according to the object segmentation loss data to obtain the target image segmentation model, thereby improving the accuracy of the model in performing image segmentation.

[0137] In step S106 of some embodiments, the target image including the target area containing the target object and the target background area not containing the target object can be input into the above-mentioned trained target image segmentation model. The target image segmentation model can obtain the target segmentation image of the target image through the methods of steps S102 to S104.

[0138] It should be noted that when the target image contains multiple target objects, the number of target segmentation images should be the same as the number of target images.

[0139] In an application scenario, taking the organ segmentation of a medical image as an example, by labeling the region where the organ is located in the medical image, sample labeling information of the medical image is obtained, and then the initial medical image organ segmentation model is used to perform image encoding on the medical image containing the sample labeling information, so as to obtain an image representation of the feature information in the medical image, that is, a target feature medical image. Furthermore, the initial medical image organ segmentation model is used to perform random noise addition processing on the target feature medical image, so as to obtain a target feature medical image containing noise, that is, a medical noise image. The initial medical image organ segmentation model is then used to perform self-supervised denoising processing on the medical noise image, so as to predict the organ image to be segmented, that is, the predicted segmented image. Secondly, according to the sample labeling information of the organ image and the above-mentioned medical image, the parameters in the initial medical image organ segmentation model are adjusted, so that the initial medical image organ segmentation model is converted into a trained target medical image organ segmentation model. Finally, the target medical image organ segmentation model is used to perform organ segmentation on the medical image without labeling information, so as to obtain the target organ image.

[0140] The present application obtains an object annotated image by acquiring an image with sample annotation information, and then uses a preset initial image segmentation model to perform image encoding processing on the object annotated image to obtain a target feature image, and injects random noise into the target feature image based on the object annotated image, so that the object annotated image can be closer to the image information under the influence of various noises in real life, thereby improving the accuracy of image segmentation. Secondly, self-supervised image denoising is performed on the noise image, and parameters of the initial image segmentation model are adjusted according to the predicted segmented image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model, thereby improving the accuracy of the model in processing image segmentation tasks. Finally, the target image segmentation model is used to perform image segmentation on the preset target image to obtain a target segmented image, thereby improving the clarity of the segmented image.

[0141] See also Figure 8 The embodiment of the present application further provides an image segmentation device, which can implement the above-mentioned image segmentation method, and the device includes:

[0142] The annotated image acquisition module 801 is used to acquire an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing a sample segmentation object, and the sample annotation information is used to annotate the sample object region;

[0143] The annotated image encoding module 802 is used to perform image encoding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image;

[0144] The feature image noise adding module 803 is used to inject random noise into the target feature image based on the object annotation image to obtain a noise image;

[0145] A noise image denoising module 804 is used to perform self-supervised image denoising on the noise image to obtain a predicted segmented image;

[0146] The initial model optimization module 805 is used to adjust the parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain the target image segmentation model;

[0147] The target image segmentation module 806 is used to perform image segmentation on a preset target image based on a target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing a target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image contains the target object and target annotation information, and the target annotation information is used to annotate the target object.

[0148] The specific implementation of the image segmentation device is basically the same as the specific implementation of the above-mentioned image segmentation method, and will not be repeated here.

[0149] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above image segmentation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0150] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0151] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0152] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the image segmentation method of the embodiment of the present application;

[0153] Input / output interface 903, used to implement information input and output;

[0154] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0155] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0156] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0157] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image segmentation method is implemented.

[0158] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The image segmentation method, image segmentation device, electronic device and storage medium provided by the embodiments of the present application obtain an object annotated image by acquiring an image including a sample object area containing a sample segmentation object and a sample background area not containing the sample segmentation object, wherein the sample annotation information is used to annotate the sample object area, and then use a preset initial image segmentation model to perform image encoding processing on the object annotated image to obtain a target feature image, and further, according to the object annotated image, random noise is injected into the target feature image to obtain a noise image, and then, self-supervised image denoising is performed on the noise image to obtain a predicted segmented image, and according to the predicted segmented image, the sample segmentation object and the sample annotation information, the parameters of the initial image segmentation model are adjusted to obtain a target image segmentation model, and finally, according to the target image segmentation model, a preset target image including a target area containing a target object and a target background area not containing a target object is segmented to obtain a target segmented image containing the target object and the target annotation information, wherein the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target annotation information is used to annotate the target object.

[0160] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0161] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0162] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0164] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0165] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0170] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An image segmentation method, characterized in that: The method comprises: Acquire an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing the sample segmentation object, and the sample annotation information is used to annotate the sample object region; Based on a preset initial image segmentation model, image encoding processing is performed on the object annotation image to obtain a target feature image; Based on the object annotation image, injecting random noise into the target feature image to obtain a noise image; Performing self-supervised image denoising on the noisy image to obtain a predicted segmented image; Based on the predicted segmented image, the sample segmentation object and the sample annotation information, adjusting parameters of the initial image segmentation model to obtain a target image segmentation model; Based on the target image segmentation model, image segmentation is performed on a preset target image to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

2. The method according to claim 1, characterized in that: The performing self-supervised image denoising on the noisy image to obtain a predicted segmented image includes: Extracting features from the noise image to obtain a noise feature vector; Performing context understanding on the noise feature vector to obtain a noise attention vector; Performing feature modification on the noise attention vector to obtain a predicted latent vector; The predicted segmented image is predicted based on the predicted latent vector and the noise feature vector.

3. The method according to claim 2, characterized in that The step of performing feature modification on the noise attention vector to obtain a predicted latent vector includes: Performing feature reconstruction on the noise attention vector to obtain a reconstructed feature vector; Feature amplification is performed on the reconstructed feature vector to obtain the predicted latent vector.

4. The method according to claim 2, characterized in that: The predicting the predicted segmented image based on the predicted latent vector and the noise feature vector includes: Performing feature fusion on the predicted latent vector and the noise feature vector to obtain a fused feature; Feature activation is performed on the fused features to obtain the predicted segmentation image.

5. The method according to claim 1, characterized in that: The step of injecting random noise into the target feature image based on the object annotation image to obtain a noise image includes: Based on the object annotated image, a target noise sequence is screened out from a preset noise data pool, wherein the target noise sequence includes target noise; Scaling the target noise to obtain random noise; Based on the random noise, the target feature image is subjected to noise addition processing to obtain the noise image.

6. The method according to any one of claims 1 to 5, characterized in that: The step of adjusting parameters of the initial image segmentation model based on the predicted segmented image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model includes: Determining the sample segmentation area of ​​the object annotation image based on the sample segmentation object and the sample annotation information; Based on the sample segmentation area and the predicted segmentation image, performing loss calculation on the initial image segmentation model to obtain object segmentation loss data; Based on the object segmentation loss data, the parameters of the initial image segmentation model are updated to obtain the target image segmentation model.

7. The method according to any one of claims 1 to 5, characterized in that: The step of performing image coding processing on the object annotation image based on a preset initial image segmentation model to obtain a target feature image includes: Performing standardization processing on the object annotated image to obtain a standard image; Based on the initial image segmentation model, convolution processing is performed on the object annotation image to obtain an initial feature image; The initial feature image is pooled to obtain the target feature image.

8. An image segmentation device, characterized in that: The device comprises: an annotated image acquisition module, used to acquire an object annotated image with sample annotation information, wherein the object annotated image includes a sample object region containing a sample segmentation object and a sample background region not containing the sample segmentation object, and the sample annotation information is used to annotate the sample object region; The annotated image encoding module is used to perform image encoding processing on the object annotated image based on a preset initial image segmentation model to obtain a target feature image; A feature image denoising module, used for injecting random noise into the target feature image based on the object annotation image to obtain a noise image; A noise image denoising module, used for performing self-supervised image denoising on the noise image to obtain a predicted segmented image; An initial model optimization module, used to adjust parameters of the initial image segmentation model based on the predicted segmentation image, the sample segmentation object and the sample annotation information to obtain a target image segmentation model; A target image segmentation module is used to perform image segmentation on a preset target image based on the target image segmentation model to obtain a target segmentation image, wherein the target image includes a target area containing a target object and a target background area not containing the target object, the target object is at least one of the sample segmentation objects, the target background area is at least one of the sample background areas, and the target segmentation image includes the target object and target annotation information, and the target annotation information is used to annotate the target object.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the image segmentation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image segmentation method according to any one of claims 1 to 7 is implemented.

Images