An image segmentation method and apparatus

Abstract

This application provides an image segmentation method, apparatus, electronic device, and computer-readable medium. First, a target image is acquired. Then, a first pixel annotation result of the target image is acquired. Subsequently, a first model is invoked to obtain at least one first aggregated annotation result based on the first pixel annotation result. Next, a second model is invoked to obtain at least one second aggregated annotation result based on the target image and at least one first aggregated annotation result. Finally, a segmentation result of the target image is obtained based on at least one second aggregated annotation result. The first pixel annotation result indicates the first classification of each pixel in the target image. Each first aggregated annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme, which are related to the first classification. Each second aggregated annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to an image segmentation method and apparatus, an electronic device, and a computer-readable medium. Background Technology

[0002] Image recognition is widely used across various industries, especially in image-intensive fields. Its main purpose is to accurately determine whether an image contains the required content, whether the image content meets specific conditions, or to locate specific elements within an image. For example, in the medical field, it is necessary to recognize medical images such as ultrasound, MRI, CT, and endoscopic images of patients to identify lesions. In meteorology, it is necessary to recognize satellite images such as cloud maps and infrared reflectance maps to identify regions with specific meteorological conditions. In mechanical engineering, it is necessary to recognize X-ray images or ultrasound images used in metal flaw detection to identify defects such as cracks. In materials science, it is necessary to recognize images from scanning electron microscopes or atomic force microscopes to identify specific microstructures.

[0003] As human society becomes increasingly information-driven, the need for image recognition in daily life and production is growing exponentially. The sheer number of images requiring recognition across various fields is placing a tremendous burden on relevant personnel. Therefore, in recent years, artificial intelligence (AI) technology has been gradually introduced into image recognition tasks to assist staff in completing their work more efficiently. Taking the medical field as an example, with increasing environmental pollution, the number of cancer patients is also rising. However, training a skilled cancer diagnostician is extremely costly. To improve doctors' diagnostic efficiency, AI-based diagnostic technology for medical images has emerged. This technology primarily uses AI networks to perform initial screening of medical images, followed by secondary screening by doctors, thereby improving doctors' work efficiency and reducing misdiagnosis due to work fatigue.

[0004] Currently, image recognition in various fields mainly includes the following three modes. First, there are image classification-based processing methods, which directly classify the image and inform staff whether the image contains a region of interest (ROI). Second, there are feature detection-based processing methods, which use detection algorithms to identify ROIs containing target elements in the image, such as the region bounded by the element's circumscribed rectangle. Third, there are image segmentation-based processing methods, which segment one or more target elements in the image from the rest (usually called the background) into different regions. This method is pixel-level processing and can obtain ROIs with very fine granularity.

[0005] Among the three methods mentioned above, image segmentation-based processing methods have great potential in applications requiring accurate image recognition due to their high precision. However, since image segmentation typically involves pixel-level operations, existing image segmentation methods often employ unsupervised or weakly supervised approaches, relying on machine learning on massive datasets to achieve relatively ideal segmentation results. With smaller sample sizes, only time-consuming and laborious strong supervision is possible, or a sacrifice in segmentation accuracy may be necessary. Furthermore, in these image segmentation schemes, human intervention usually only occurs in the model training domain for sample annotation, without participating in the actual image segmentation task, thus wasting the experience of professionals to some extent. Summary of the Invention

[0006] In view of this, the embodiments of this application mainly provide an image segmentation method and apparatus, electronic device and computer-readable medium, which can accurately segment complex images using pre-annotation even with limited sample size. To achieve the above technical objectives, the embodiments of this application provide the following technical solutions.

[0007] One aspect of this application provides an image segmentation method, including:

[0008] Acquire the target image;

[0009] Obtain the first pixel annotation result of the target image, wherein the first pixel annotation result is used to indicate the first category of each pixel in the target image;

[0010] The first model is invoked to obtain at least one first aggregated annotation result based on the first pixel annotation result, wherein each first aggregated annotation result is used to indicate the features related to the first classification of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the first aggregated annotation result; and,

[0011] The second model is invoked to obtain at least one second aggregated annotation result based on the target image and the at least one first aggregated annotation result, wherein each second aggregated annotation result is used to indicate the features of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the second aggregated annotation result;

[0012] Based on the at least one second aggregated annotation result, the segmentation result of the target image is obtained.

[0013] In some embodiments, the image segmentation apparatus further includes: training an initial first model and an initial second model based on a training sample set to obtain the first model and the second model. The training sample set includes at least one training image, and each training image has a preset aggregation annotation result and a preset segmentation result.

[0014] Another aspect of this application provides an image segmentation apparatus, including:

[0015] The acquisition module is used to acquire the target image;

[0016] The first annotation module is used to obtain the first pixel annotation result for each target, wherein the first pixel annotation result is used to indicate the first category of each pixel in the target;

[0017] The feature extraction module is used to call the first model to obtain at least one first aggregated annotation result based on the first pixel annotation result, wherein each first aggregated annotation result is used to indicate the first feature related to the first classification of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the first aggregated annotation result;

[0018] The second annotation module is used to invoke the second model to obtain at least one second aggregated annotation result based on the target image and the at least one first aggregated annotation result. Each second aggregated annotation result is used to indicate the second feature of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to that second aggregated annotation result; and...

[0019] The segmentation module is used to obtain the segmentation result of the target image based on the at least one second aggregation annotation result.

[0020] In some embodiments, the image segmentation apparatus further includes a training module for: training an initial first model and an initial second model based on a training sample set to obtain the first model and the second model. The training sample set includes at least one training image, and each training image has a preset aggregation annotation result and a preset segmentation result.

[0021] Another aspect of this application provides an electronic device that can be used to implement the aforementioned image segmentation method. In some embodiments, the electronic device includes a processor and a memory. The memory stores instructions, and when executed by the processor, the instructions cause the electronic device to perform any of the aforementioned image segmentation methods.

[0022] Another aspect of this application provides a computer-readable storage medium. This computer-readable storage medium stores computer instructions, which, when executed by a processor, cause the computer to perform any of the aforementioned image segmentation methods.

[0023] Therefore, the image segmentation method, apparatus, electronic device, and computer-readable medium provided in this application first acquire a target image, then acquire the first pixel annotation result of the target image, subsequently call a first model to obtain at least one first aggregate annotation result based on the first pixel annotation result, then call a second model to obtain at least one second aggregate annotation result based on the target image and at least one first aggregate annotation result, and finally obtain the segmentation result of the target image based on at least one second aggregate annotation result. The first pixel annotation result indicates the first classification of each pixel in the target image, each first aggregate annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme, which are related to the first classification, and each second aggregate annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme. In the above scheme, the target image is pre-annotated before being segmented by the second model, and the pre-annotated features are extracted by the first model and used to supervise the segmentation process. Because the segmentation result is influenced by the pre-annotation, the first and second models can obtain sufficient accuracy without a large number of samples during training, thereby efficiently completing the image segmentation task. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings include:

[0025] Figure 1 A flowchart of an image segmentation method according to an embodiment of this application is shown;

[0026] Figure 2 A partial flowchart of an image segmentation method according to an embodiment of this application is shown;

[0027] Figure 3 A partial flowchart of another image segmentation method in an embodiment of this application is shown;

[0028] Figure 4 A partial flowchart of yet another image segmentation method in an embodiment of this application is shown;

[0029] Figure 5A partial flowchart of yet another image segmentation method in an embodiment of this application is shown;

[0030] Figure 6 A partial flowchart of yet another image segmentation method in an embodiment of this application is shown;

[0031] Figure 7 A partial flowchart of yet another image segmentation method in an embodiment of this application is shown;

[0032] Figure 8 A partial flowchart of yet another image segmentation method in an embodiment of this application is shown;

[0033] Figure 9 This illustration shows a schematic diagram of information flow related to the second model in an embodiment of this application;

[0034] Figure 10 This illustration shows an information flow diagram of an image segmentation method according to an embodiment of this application;

[0035] Figure 11 This illustration shows a schematic diagram of an information flow related to a preset loss in an embodiment of this application;

[0036] Figure 12 A schematic diagram of the structure of an image segmentation apparatus according to an embodiment of this application is shown; and,

[0037] Figure 13 A schematic diagram of another image segmentation apparatus in an embodiment of this application is shown. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0040] As introduced in the background, image segmentation-based processing methods have great potential in applications requiring accurate image recognition due to their high precision. However, since image segmentation typically involves pixel-level operations, existing image segmentation methods often employ unsupervised or weakly supervised approaches, relying on machine learning on massive datasets to achieve relatively ideal segmentation results. With smaller sample sizes, only time-consuming and laborious strong supervision is possible, or a sacrifice in segmentation accuracy may be necessary. Furthermore, in these image segmentation schemes, human intervention usually only occurs in the model training domain for sample annotation, without participating in the actual image segmentation task, thus wasting the experience of specialists to some extent.

[0041] To at least address the aforementioned technical problems, embodiments of this application primarily provide an image segmentation method and apparatus, electronic device, and computer-readable medium, which can accurately segment complex images using pre-annotation even with limited sample size.

[0042] The first aspect of this application provides an image segmentation method. In this embodiment, the method can be applied to an artificial intelligence network containing an information classification model, which can be implemented through hardware, software, or a combination of both. For example, the artificial intelligence network can exist in a hardware environment composed of servers and terminals, connected via a network, including but not limited to a local area network, a metropolitan area network, or a wide area network. In this case, the information classification model can be implemented through software running on the server, terminal, or both, or through commands embedded in the hardware of the server, terminal, or both. It should be noted that the above information classification model can also run independently entirely on the server or terminal; this embodiment does not impose any restrictions on this.

[0043] The following is combined with Figure 1 Let's further describe the above information classification method. Figure 1This is a flowchart illustrating an image segmentation method according to an embodiment of this application. For example... Figure 1 As shown, this information classification method may include the following steps:

[0044] S101. Obtain the target image.

[0045] The target image, or image to be segmented, can be obtained through multiple methods in practical applications. The specific type of the target image and the specific method of acquisition mainly depend on the specific application scenario. For example, in the medical field, when the target image is a medical image such as an ultrasound, CT scan, MRI, or gastrointestinal endoscopy image, the target image is the medical image to be segmented. The acquisition method can be direct acquisition from the corresponding medical imaging equipment, acquisition from a storage device (such as a hard disk or optical disc) storing the medical image, or acquisition from a networked medical imaging device or storage device to a local terminal. Additionally, the target image can also be an image to be segmented from other fields, such as meteorological cloud images, scanning electron microscope images, etc. It should be noted that the term "image" in this application is used in a broad sense, encompassing both two-dimensional planar images and three-dimensional stereoscopic images. This embodiment does not limit the specific type of the target image or the specific method of acquisition.

[0046] S102. Obtain the first pixel annotation result of the target image. The first pixel annotation result is used to indicate the first category of each pixel in the target image.

[0047] In this embodiment, the first classification is the classification of each pixel in the target image. For example, in medical imaging, it can be determined whether a certain area in images such as gastroscopy, colonoscopy, and ultrasound belongs to normal tissue, suspected diseased tissue, or confirmed diseased tissue, and thus the first classification of all pixels in that area is labeled as the corresponding tissue type. As another example, in meteorological cloud images, it can be determined whether extreme weather phenomena exist in the cloud image, and thus the first classification of all pixels covered by that extreme weather is labeled as the corresponding weather type. Furthermore, in ultrasonic images used for material flaw detection, it can be determined whether various areas inside the material are cracked, aged, or normal areas, and thus the first classification of all pixels in that area is labeled as the corresponding structural type.

[0048] In this embodiment, the first pixel annotation result can take various forms. For example, different types in the first category can be represented by different numerical values ​​to facilitate subsequent data processing. Taking the aforementioned medical image as an example, normal tissue, suspected lesion tissue, and confirmed lesion tissue can be represented by the numbers "0", "1", and "2", respectively. Simultaneously, the first pixel annotation result can be stored as a numerical matrix in the relevant device. The size of the numerical matrix can correspond to the size of the pixel matrix in the target image, and each element in the numerical matrix represents the first category of the corresponding pixel in the target image. Referring again to the aforementioned medical image example, if the medical image is a 700×700 pixel two-dimensional image, then the numerical matrix is ​​a 700×700 matrix containing only the elements 0, 1, and 2. Of course, the first pixel annotation result can also be a vector formed by arranging the above matrix in rows or columns, or a matrix of other sizes. It can be understood that the "pixel" in the target image in this application is a pixel in a broad sense, used to represent the smallest constituent unit of the target image. When the target image is a three-dimensional image, the pixel here can be represented as a voxel (also called a voxel element).

[0049] As can be seen, obtaining the first pixel annotation result in this step is equivalent to classifying the pixels of the image, which is pre-annotating the image before image segmentation. Typically, the first pixel annotation result can achieve low-precision segmentation, i.e., simple segmentation at the region or color block level. For example, a region of interest (ROI) containing a specific object can be selected from the image using polygons, closed curves, or closed shapes composed of straight lines and arcs. This segmentation can be implemented through manual annotation, low-precision algorithms, or a combination of both. This embodiment imposes specific limitations on the method of obtaining the first pixel annotation result, as long as the first pixel annotation result can indicate the first category of each pixel in the target image. It is understood that, due to the low precision requirement, the aforementioned manual annotation does not require general workers (such as doctors or engineers in related fields) to expend excessive effort on the target image, and the aforementioned low-precision algorithms (such as software or artificial intelligence networks) do not require excessive computational resources.

[0050] S103. Invoke the first model and obtain at least one first aggregated annotation result based on the first pixel annotation result. Each first aggregated annotation result is used to indicate the features related to the first classification of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the first aggregated annotation result.

[0051] The segmentation scheme here refers to a segmentation mode for a target image, that is, how to divide the target image into multiple pixel blocks, each pixel block being a pixel array consisting of one or more pixels. In other words, each pixel block obtained according to the segmentation scheme can be considered a component of the target image, and these pixel blocks can be seamlessly stitched together without overlap to completely reconstruct the entire target image. In some embodiments, the pixel blocks obtained by the segmentation scheme are arranged in the target image as pixel block arrays, and have the same size. Taking an 800×800 pixel image as an example, one segmentation scheme can divide it into a 400×400 pixel block array, with each pixel block being a 2×2 pixel array; another segmentation scheme can divide it into a 100×100 pixel block array, with each pixel block being an 8×8 pixel array; yet another segmentation scheme can divide it into a 150×200 pixel block array, with each pixel block being a 6×4 pixel array. At least one first aggregation annotation result implies corresponding to at least one of these segmentation schemes. In some embodiments, these segmentation schemes are different from each other, resulting in different pixel block arrays and different pixel block sizes.

[0052] The first model here can be any AI network model capable of feature extraction, and the first aggregated annotation result is the feature extracted by the first model from the first pixel annotation result. It can be understood that since the first pixel annotation result indicates the first classification of each pixel, the features indicated by the first aggregated annotation result reflect the features embodied by the first classification of multiple pixels in each pixel block. In practical applications, feature extraction methods include, but are not limited to, convolution and / or downsampling. Taking the example of dividing an 800×800 pixel image into a 400×400 pixel block array, the first pixel annotation result can be an 800×800 numerical matrix, and feature extraction can be performed on this numerical matrix using a convolution with a kernel size of 2×2 and a stride of 2×2, thereby obtaining a 400×400 feature map as the first aggregated annotation result. At this point, if another first aggregation annotation result corresponds to dividing the above 800×800 pixel image into a 100×100 pixel block array, then feature extraction can be performed on the numerical matrix with a convolution kernel size of 8×8 and a stride of 8×8, or on the 400×400 feature map with a convolution kernel size of 4×4 and a stride of 4×4, so that the resulting 100×100 feature map is used as the first aggregation annotation result.

[0053] S104, invoke the second model to obtain at least one second aggregated annotation result based on the target image and at least one first aggregated annotation result. Each second aggregated annotation result indicates the features of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to that second aggregated annotation result.

[0054] The segmentation scheme in this step is similar to that in step S103, and will not be repeated here. Similarly, like the first model, the second model here can be an artificial intelligence network model capable of at least feature extraction. The second aggregation annotation result is the feature extracted from the target image by the second model. It can be understood that since the target image is composed of pixels or pixel blocks, the features indicated by the second aggregation annotation result reflect, to some extent, the features of the pixels or pixel blocks themselves. Furthermore, since the first aggregation annotation result reflects the features of each pixel's first classification at the pixel block level, the features indicated by the second aggregation annotation result are also influenced by the first classification of the pixels in the corresponding pixel block. The second aggregation annotation result in this step serves as the basis for obtaining the target image segmentation result in subsequent steps. The features indicated by the second aggregation annotation result can be understood as the basis for pixel-level or pixel block-level segmentation of the image. Therefore, the first classification indicated in the first pixel annotation result also affects the final segmentation result.

[0055] S105. Based on the at least one second aggregation annotation result, obtain the segmentation result of the target image.

[0056] This step involves the specific steps of segmentation, typically manifested as the classification of pixels or pixel blocks. Compared to the pre-classification in step S102, the classification granularity in this step is smaller and the accuracy is higher, often requiring the assistance of an artificial intelligence network model. This artificial intelligence network can be embedded within the second model called in step S104, or it can be a separate model outside of the second model; this application does not impose any restrictions on this.

[0057] As described in step S104, the second aggregation annotation result may indicate pixel-level features. Therefore, if a finer-grained (i.e., smaller pixel blocks or pixel-level) image segmentation is desired in this step, it is necessary to utilize the features of the pixel blocks to obtain the features of the pixels they contain or even smaller pixel blocks. Obtaining the segmentation result of the target image from the second aggregation annotation result can be accomplished in various ways. In practical applications, this method includes, but is not limited to, deconvolution or upsampling. Taking an 800×800 pixel image as an example, if the pixel-level segmentation result of the target image is obtained based on the second aggregation annotation result, which is represented as a 100×100 feature map, it is necessary to use deconvolution or upsampling to convert the 100×100 feature matrix into an 800×800 feature matrix, and then determine the final segmentation result based on the element values ​​corresponding to each pixel. It can be understood that if the second aggregation annotation result used to obtain the segmentation result itself reflects pixel-level features (such as an 800×800 feature map), the final segmentation result can be obtained directly from the feature map according to the preset segmentation rules without deconvolution or upsampling. In some embodiments, a separate convolutional layer with a kernel of 1×1 and a stride of 1×1 can be set up to perform the final processing of the features.

[0058] The segmentation result can be understood as obtaining the final segmentation scheme for the target pattern. It should be noted that the segmentation rules used to obtain the final segmentation result can take many forms, such as segmentation based on the pixel feature values ​​themselves or the range of the feature values. This embodiment does not limit this and can be determined according to the specific application scenario.

[0059] Therefore, in the image segmentation method provided in this application embodiment, the target image is first acquired, then the first pixel annotation result of the target image is acquired, followed by the invocation of a first model to obtain at least one first aggregated annotation result based on the first pixel annotation result, then the invocation of a second model to obtain at least one second aggregated annotation result based on the target image and at least one first aggregated annotation result, and finally the segmentation result of the target image is obtained based on at least one second aggregated annotation result. The first pixel annotation result indicates the first classification of each pixel in the target image, each first aggregated annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme, which are related to the first classification, and each second aggregated annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme. In the above scheme, the target image is pre-annotated before being segmented by the second model, and the pre-annotated features are extracted by the first model and used to supervise the segmentation process. Because the segmentation result is influenced by the pre-annotation, the first and second models can obtain sufficient accuracy without a large number of samples during training, thereby efficiently completing the image segmentation task.

[0060] The following Figure 1 Based on the illustrated embodiments, combined with Figures 2 to 8 The details of the image segmentation method in the embodiments of this application will be further explained.

[0061] Please refer to Figure 2 In some embodiments, step S102 includes the following steps:

[0062] S1021. Divide the target image into multiple target sub-images.

[0063] It's understandable that each target sub-image here is a part of the target image, and all target sub-images can be seamlessly stitched together without overlap to reconstruct the entire target image. Segmenting the target image into multiple target sub-images is primarily for facilitating the acquisition of the first pixel annotation result. For example, a medical image can be segmented into several parts and provided to several physicians for annotation to improve annotation efficiency, or it can be provided to the same algorithm multiple times for annotation to reduce the instantaneous consumption of computational resources. The segmentation scheme for obtaining multiple target sub-images can be chosen based on the actual situation. Taking a medical image as the target image again, it can be segmented into multiple regions based on the physician's work habits. Regions with more details or more complex structures contain fewer pixels, while regions with fewer details or simpler structures contain more pixels. Alternatively, sub-images of appropriate sizes can be segmented directly based on the physician's terminal screen, ensuring that the terminal screen clearly displays the details that need to be focused on in the sub-image.

[0064] Step S1022: Enlarge each target sub-image to obtain the corresponding extended sub-image.

[0065] It's understandable that an expanded target sub-image has more pixels than its corresponding counterpart. For example, scaling up a 200×200 pixel target sub-image by a factor of 4×4 yields an 800×800 expanded sub-image. The primary purpose of scaling up the target sub-image is to more clearly present the details contained within the target image, thereby obtaining more accurate first-pixel annotation results. In practical applications, the process of scaling up the target sub-image may include, but is not limited to, interpolation operations.

[0066] In some embodiments, each target sub-image has the same magnification. This allows multiple extended sub-images to be stitched together to reconstruct an enlarged but undistorted target image. In other embodiments, the magnification of each target sub-image depends on the level of detail or complexity it contains. For example, target sub-images containing more detail or complex structures can be set with a higher magnification.

[0067] In some embodiments, each target sub-image has the same size. That is, in step S1011, the target image is divided into several parts of the same size. For example, the target image can be divided into four sub-images of the same size using a 2×2 method, or into six sub-images of the same size using a 2×3 method. It is understood that if the magnification is the same, the resulting extended sub-images will also have the same size.

[0068] Step S1023: Obtain the second pixel annotation result for each extended sub-image. The second pixel annotation result indicates the second classification of each pixel in the extended sub-image, and the first pixel annotation result is composed of the second pixel annotation result for each extended sub-image.

[0069] The second pixel annotation result obtained in this step is the pre-annotation process described in S103 above. Details can be found in step S103 above, and will not be repeated here. The first pixel annotation result includes the second pixel annotation result corresponding to each extended sub-image. For example, if the magnification of each target sub-image is 4×4, then the extended sub-images can be stitched together to obtain a target image with a magnification of 4×4. In this case, if all the second pixel annotation results are represented as a numerical matrix corresponding to the pixel size, then the first pixel annotation result can be understood as using a 4×4 numerical matrix to indicate the first category of each pixel in the target image, where each element in the first category is a numerical value representing the second category. Of course, if multiple target sub-images have different magnifications, such as in the example above where one target sub-image has a magnification of 2×3, then in the region corresponding to that target sub-image, the first pixel annotation result uses a 2×3 numerical matrix to indicate the first category of each pixel in the target image.

[0070] Similar to step S102 above, step S1023 can be implemented through manual annotation or algorithmic annotation. Please refer to [link / reference]. Figure 3 In some embodiments, step S1023 includes the following steps:

[0071] S10231. Manually annotate each extended sub-image to obtain at least one manually annotated region.

[0072] Manual annotation allows for better utilization of staff experience in the pre-annotation process, thereby strengthening control and leading to higher accuracy in supervised image segmentation. Manual annotation can be implemented in various ways. For example, 2n staff members (e.g., 6, 8, 10, etc.) can be selected to perform manual annotation in a "back-to-back" manner. These 2n staff members are randomly divided into n groups (i.e., two people per group). Simultaneously, the target image is divided into n equal parts, and these parts are randomly assigned to each group for manual annotation. After the bounding boxes are manually annotated, the annotated regions obtained by each group of two staff members are compared, and the consistency between the two annotated regions is evaluated to determine the final manually annotated regions.

[0073] Typically, manually labeled areas are closed areas enclosed by lines drawn by staff or by selecting areas within a frame. They can also be closed areas formed by lines drawn by staff and the image boundary.

[0074] S10232. Based on the at least one manually labeled region, obtain a second pixel labeling result, wherein the second category of each pixel in each manually labeled region is the same.

[0075] In this embodiment, the manually labeled regions serve as the basis for obtaining the second classification of each pixel in the extended sub-image. For example, if the staff obtains two different manually labeled regions, the second classification of pixels in one manually labeled region can be marked with the value "1", the second classification of pixels in the other manually labeled region can be marked with the value "2", and pixels outside the two manually labeled regions can be marked with the value "0".

[0076] Regarding the aforementioned case of segmenting a target image into target sub-images, please... Figure 2 Based on the embodiments shown, refer to Figure 4 In some embodiments, for each first aggregation annotation result, step S103 includes the following steps:

[0077] S1031. Call the first model to process the second pixel annotation results corresponding to each extended sub-image, and obtain the corresponding third aggregate annotation results.

[0078] The process of obtaining the third aggregated annotation result from the second pixel annotation result can be referred to the process of obtaining the first aggregated annotation result from the first pixel annotation result in step S103 above, and will not be repeated here. In this step, the first model is equivalent to processing each extended sub-image "separately". The processing of each extended sub-image can be performed in time-sharing, which helps to reduce the instantaneous occupation of computing resources in step S103.

[0079] S1032. The third aggregation annotation results are concatenated in a first order to obtain the first aggregation annotation result. The first order is determined by the spatial order of the multiple target sub-images in the target image.

[0080] If the third aggregation annotation result is represented as a matrix or vector as described above, then the concatenation mentioned here can be based on matrices or vectors. For example, six 100×100 matrices can be concatenated into a 200×300 matrix using a 2×3 array. Multiple target sub-images correspond one-to-one with multiple extended target sub-images, and therefore also one-to-one with multiple third aggregation annotation results. The spatial order of multiple targets in the target image determines the concatenation order of their respective third aggregation annotation results, ensuring that the content of the first aggregation annotation result obtained after concatenation strictly corresponds to the pixel structure in the target image. For example, if multiple target sub-images are arranged in a "top left-bottom right" order in the target image, then their respective third aggregation annotation results are still concatenated in a "top left-bottom right" order to obtain the first aggregation annotation result.

[0081] It should be noted that if different magnification factors are used for each target sub-image in the aforementioned step S1022, then the second pixel annotation results of each expanded sub-image need to be extracted with different density features (such as using convolution kernels of different sizes) so that the final third aggregated annotation results can be stitched together in step S1032.

[0082] Steps S1031 and S1032 described above can be considered as a process of "feature extraction first, then concatenation". In some other embodiments, a process of "concatenation first, then feature extraction" can also be used. Please refer to [link / reference]. Figure 5 In some embodiments, step S103 includes the following steps:

[0083] S1033. The second pixel annotation results corresponding to the multiple extended sub-images are concatenated in a second order to obtain the extended pixel annotation results. The second order is determined by the spatial order of the multiple target sub-images in the target image.

[0084] S1034. Call the first model and obtain at least one first aggregated annotation result based on the extended pixel annotation result.

[0085] Unlike the previous S1031, S1034 is equivalent to calling the first model to extract features for the first classification of all pixels in the entire target image. It does not require inputting the first model multiple times, thus contributing to the computation speed of the feature step S103 to some extent.

[0086] It should be noted that if different magnification factors are used for the target sub-images in step S1022, step S1033 requires scaling each extended sub-image first so that the scaled extended sub-images can be stitched together. For example, all scaled extended sub-images can be made to have the same magnification factor relative to the original target sub-image. In this process, the second classification of each pixel in the reduced extended sub-image can be obtained by numerical pooling of the second classification of the corresponding pixel in the original extended sub-image, and the second classification of each pixel in the magnified extended sub-image can be obtained by interpolation or copying the second classification of the corresponding pixel in the original extended sub-image.

[0087] In some embodiments, at least one first aggregated annotation result can be obtained through stepwise feature extraction in step S103. That is, feature extraction can be performed on the obtained first aggregated annotation result to obtain a new first aggregated annotation result. Please refer to... Figure 6 In one embodiment, there are K first aggregation annotation results, and step S103 includes the following steps:

[0088] S1035. Call the first model and obtain the first first aggregated annotation result based on the first pixel annotation result.

[0089] S1036. Based on the kth first aggregation annotation result, obtain the (k+1)th first aggregation annotation result. Here, k is initially set to 1.

[0090] S1037. Determine if k is equal to K. If yes, proceed to step S104. Otherwise, execute k=k+1 and return to step S1036.

[0091] In this embodiment, the first first aggregated annotation result is used to represent the first obtained first aggregated annotation result, the second first aggregated annotation result is used to describe the second obtained first aggregated annotation result, and so on, the Kth first aggregated annotation result is used to represent the Kth obtained first aggregated annotation result.

[0092] It is understandable that steps S1036 and S1037 are used to achieve the following: for any 1 ≤ k < K, obtain the (k+1)th first aggregate annotation result based on the kth first aggregate annotation result. The above effect can also be achieved through other logic, such as first executing k=k+1 in step S1037, then determining whether k equals K+1; if so, execute step S104; otherwise, return to step S1036. This embodiment is not limited to the above logic; any logic that achieves the above effect is acceptable.

[0093] In this embodiment, the stepwise feature extraction can establish a hierarchical relationship between the first aggregated annotation results, so that at least one first aggregated result can reflect the features of pixel blocks of different granularities in the target image, while minimizing the computational complexity when the first model is used.

[0094] Furthermore, in some embodiments, for any 1 ≤ k < K, each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the kth first aggregation annotation result includes multiple pixel blocks obtained by segmenting the target image according to the segmentation scheme corresponding to the (k+1)th first aggregation annotation result. Correspondingly, each step in the stepwise feature extraction may include, but is not limited to, convolution or downsampling operations.

[0095] In some embodiments, at least one second aggregated annotation result corresponds one-to-one with at least one first aggregated annotation result, and the segmentation scheme corresponding to each first aggregated annotation result is the same as the segmentation scheme corresponding to the second aggregated annotation result corresponding to the first aggregated annotation result. That is, for each pair of corresponding second aggregated annotation results and first aggregated annotation results, the features they represent are for the same set of pixel block arrays. In other words, the second aggregated annotation result represents one feature of each pixel block in the set of pixel blocks, and the corresponding first aggregated annotation result represents another feature of each pixel block in the set of pixel blocks that is related to the first classification of each pixel. Therefore, in some embodiments, in order to facilitate the use of the first aggregated annotation result to supervise the second aggregated annotation result, thereby realizing the supervision of the final segmentation result by the first pixel annotation result, step S104 includes: obtaining the corresponding second aggregated annotation result based on the first aggregated annotation result.

[0096] Taking the example where the number of the first aggregated annotation results equals K, when there is a one-to-one correspondence between at least one second aggregated annotation result and at least one first aggregated annotation result, the number of second aggregated annotation results also equals K. Please... Figure 6 Further reference Figure 7 In some embodiments, step S104 may include the following steps:

[0097] S1041. Call the second model to obtain the first second aggregation annotation result based on the target image and the Kth first aggregation annotation result.

[0098] S1042. Based on the k-th second aggregation labeling result and the (K-k+1)-th first aggregation result, obtain the (k+1)-th second aggregation labeling result. Before executing S1042, let the initial value of k be 1.

[0099] S1043. Determine if k is equal to K. If yes, proceed to step S105. Otherwise, execute k=k+1 and return to step S1042.

[0100] In this embodiment, the first second aggregation annotation result represents the first obtained second aggregation annotation result, the second second aggregation annotation result represents the second obtained first aggregation annotation result, and so on, with the Kth second aggregation annotation result representing the Kth obtained second aggregation annotation result. It can be understood that in this embodiment, the kth second aggregation annotation result corresponds to the (K-k+1)th first aggregation result.

[0101] Similar to the aforementioned embodiments, steps S1042 and S1043 are used to achieve the following: for any 1 ≤ k < K, obtain the (k+1)th second aggregation labeling result based on the kth second aggregation labeling result and the (K-k+1)th first aggregation result. The above effect can also be achieved through other logic, such as first executing k=k+1 in step S1043, then determining whether k equals K+1; if yes, then executing step S105; otherwise, returning to step S1042. This embodiment is not limited to the above logic; any logic that achieves the above effect is acceptable.

[0102] In this embodiment, the K first aggregated annotation results and the K second aggregated annotation results correspond in reverse order of the acquired words. While the first model extracts features from small to large granularity, the second model generates the second aggregated annotation results from large to small granularity, that is, gradually generating features with deeper details. This progressively refined annotation helps to ultimately achieve image segmentation at the small granularity (e.g., pixel) level.

[0103] Since the (k+1)th second aggregation annotation result depends on the kth second aggregation annotation result, the generation of the Kth second aggregation annotation results in this embodiment is also gradual. The later generated small-granularity second aggregation annotation results can include all the previously generated larger-granularity second aggregation annotation results. Therefore, when obtaining the final segmentation result, only the kth second aggregation annotation result needs to be used, which to some extent makes the operation in step S105 simpler. In one embodiment, based on the above embodiment, step S105 may include the following steps:

[0104] S1051. Based on the Kth second aggregation annotation result, obtain the segmentation result of the target image.

[0105] Similar to the manual annotation in the aforementioned pre-annotation, the segmentation result in step S105 can be represented as multiple regions in the target image. Please refer to... Figure 8 In some embodiments, step S105 includes the following steps:

[0106] S1052. Based on at least one second aggregation annotation result, obtain the third pixel annotation result of the target image. The third pixel annotation result is used for the third classification of each pixel in the target image.

[0107] In other words, step S105 first obtains pixel-level classification results. It's understandable that while the third classification for each pixel and the aforementioned first classification both pertain to each pixel in the target image, they may be the same or different. Taking an organ image as an example, pre-annotation might classify each pixel into three categories: "non-lesion," "suspected lesion," and "definite lesion." The segmentation process might still label each pixel as one of these three categories, but due to improved accuracy or finer detail (e.g., pre-annotation is only a rough manual selection), some pixels selected as one type during pre-annotation might be labeled as another type during segmentation. Alternatively, pre-annotation might further refine the classification of each pixel into four categories: "lesion probability less than 5%," "lesion probability between 5% and 50%," "lesion probability between 50% and 90%," and "lesion probability between 90% and 100%." ​​The segmentation process would then label each pixel as a completely different type compared to the pre-annotation. Of course, in some embodiments, the type during pre-annotation might be partially the same as the type labeled during final segmentation.

[0108] S1053. Obtain the segmentation result based on the third pixel annotation result. The segmentation result includes at least one segmented region, which is composed of multiple consecutive pixels in the target image that belong to the same third category.

[0109] After performing step S1053, the segmentation results can be visualized. For example, different segmented regions can be displayed on the screen with different colors, gray levels, textures, etc., or dividing lines can be displayed between different segmented regions. Alternatively, different segmented regions can be displayed separately.

[0110] The following mainly introduces the architecture of the first and second models.

[0111] In some embodiments, the first model comprises multiple cascaded convolutional layers. At least one first aggregated annotation result corresponds one-to-one with at least one of the multiple convolutional layers, and each first aggregated annotation result is output by its corresponding convolutional layer.

[0112] The convolutional layers here primarily perform feature extraction, and the first aggregated annotation result can be used as the output of the convolutional layers. In some scenarios, in addition to multiple convolutional layers, the first model can also contain other functional structures, such as downsampling layers or pooling layers, etc.

[0113] In some embodiments, the second model includes a first sub-model and a second sub-model. Correspondingly, step S104 includes the following steps:

[0114] S1044. Call the first sub-model to obtain intermediate results based on the target image.

[0115] S1045. Call the second sub-model and obtain at least one second aggregated annotation result based on the intermediate results and at least one first aggregated annotation result.

[0116] In other words, in step S104, the target image is first processed using the first sub-model, and then the processing result and the first aggregated annotation result are input into the second sub-model to obtain the second aggregated annotation result.

[0117] In some embodiments, the second model may employ a structure including a contraction path and an expansion path. The contraction path is used to progressively extract features from the target image from small to large granularity, while the expansion path is used to progressively combine the extracted features with details in the target image from large to small granularity, thereby obtaining classification results for various details in the target image at multiple granularities. Please refer to [reference needed]. Figure 9 In some embodiments, the first sub-model and the second sub-model are the contraction path and the expansion path, respectively, thus forming a U-shaped network structure. Specifically, the first sub-model contains M cascaded contraction modules, the second sub-model contains M cascaded expansion modules, and the intermediate result is the output of the Mth contraction module. In this case, step S1045 may include the following steps:

[0118] S10451. Call the first extension module and obtain the output of the first extension module based on the output of the Mth shrink module.

[0119] S10452. For any 1 ≤ m < M, perform a concatenation operation on the output of the Mm-th shrinking module and the output of the m-th expanding module to obtain the concatenation result corresponding to the m-th expanding module, and call the (m+1)-th expanding module to obtain the output of the (m+1)-th expanding module based on the concatenation result.

[0120] When the at least one extension module includes the Mth extension module, step S1045 may further include the following steps:

[0121] S10453. Perform a fusion operation on the output corresponding to the Mth extension module and the first aggregation annotation result corresponding to the Mth extension module to obtain the second fusion result.

[0122] The at least one second aggregation annotation result includes the second fusion result and the output of the (m+1)th expansion module.

[0123] The shrinking module may contain a downsampling layer and / or at least one first convolutional layer. From Figure 9 As can be seen, the target image is first input into the first shrinking module, which may contain one or more convolutional layers to extract features from the target image. For example, when the target image is 800×800 pixels in size, the first shrinking module may contain two cascaded convolutional layers with a kernel size of 3×3 and a stride of 1×1. The first convolutional layer outputs a feature map of size 798×798, and the second convolutional layer outputs a feature map of size 796×796. Then, the output of the first shrinking module is input into the second shrinking module. Further, the second shrinking module may contain a downsampling layer that performs 2×2 pooling, and two cascaded convolutional layers with a kernel size of 3×3 and a stride of 1×1. The downsampling layer inputs a feature map of size 398×398, the first convolutional layer outputs a feature map of size 396×396, and the second convolutional layer outputs a feature map of size 394×394. Then, the output of the second shrinking module is input into the third shrinking module. This process continues until the output of the Mth shrinking module is obtained, which serves as an intermediate result.

[0124] The extension module may contain an upsampling layer and / or at least one first convolutional layer. From Figure 9 As can be seen, the intermediate results are first input into the first expansion module. This contraction module may contain one or more convolutional layers and / or an upsampling layer to extract features and / or enlarge the size of the intermediate results. For example, when the intermediate result is a 100×100 feature map, the first contraction module may contain two cascaded convolutional layers with 3×3 kernels and a stride of 1×1, and an upsampling layer for 2×2 upsampling. In this case, the first convolutional layer outputs a feature map of size 98×98, the second convolutional layer outputs a feature map of size 96×96, and the upsampling layer outputs a feature map of size 192×192. Then, the output of the first expansion module is concatenated with the output of the (M-1)th contraction module, and the concatenated result is directly input into the second expansion module or processed before being input into the second expansion module.

[0125] It's important to note that the stitching mentioned here and later can be implemented in various ways. Typically, this stitching can be done on a channel basis. For example, if the output of the first expansion module is a feature map of size 192×192 with 32 channels, and the output of the (M-1)th contraction module is also a feature map of size 192×192 with 32 channels, then channel stitching will result in a feature map of size 192×192 with 64 channels. It can be understood that if the objects to be stitched have different sizes, the larger feature map can first be cropped to obtain the same size as the smaller feature map, and then the two can be stitched together.

[0126] Furthermore, the second expansion module may contain two cascaded convolutional layers with 3×3 kernels and a stride of 1×1, and an upsampling layer for 2×2 upsampling. Similarly, if the input to the second expansion module is a feature map of size 192×192, its output will be a feature map of size 376×376. This process continues until the output of the (M-1)th contraction module is obtained, which is then input into the Mth contraction module. The Mth contraction module may or may not contain an upsampling module.

[0127] Based on the above process, the first aggregated annotation result is input into the second sub-model to obtain the second aggregated annotation result. In some embodiments, the first aggregated annotation result can be used to adjust the output of at least one of the M expansion modules to obtain the corresponding second aggregated annotation result. The obtained second aggregated annotation result is used as the input of the next-level expansion module (e.g., when the corresponding expansion module is the 1st to the (M-1)th expansion module), or directly used to obtain the final segmentation result (e.g., when the corresponding expansion module is the Mth expansion module).

[0128] Specifically, in one embodiment, at least one first aggregation annotation result corresponds one-to-one with at least one of the multiple extended modules. When at least one extended module contains the m-th extended module, step S10452, which obtains the output of the (m+1)-th extended module based on the splicing result, includes the following steps:

[0129] S1. Perform a fusion operation between the splicing result corresponding to the extension module and the first aggregation annotation result corresponding to the extension module to obtain the first fusion result.

[0130] In some embodiments, the splicing result corresponding to the extension module and the first aggregation annotation result corresponding to the extension module are matrices or vectors of the same size, and the fusion operation is a dot product operation.

[0131] In other embodiments, if the concatenation result corresponding to the extension module and the first aggregation annotation result corresponding to the extension module are matrices or vectors of different sizes, the fusion operation first upsamples the smaller matrix or vector or downsamples the larger matrix or vector to make their sizes consistent. Then, operations such as dot product are performed on the two.

[0132] S2. Call the (m+1)th extension module and obtain the output of the (m+1)th extension module based on the fusion result.

[0133] In the above embodiments, the fusion operation is equivalent to applying an attention mechanism to the output of the corresponding extended module, so that the pre-labeled results can intervene in the segmentation results of the second model.

[0134] Please refer to Figure 10 This illustrates an example of information flow in the technical solution of this application. Figure 10 In this process, the target image undergoes segmentation, expansion, and manual annotation to obtain the first pixel annotation result, which is then input into the first model. Simultaneously, it is input into the first sub-model of the second model for feature extraction. In this example, the first model contains five cascaded convolutional layers 1-5, whose outputs correspond to features related to the first classification of pixels in a pixel block array of five granularities, from smallest to largest. The outputs of convolutional layers 2 to 5 serve as the first aggregated annotation results 1-4, which are input into the second sub-model of the second model. Simultaneously, the first sub-model of the second model contains five cascaded contraction modules 1-5, whose outputs correspond to the feature maps of the pixel block array of five granularities, from smallest to largest. The second model also contains five corresponding cascaded expansion modules 1-5, whose reverse order (i.e., from largest to smallest granularity) corresponds to these five granularity pixel block arrays. In this example, the output of contraction module 4 is concatenated with the output of expansion module 1, and then directly input into expansion module 2; the outputs of contraction modules 3, 2, and 1 are concatenated with the outputs of expansion modules 2, 3, and 4, respectively, and then fused with the first aggregated annotation results 4, 3, and 2, respectively, before being directly input into expansion modules 3, 4, and 5; the output of expansion module 5 is fused with the first aggregated result 1 to obtain the second pixel annotation result. Finally, the final pixel-level segmentation result is obtained based on the second pixel annotation result.

[0135] In the above embodiments, the first model and the second model used by the image segmentation method can be preset or trained models. When the first model and / or the second model has not been trained, the image segmentation method further includes a training step:

[0136] S100. Train the initial first model and the initial second model based on the training sample set to obtain the first model and the second model. The training sample set contains at least one training image, and each training image has a preset aggregation annotation result and a preset segmentation result.

[0137] The preset aggregated annotation result here is used to indicate the features of each pixel block obtained by segmenting the training image according to the preset segmentation scheme corresponding to the first aggregated annotation result, which are related to the aforementioned first classification. This preset aggregated annotation result can be directly used as part of the training sample set annotation, or it can be stored as part of the training sample set annotation in the form of preset pixel annotations. The preset pixel annotations here are used to indicate the first classification of each pixel in the training image. The process of obtaining the preset aggregated annotation result from the preset pixel annotations can refer to the aforementioned process of obtaining the first aggregated annotation result from the first pixel annotation result, and will not be repeated here.

[0138] The preset segmentation result here is used to indicate how to segment the target image. For example, the preset segmentation result may contain one or more segmentation regions containing consecutive pixels.

[0139] In some embodiments, step S100 includes the following steps:

[0140] S1001. Adjust the parameters of the initial first model and the initial second model until the target loss meets the preset conditions.

[0141] The adjustments here are usually iterative, and the preset conditions can take many forms. For example, the preset conditions may include one or more of the following: the target loss decreases to a first threshold, the number of iterations of the target loss reaches a second threshold, the descent gradient of the target loss does not exceed a third threshold within a preset number of iterations, etc.

[0142] In some embodiments, the process of obtaining the target loss is a preset process similar to the aforementioned image segmentation process. Please refer to... Figure 11 The information flow in the training sample set, specifically, for each training image in step S1001, includes the following steps for calculating the target loss:

[0143] S201. Obtain the predicted pixel annotation result of the training image. The predicted pixel annotation result is used to indicate the prediction result of the first classification for each pixel in the training image.

[0144] S202. Invoke the initial first model and obtain at least one predicted first aggregated annotation result based on the predicted pixel annotation results. Each predicted first aggregated annotation result indicates the predicted feature related to the first classification for each pixel block obtained by segmenting the training image according to the segmentation scheme corresponding to the corresponding first aggregated annotation result.

[0145] It is understandable that the initial first model here refers to the untrained first model. At least one predicted first aggregated annotation result corresponds one-to-one with the aforementioned at least one first aggregated annotation result, and each predicted first annotation result is the predicted value of the corresponding first aggregated annotation result.

[0146] S203. Invoke the initial second model and obtain at least one predicted second aggregated annotation result based on the training image and at least one predicted first aggregated annotation result. Each predicted second aggregated annotation result indicates the predicted feature result of each pixel block obtained by segmenting the training image according to the segmentation scheme corresponding to the corresponding second aggregated annotation result.

[0147] It is understandable that the initial second model here is the untrained second model. At least one predicted second aggregated annotation result corresponds one-to-one with the aforementioned at least one second aggregated annotation result, and each predicted first annotation result is the predicted value of the corresponding second aggregated annotation result.

[0148] S204. Based on at least one predicted second aggregated annotation result, obtain the predicted segmentation result of the training image.

[0149] It can be understood that the predicted segmentation result here is the predicted value of the aforementioned segmentation result.

[0150] Steps S201 to S204 process each training image to obtain its corresponding prediction aggregation annotation result and prediction segmentation result. The details of steps S201 to S204 can be referred to the processing of the target image in steps S102 to S105 above, and will not be repeated here.

[0151] S205. Based on the difference between the predicted first aggregated annotation result and the preset aggregated annotation result for each training image, obtain the classification loss.

[0152] It is understandable that the classification loss here is used to characterize the accuracy of predicting the pre-labeled results. The smaller the classification loss, the more accurate the initial first model is in extracting features for predicting the pixel classification results. In some embodiments, the classification loss can be obtained by calculating the first aggregated labeling result and the preset aggregated labeling result in their corresponding vector spaces.

[0153] S206. Obtain the segmentation loss based on the difference between the predicted segmentation results of each training image and the preset segmentation results of each training image.

[0154] It is understandable that the classification loss here is used to characterize the accuracy of the final image segmentation. The smaller the segmentation loss, the more accurate the segmentation of the image obtained by the initial first model and the initial second model. In some embodiments, the segmentation loss can be calculated by the overlap area of ​​the corresponding segmented regions in the predicted segmentation result and the preset segmentation result.

[0155] S207. Obtain the target loss based on the classification loss and segmentation loss.

[0156] In this embodiment, the specific formula for deriving the target loss from the classification loss and segmentation loss is not limited, as long as the target loss is positively correlated with both. For example, the target loss can be the sum, product, or weighted sum of the classification loss and segmentation loss. The main purpose of using classification loss to calculate the target loss is to ensure that the pre-labeling constrains the final segmentation loss. Without classification loss, the first model after training may try to eliminate the influence of pre-labeling as much as possible to obtain a smaller segmentation loss, which would result in the final image segmentation not being able to effectively utilize pre-labeling for intervention and supervision.

[0157] In one specific application of this application, both the training image and the target image are derived from gastroscopy images. The segmentation target is to obtain positive regions belonging to tumorous lesions and negative regions not belonging to tumorous lesions from the gastroscopy images. A mini-batch gradient descent method is employed, which generates a mini-batch containing multiple positive candidate regions (positive samples) and negative candidate regions (negative samples) for each training image. The learning rate for the first 50,000 mini-batches is set to 0.001, and the learning rate for the next 50,000 mini-batches is set to 0.0001; the momentum term is preferably set to 0.9, and the weight decay is preferably set to 0.0005. The loss function used during training is Loss = Lcls + Lseg, where Lcls is the classification loss and Lseg is the segmentation loss.

[0158] In this application scenario, a test dataset (including 50 images containing neoplastic lesions and 50 images of the stomach without neoplastic lesions) was used to test the artificial intelligence system and gastroenterologists of different experience levels. The sensitivity, specificity, accuracy, and consistency of the two systems in diagnosis were compared and evaluated, and statistical analysis was performed. During the test, the classification score of the trained deep learning network (containing both the first and second models) for identifying target lesions in endoscopic images was set to 0.85. This means that the deep learning network only identified lesions with a probability exceeding 85% as cancerous; conversely, if no suspicious lesions were detected in the image, the entire image was not identified. Based on the platform of the National Clinical Research Center for Digestive Diseases, 89 physicians participated in the test as a control group for the trained network. The test results showed that the sensitivity of the trained network was significantly higher than that of the physician group, while its specificity and accuracy were only slightly lower. Furthermore, it significantly outperformed the physician group in processing speed. Therefore, using artificial intelligence to identify lesions can effectively eliminate diagnostic biases caused by individual physician differences, thus demonstrating promising application prospects. Sensitivity, also known as the true positive rate, is the percentage of individuals who are actually infected but are correctly diagnosed by diagnostic criteria. Specificity, also known as the true negative rate, reflects the ability of a screening test to identify non-patients. Accuracy is the ratio of the total number of correctly identified individuals to the total number of identified individuals.

[0159] A second aspect of this application provides an image segmentation apparatus. Please refer to... Figure 12 In some embodiments, the image segmentation device includes an acquisition module 1201, a first annotation module 1202, a feature extraction module 1203, a second annotation module 1204, and a segmentation module 1205.

[0160] The acquisition module 1201 is used to acquire the target image.

[0161] The first annotation module 1202 is used to obtain the first pixel annotation result for each target. The first pixel annotation result is used to indicate the first category of each pixel in the target.

[0162] The feature extraction module 1203 is used to call the first model to obtain at least one first aggregated annotation result based on the first pixel annotation result. Each first aggregated annotation result is used to indicate the first feature related to the first classification of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the first aggregated annotation result.

[0163] The second annotation module 1204 is used to invoke the second model to obtain at least one second aggregated annotation result based on the target image and the at least one first aggregated annotation result. Each second aggregated annotation result is used to indicate the second feature of each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to that second aggregated annotation result.

[0164] The segmentation module 1205 is used to obtain the segmentation result of the target image based on at least one second aggregation annotation result.

[0165] In some embodiments, the first annotation module 1202 includes a segmentation submodule, a magnification submodule, and an acquisition submodule.

[0166] The segmentation submodule is used to segment a target image into multiple target sub-images.

[0167] The magnification submodule is used to magnify each target sub-image to obtain the corresponding extended sub-image. Each target sub-image has the same magnification ratio.

[0168] The acquisition submodule is used to obtain the second pixel annotation result for each expanded sub-image. The second pixel annotation result indicates the second classification of each pixel in the expanded sub-image.

[0169] The first pixel annotation result is composed of the second pixel annotation result of each extended sub-image.

[0170] In some embodiments, each target sub-image has the same size.

[0171] In some embodiments, the acquisition submodule includes manually labeled units and partitioned units.

[0172] The manual annotation unit is used to obtain manual annotations for each extended sub-image, thereby obtaining at least one manually annotated region.

[0173] The partitioning unit is used to obtain the second pixel annotation result based on at least one manually annotated region. In each manually annotated region, the second classification of each pixel is the same.

[0174] In some embodiments, the feature extraction module 1203 includes a first processing submodule and a first splicing submodule.

[0175] For each first aggregated annotation result, the first processing submodule is used to call the first model to process the second pixel annotation results corresponding to each extended sub-image, and obtain the corresponding third aggregated annotation results.

[0176] For each first aggregated annotation result, the first stitching submodule is used to stitch together the various third aggregated annotation results in a first order to obtain the first aggregated annotation result. The first order is determined by the spatial order of the multiple target sub-images within the target image.

[0177] In some embodiments, the feature extraction module 1203 includes a second splicing submodule and a second processing submodule.

[0178] The second stitching submodule is used to stitch together the first pixel annotation results corresponding to multiple extended sub-images in a second order to obtain extended pixel annotation results. The second order is determined by the spatial order of the multiple target sub-images in the target image.

[0179] The second processing submodule is used to call the first model and obtain at least one first aggregated annotation result based on the extended pixel annotation result.

[0180] In some embodiments, at least one second aggregation annotation result corresponds one-to-one with at least one first aggregation annotation result, and the segmentation scheme corresponding to each first aggregation result is the same as the segmentation scheme corresponding to the second aggregation result corresponding to the first aggregation result.

[0181] In some embodiments, the feature extraction module 1203 is specifically used to: call the first model to obtain the first first aggregated annotation result based on the first pixel annotation result; and, for any 1 ≤ k < K, obtain the (k+1)th first aggregated annotation result based on the kth first aggregated annotation result.

[0182] The number of the first aggregation annotation results is K, where k and K are both positive integers.

[0183] In some embodiments, for any 1 ≤ k < K, each pixel block obtained by segmenting the target image according to the segmentation scheme corresponding to the kth first aggregation annotation result includes multiple pixel blocks obtained by segmenting the target image according to the segmentation scheme corresponding to the (k+1)th first aggregation annotation result.

[0184] In some embodiments, the second annotation module 1204 is specifically used to: call the second model to obtain the first second aggregate annotation result based on the target image and the Kth first aggregate annotation result; and, for any 1 ≤ k < K, obtain the (k+1)th second aggregate annotation result based on the kth second aggregate annotation result and the (K-k+1)th first aggregate result.

[0185] In some embodiments, the segmentation module 1205 is specifically used to: obtain a segmentation result based on the Kth second aggregation annotation result.

[0186] In some embodiments, the segmentation module 1205 includes a pixel annotation submodule and a segmentation submodule.

[0187] The pixel annotation submodule is used to obtain the third pixel annotation result of the target image based on at least one second aggregated annotation result. The third pixel annotation result is used to indicate the third category of each pixel in the target image.

[0188] The segmentation submodule is used to obtain the segmentation result based on the third pixel annotation result. The segmentation result contains at least one segmented region, which is composed of multiple consecutive pixels in the target image that belong to the same third category.

[0189] In some embodiments, the first model includes multiple cascaded convolutional layers, at least one first aggregated annotation result corresponds one-to-one with at least one of the multiple convolutional layers, and each first aggregated annotation result is output by its corresponding convolutional layer.

[0190] In some embodiments, the second model includes a first sub-model and a second sub-model. The second annotation module includes a first annotation sub-module and a second annotation sub-module.

[0191] The first annotation submodule is used to call the first sub-model and obtain intermediate results based on the target image.

[0192] The second annotation submodule is used to call the second sub-model and obtain at least one second aggregated annotation result based on the intermediate results and at least one first aggregated annotation result.

[0193] In some embodiments, the first sub-model contains M cascaded shrinking modules, the second sub-model contains M cascaded expanding modules, and the intermediate result is the output of the Mth shrinking module. The second annotation sub-module is specifically used for: calling the first expanding module and obtaining its output based on the output of the Mth shrinking module; for any 1 ≤ m < M, performing a concatenation operation between the output of the Mmth shrinking module and the output of the mth expanding module to obtain the concatenation result corresponding to the mth expanding module; and calling the (m+1)th expanding module and obtaining its output based on the concatenation result.

[0194] In some embodiments, at least one first aggregation annotation result corresponds one-to-one with at least one of the multiple extension modules. When at least one extension module includes the m-th extension module, the second annotation submodule is further configured to: perform a fusion operation on the splicing result corresponding to the extension module and the first aggregation annotation result corresponding to the extension module to obtain a first fusion result; and call the (m+1)-th extension module to obtain the output of the (m+1)-th extension module based on the fusion result. Moreover, when at least one extension module includes the M-th extension module, the second annotation submodule is further configured to: perform a fusion operation on the output corresponding to the M-th extension module and the first aggregation annotation result corresponding to the M-th extension module to obtain a second fusion result. Wherein, at least one second aggregation annotation result includes the second fusion result and the output of the (m+1)-th extension module.

[0195] In some embodiments, the shrinking module includes a downsampling layer and at least one first convolutional layer, and the expanding module includes an upsampling layer and at least one second convolutional layer.

[0196] In some embodiments, for one of the at least one extension module, the concatenation result corresponding to the extension module and the first aggregation annotation result corresponding to the extension module are matrices or vectors of the same size, and the fusion operation is a dot product operation.

[0197] Please refer to Figure 13 In some embodiments, the image segmentation apparatus further includes a training module 1206.

[0198] The training module 1206 is used to train the initial first model and the initial second model based on the training sample set to obtain the first model and the second model. The training sample set contains at least one training image, and each training image has a preset aggregation annotation result and a preset segmentation result.

[0199] In some embodiments, the training module 1206 is further configured to: adjust the parameters of the initial first model and the initial second model until the target loss meets the preset conditions.

[0200] In some embodiments, the training module 1206 is configured to: for each training image, obtain a predicted pixel annotation result for the training image, wherein the predicted pixel annotation result is used to indicate the prediction result of a first classification for each pixel in the training image; invoke an initial first model to obtain at least one predicted first aggregate annotation result based on the predicted pixel annotation result, wherein each predicted first aggregate annotation result is used to indicate the prediction result of the first classification-related features of each pixel block obtained by segmenting the training image according to the segmentation scheme corresponding to the corresponding first aggregate annotation result; invoke an initial second model to obtain at least one predicted second aggregate annotation result based on the training image and at least one predicted first aggregate annotation result, wherein each predicted second aggregate annotation result is used to indicate the prediction result of the features of each pixel block obtained by segmenting the training image according to the segmentation scheme corresponding to the corresponding second aggregate annotation result; and obtain a predicted segmentation result for the training image based on at least one predicted second aggregate annotation result.

[0201] The training module 1206 also includes a classification loss calculation submodule, a segmentation loss calculation submodule, and a target loss calculation submodule. The classification loss calculation submodule calculates the classification loss based on the difference between the predicted first aggregated annotation result and the preset aggregated annotation result for each training image. The segmentation loss calculation submodule calculates the segmentation loss based on the difference between the predicted segmentation result and the preset segmentation result for each training image. The target loss calculation submodule calculates the target loss based on the classification loss and the segmentation loss.

[0202] Therefore, the image segmentation apparatus provided in this application first acquires a target image, then acquires the first pixel annotation result of the target image, subsequently calls a first model to obtain at least one first aggregate annotation result based on the first pixel annotation result, then calls a second model to obtain at least one second aggregate annotation result based on the target image and at least one first aggregate annotation result, and finally obtains the segmentation result of the target image based on at least one second aggregate annotation result. The first pixel annotation result indicates the first classification of each pixel in the target image, each first aggregate annotation result indicates the features related to the first classification of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme, and each second aggregate annotation result indicates the features of each pixel block obtained by segmenting the target image according to its corresponding segmentation scheme. In the above scheme, the target image is pre-annotated before being segmented by the second model, and the pre-annotated features are extracted by the first model and used to supervise the segmentation process. Because the segmentation result is influenced by the pre-annotation, the first and second models can obtain sufficient accuracy without a large number of samples during training, thereby efficiently completing the image segmentation task.

[0203] A third aspect of this application provides an electronic device that can be used to implement the aforementioned image segmentation method. In some embodiments, the electronic device includes a processor and a memory. The memory stores instructions, and when executed by the processor, the instructions cause the electronic device to perform any of the aforementioned image segmentation methods.

[0204] A fourth aspect of this application provides a computer-readable storage medium. The computer-readable storage medium stores computer instructions, which, when executed by a processor, cause a computer to perform any of the aforementioned image segmentation methods.

[0205] The computer-readable storage medium includes program commands, data files, data structures, etc., or combinations thereof. The program recorded in the computer-readable storage medium can be designed or configured to implement the methods of the present invention. The computer-readable storage medium includes a hardware system for storing and executing program commands. Examples of hardware systems include magnetic media (such as hard disks, floppy disks, magnetic tapes), optical media (such as CD-ROMs and DVDs), and magneto-optical media (such as floppy disks, ROMs, RAMs, flash memory, etc.). The program includes assembly language code or machine code compiled by a compiler and higher-level language code interpreted by an interpreter. The hardware system can be implemented using at least one software module to conform to the present invention.

[0206] A fifth aspect of this application provides a computer program product. The computer program product includes computer instructions, which, when executed on a computer, cause the computer to perform any of the aforementioned image segmentation methods.

[0207] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method-related sections.

[0208] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been described in a generally functional manner in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0209] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image segmentation method characterized by, The method comprises: obtaining a target image; obtaining a first pixel annotation result of the target image, wherein the first pixel annotation result is used to indicate a first classification of each pixel in the target image; calling a first model to obtain at least one first aggregated annotation result according to the first pixel annotation result, wherein each first aggregated annotation result is used to indicate a feature related to the first classification of each pixel block obtained by segmenting the target image according to a segmentation scheme corresponding to the first aggregated annotation result; and calling a second model to obtain at least one second aggregated annotation result according to the target image and the at least one first aggregated annotation result, wherein each second aggregated annotation result is used to indicate a feature of each pixel block obtained by segmenting the target image according to a segmentation scheme corresponding to the second aggregated annotation result; obtaining a segmentation result of the target image according to the at least one second aggregated annotation result; wherein the step of calling the first model to obtain at least one first aggregated annotation result according to the first pixel annotation result comprises: calling the first model to obtain a first first aggregated annotation result according to the first pixel annotation result, and for any 1 ≤ k < K, obtaining a k+1th first aggregated annotation result according to a kth first aggregated annotation result; wherein the number of the first aggregated annotation results is K, and k and K are positive integers.

2. The image segmentation method of claim 1, wherein, The step of obtaining the first pixel annotation result of the target image comprises: segmenting the target image into a plurality of target sub-images; enlarging each target sub-image to obtain a corresponding expanded sub-image, wherein the enlargement ratio of each target sub-image is the same; obtaining a second pixel annotation result of each expanded sub-image, wherein the second pixel annotation result is used to indicate a second classification of each pixel in the expanded sub-image; wherein the first pixel annotation result is composed of the second pixel annotation result of each expanded sub-image.

3. The image segmentation method of claim 2, wherein, Each target sub-image has the same size.

4. The image segmentation method of claim 2, wherein, The step of obtaining the second pixel annotation result of each expanded sub-image comprises: manually annotating each expanded sub-image to obtain at least one manually annotated region; obtaining the second pixel annotation result according to the at least one manually annotated region, wherein the second classification of each pixel in each manually annotated region is the same.

5. The image segmentation method of claim 2, wherein, The step of calling the first model to obtain at least one first aggregated annotation result according to the first pixel annotation result comprises: for each first aggregated annotation result, calling the first model to process the second pixel annotation result corresponding to each expanded sub-image respectively to obtain a corresponding third aggregated annotation result; and splicing each third aggregated annotation result in a first order to obtain the first aggregated annotation result, wherein the first order is determined by the spatial order of the plurality of target sub-images in the target image.

6. The image segmentation method of claim 2, wherein, The step of calling the first model to obtain at least one first aggregated annotation result according to the first pixel annotation result comprises: splicing, according to a second order, second pixel label results corresponding to the plurality of extended sub-images, to obtain an extended pixel label result, wherein the second order is determined by spatial orders of the plurality of target sub-images in the target image; and calling the first model to obtain the at least one first aggregated label result according to the extended pixel label result.

7. The image segmentation method of claim 1, wherein, The at least one second aggregated label result corresponds to the at least one first aggregated label result one by one, and a segmentation scheme corresponding to each first aggregated result is the same as a segmentation scheme corresponding to a second aggregated result corresponding to the first aggregated result.

8. The image segmentation method of claim 7, wherein, For any 1 ≤ k < K: Each pixel block obtained by segmenting the target image according to a segmentation scheme corresponding to the kth first aggregated label result contains a plurality of pixel blocks obtained by segmenting the target image according to a segmentation scheme corresponding to the k+1th first aggregated label result.

9. The image segmentation method of claim 8, wherein, The step of calling the second model to obtain at least one second aggregated label result according to the target image and the at least one first aggregated label result comprises: For any 1 ≤ k < K, calling the second model to obtain the k+1th second aggregated label result according to the kth second aggregated label result and the K-k+1th first aggregated result.

10. The image segmentation method of claim 9, wherein, The step of obtaining a segmentation result of the target image according to the at least one second aggregated label result comprises: Obtaining the segmentation result according to the Kth second aggregated label result.

11. The image segmentation method of claim 1, wherein, The step of obtaining a segmentation result of the target image according to the at least one second aggregated label result comprises: Obtaining a third pixel label result of the target image according to the at least one second aggregated label result, wherein the third pixel label result is used to indicate a third classification of each pixel in the target image; and Obtaining the segmentation result according to the third pixel label result, wherein the segmentation result contains at least one segmentation region, and the segmentation region is composed of a plurality of pixels that are continuous and have the same third classification in the target image.

12. The image segmentation method of any one of claims 1-11, wherein, The first model contains a plurality of cascaded convolution layers, the at least one first aggregated label result corresponds to at least one layer of the plurality of cascaded convolution layers one by one, and each first aggregated label result is output by its corresponding convolution layer.

13. The image segmentation method of any one of claims 1-11, wherein, The second model contains a first sub-model and a second sub-model, and The step of calling the second model to obtain at least one second aggregated label result according to the target image and the at least one first aggregated label result comprises: Calling the first sub-model to obtain an intermediate result according to the target image; and Calling the second sub-model to obtain the at least one second aggregated label result according to the intermediate result and the at least one first aggregated label result.

14. The image segmentation method of claim 13, wherein, The first sub-model contains M cascaded contraction modules, the second sub-model contains M cascaded expansion modules, the intermediate result is an output of the Mth contraction module, and The step of calling the second sub-model to obtain the at least one second aggregated annotation result according to the intermediate result input and the at least one first aggregated annotation result comprises: calling the first extension module to obtain the output of the first extension module according to the output of the Mth contraction module; for any 1 ≤ m < M, performing splicing operation on the output of the M-mth contraction module and the output of the mth extension module to obtain a splicing result corresponding to the mth extension module, and calling the m+1th extension module to obtain the output of the m+1th extension module according to the splicing result.

15. The image segmentation method of claim 14, wherein, The at least one first aggregated annotation result corresponds to at least one extension module in the plurality of extension modules one by one, when the at least one extension module includes the mth extension module, the step of obtaining the output of the m+1th extension module according to the splicing result comprises: performing fusion operation on the splicing result corresponding to the extension module and the first aggregated annotation result corresponding to the extension module to obtain a first fusion result; and calling the m+1th extension module to obtain the output of the m+1th extension module according to the fusion result; and when the at least one extension module includes the Mth extension module, the step of calling the second sub-model to obtain the at least one second aggregated annotation result according to the intermediate result input and the at least one first aggregated annotation result further comprises: performing fusion operation on the output corresponding to the Mth extension module and the first aggregated annotation result corresponding to the Mth extension module to obtain a second fusion result; wherein the at least one second aggregated annotation result includes the second fusion result and the output of the m+1th extension module.

16. The image segmentation method of claim 15, wherein, The contraction module includes a down-sampling layer and at least one first convolutional layer, and the extension module includes an up-sampling layer and at least one second convolutional layer.

17. The image segmentation method of claim 15, wherein, For one extension module in the at least one extension module, the splicing result corresponding to the extension module and the first aggregated annotation result corresponding to the extension module are matrices or vectors with the same size, and the fusion operation is dot product operation.

18. The image segmentation method of any one of claims 1-11, wherein, Further comprising: training the initial first model and the initial second model based on a training sample set to obtain the first model and the second model, wherein the training sample set includes at least one training image, each training image, each training image has a preset aggregated annotation result and a preset segmentation result; wherein the step of training the initial first model and the initial second model based on the training sample set comprises: adjusting the parameters of the initial first model and the initial second model until the target loss meets a preset condition.

19. The image segmentation method of claim 18, wherein, The target loss obtaining process comprises: for each training image, obtaining a predicted pixel annotation result of the training image, wherein the predicted pixel annotation result is used to indicate the predicted result of the first classification of each pixel in the training image; calling the initial first model to obtain at least one predicted first aggregated annotation result according to the predicted pixel annotation result, where each predicted first aggregated annotation result is used to indicate a predicted result of a first classification-related feature of each pixel block obtained by segmenting the training image according to a segmentation scheme corresponding to the corresponding first aggregated annotation result; calling the initial second model to obtain at least one predicted second aggregated annotation result according to the training image and the at least one predicted first aggregated annotation result, where each predicted second aggregated annotation result is used to indicate a predicted result of a feature of each pixel block obtained by segmenting the training image according to a segmentation scheme corresponding to the corresponding second aggregated annotation result; and obtaining a predicted segmentation result of the training image according to the at least one predicted second aggregated annotation result; and the obtaining process of the target loss further includes: obtaining a classification loss according to a difference between the predicted first aggregated annotation result of each training image and a preset aggregated annotation result; obtaining a segmentation loss according to a difference between the predicted segmentation result of each training image and a preset segmentation result of each training image; and obtaining the target loss according to the classification loss and the segmentation loss.

20. An image segmentation apparatus characterized by comprising: comprise: an obtaining module, configured to obtain a target image; a first annotation module, configured to obtain a first pixel annotation result of each target, where the first pixel annotation result is used to indicate a first classification of each pixel in the target; a feature extraction module, configured to call a first model to obtain at least one first aggregated annotation result according to the first pixel annotation result, where each first aggregated annotation result is used to indicate a first classification-related first feature of each pixel block obtained by segmenting the target image according to a segmentation scheme corresponding to the first aggregated annotation result; a second annotation module, configured to call a second model to obtain at least one second aggregated annotation result according to the target image and the at least one first aggregated annotation result, where each second aggregated annotation result is used to indicate a second feature of each pixel block obtained by segmenting the target image according to a segmentation scheme corresponding to the second aggregated annotation result; and a segmentation module, configured to obtain a segmentation result of the target image according to the at least one second aggregated annotation result; wherein the feature extraction module is configured to: call the first model to obtain a first first aggregated annotation result according to the first pixel annotation result, and obtain a k+1th first aggregated annotation result according to a kth first aggregated annotation result for any 1≤k wherein the number of the first aggregated annotation results is K, and k and K are positive integers.

21. An electronic device, comprising: comprise a processor and a memory having instructions stored therein, wherein the instructions, when executed by the processor, cause the electronic device to perform the image segmentation method according to any one of claims 1 to 19.

22. A computer-readable storage medium, characterized in that the storage medium stores instructions, and The instructions, when executed by a processor, cause the computer to perform the image segmentation method according to any one of claims 1-19. The instructions, when executed by a processor, cause the computer to perform the image segmentation method according to any one of claims 1-19.

Images