Image segmentation method and apparatus, storage medium, and electronic device

Abstract

The application discloses an image segmentation method and device, a storage medium and an electronic device. It relates to the technical field of artificial intelligence. The method comprises the following steps: performing down-sampling processing on a first image to be segmented by an embedding module of a target model to obtain a plurality of scale second images; performing multi-scale coding processing on the first target image by a global feature extraction module and a local feature extraction module of the target model to obtain a plurality of scale target feature images, wherein the first target image is obtained from the plurality of scale second images and the first image; and performing multi-scale decoding processing on the plurality of scale target feature images by the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image. The application solves the technical problem that the accuracy of image segmentation is relatively low in the related art because the image is segmented according to the local features of the image.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to an image segmentation method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] In the field of image processing and analysis, especially in the development of computer vision technology, image segmentation is a core task with significant value for precision medical diagnosis, autonomous driving systems, and remote sensing image analysis. Traditional methods, including threshold-based segmentation, edge detection, and early machine learning models, while performing well in specific scenarios, often face challenges when processing complex images, such as those with diverse textures, varying lighting, or blurred boundaries.

[0003] In medical imaging, such as pathological image analysis, the accurate identification and segmentation of skin lesions and tissue structures are crucial. However, current technologies often focus too much on local image features while neglecting the importance of global contextual information. While local features can capture details, they are insufficient for accurate judgments based solely on local information when lesion boundaries are blurred or morphology is variable.

[0004] There is currently no effective solution to the problem that image segmentation accuracy is relatively low when segmenting images based on local features, as mentioned above. Summary of the Invention

[0005] This application provides an image segmentation method, apparatus, storage medium, and electronic device to at least solve the technical problem in related technologies where image segmentation based on local features results in relatively low accuracy of image segmentation.

[0006] According to one aspect of the embodiments of this application, an image segmentation method is provided, comprising: performing downsampling processing on a first image to be segmented through an embedding module of a target model to obtain a second image at multiple scales; performing multi-scale encoding processing on the first target image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image; and performing multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0007] Further, the first target image is subjected to multi-scale encoding processing through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, including: upsampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; downsampling the second target image through the global feature extraction module and the local feature extraction module to obtain the target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

[0008] Further, the upsampling process performed on the first target image by the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales includes: performing global feature extraction on the first image and the fourth image by the global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a partial image of the second images at multiple scales; performing local feature extraction on the second image at the first scale of the second images at multiple scales by the local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and obtaining the third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales.

[0009] Further, obtaining the third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales includes: fusing features of the second initial feature image at the second scale and the first initial feature images at multiple scales from the second initial feature image at the second scale using the feature fusion module in the target model to obtain the third image at the second scale; extracting local features from the third image at the second scale using the local feature extraction module to obtain the second initial feature image at the third scale; fusing features of the second initial feature image at the third scale and the second initial feature image at the third scale from the first initial feature images at multiple scales using the feature fusion module to obtain the third image at the third scale, wherein the second scale is smaller than the third scale; and repeating the step of extracting local features from the third image at the third scale using the local feature extraction module until all images in the first initial feature images at multiple scales have been processed, and determining the currently obtained third image as the third image at multiple scales.

[0010] Further, the downsampling process performed on the second target image by the global feature extraction module and the local feature extraction module to obtain the target feature images at multiple scales includes: performing global feature extraction on the fifth image and the second image at the first target scale among the second images at multiple scales by the global feature extraction module to obtain a third initial feature image at multiple scales, wherein the fifth image is a partial image among the third images at multiple scales, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the second images at multiple scales; performing local feature extraction on the third image at the Nth scale among the third images at multiple scales by the local feature extraction module to obtain a fourth initial feature image at the N-1th scale; and obtaining the target feature images at multiple scales based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales.

[0011] Further, obtaining the target feature image at multiple scales based on the fourth initial feature image at the N-1 scale and the third initial feature image at multiple scales includes: fusing features of the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale from the third initial feature images at multiple scales through the feature fusion module in the target model to obtain the target feature image at the N-1 scale; extracting local features from the target feature image at the N-1 scale through the local feature extraction module to obtain the fourth initial feature image at the N-2 scale; fusing features of the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale from the third initial feature images at multiple scales through the feature fusion module to obtain the target feature image at the N-2 scale; and repeating the step of extracting local features from the target feature image at the N-2 scale through the local feature extraction module until all images in the third initial feature images at multiple scales have been processed, and determining the currently obtained target feature image as the target feature image at multiple scales.

[0012] Further, the multi-scale decoding processing of the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image includes: performing global feature extraction on the sixth and seventh images through the global feature extraction module to obtain first feature maps at multiple scales, wherein the sixth image is a partial image among the target feature images at multiple scales, and the seventh image is an image at the second target scale among the third images at multiple scales; performing local feature extraction on the target feature images at the first scale in the target feature images at multiple scales through the local feature extraction module to obtain a second feature map at the second scale; performing feature fusion on the second feature map at the second scale and the first feature map at the second scale in the first feature images at multiple scales to obtain a feature fusion map at the second scale; repeating the step of performing local feature extraction on the feature fusion map at the second scale through the local feature extraction module until a feature fusion map at the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map at the Nth scale.

[0013] Furthermore, the target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0014] According to another aspect of the embodiments of this application, an image segmentation method is also provided, comprising: downsampling a first medical image through an embedding module of a target model to obtain a second medical image at multiple scales; performing multi-scale encoding processing on the first target medical image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target medical image is obtained from the second medical image at multiple scales and the first medical image; and performing multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first medical image.

[0015] According to another aspect of the embodiments of this application, an image segmentation method is also provided, comprising: acquiring a first image to be segmented uploaded by a client; performing downsampling processing on the first image to be segmented in a cloud server through an embedding module of a target model to obtain a second image at multiple scales; performing multi-scale encoding processing on the first target image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image; performing multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image; and returning the image segmentation result to the client.

[0016] According to another aspect of the embodiments of this application, an image segmentation apparatus is also provided, comprising: a first processing unit, configured to perform downsampling processing on a first image to be segmented through an embedding module of a target model to obtain a second image at multiple scales; a second processing unit, configured to perform multi-scale encoding processing on the first target image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image; and a determining unit, configured to perform multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0017] Further, the second processing unit includes: a first processing subunit, used to upsample the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; and a second processing subunit, used to downsample the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

[0018] Further, the first processing subunit includes: a first extraction module, configured to perform global feature extraction on the first image and the fourth image through the global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a portion of the second images at multiple scales; a second extraction module, configured to perform local feature extraction on the second image at the first scale in the second images at multiple scales through the local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and a first determination module, configured to obtain a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales.

[0019] Further, the first determining module includes: a first fusion submodule, used to perform feature fusion on the second initial feature image at the second scale and the first initial feature image at the second scale among the multiple scales of the first initial feature images through the feature fusion module in the target model to obtain a third image at the second scale; a first extraction submodule, used to perform local feature extraction on the third image at the second scale through the local feature extraction module to obtain a second initial feature image at the third scale; a second fusion submodule, used to perform feature fusion on the second initial feature image at the third scale and the second initial feature image at the third scale among the multiple scales of the first initial feature images through the feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; and a first determining submodule, used to repeatedly execute the step of performing local feature extraction on the third image at the third scale through the local feature extraction module until all images in the multiple scales of the first initial feature images have been processed, and to determine the currently obtained third image as the third image at the multiple scales.

[0020] Further, the second processing subunit includes: a third extraction module, used to perform global feature extraction on the fifth image and the second image at the first target scale among the multiple scales of the second images through the global feature extraction module, to obtain a third initial feature image at multiple scales, wherein the fifth image is a partial image among the multiple scales of the third images, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the multiple scales of the second images; a fourth extraction module, used to perform local feature extraction on the third image at the Nth scale among the multiple scales of the third images through the local feature extraction module, to obtain a fourth initial feature image at the N-1th scale; and a second determination module, used to obtain the target feature image at the multiple scales based on the fourth initial feature image at the N-1th scale and the third initial feature image at the multiple scales.

[0021] Further, the second determining module includes: a third fusion submodule, used to perform feature fusion on the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale among the multiple scales of third initial feature images through the feature fusion module in the target model, to obtain a target feature image at the N-1 scale; a second extraction submodule, used to perform local feature extraction on the target feature image at the N-1 scale through the local feature extraction module, to obtain a fourth initial feature image at the N-2 scale; a fourth fusion submodule, used to perform feature fusion on the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale among the multiple scales of third initial feature images through the feature fusion module, to obtain a target feature image at the N-2 scale; and a second determining submodule, used to repeatedly execute the step of performing local feature extraction on the target feature image at the N-2 scale through the local feature extraction module until all images in the multiple scales of third initial feature images have been processed, and to determine the currently obtained target feature image as the target feature image at the multiple scales.

[0022] Further, the determining unit includes: a first extraction subunit, used to perform global feature extraction on the sixth image and the seventh image through the global feature extraction module to obtain a first feature map at multiple scales, wherein the sixth image is a partial image among the target feature images at multiple scales, and the seventh image is an image at the second target scale among the third images at multiple scales; a second extraction subunit, used to perform local feature extraction on the target feature image at the first scale among the target feature images at multiple scales through the local feature extraction module to obtain a second feature map at the second scale; a fusion subunit, used to perform feature fusion on the second feature map at the second scale and the first feature map at the second scale among the first feature images at multiple scales to obtain a feature fusion map at the second scale; and a determining subunit, used to repeatedly execute the step of performing local feature extraction on the feature fusion map at the second scale through the local feature extraction module until a feature fusion map at the Nth scale is obtained, and to obtain the image segmentation result based on the feature fusion map at the Nth scale.

[0023] Furthermore, the target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0024] According to another aspect of the present invention, an electronic device is also provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the image segmentation method described above during runtime.

[0025] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein the storage medium stores a program, wherein the program controls the device where the storage medium is located to execute the image segmentation method described above during runtime.

[0026] According to another aspect of the present invention, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the image segmentation method described above.

[0027] In this embodiment, the following steps are employed: The first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales; the first target image is multi-scale encoded using the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second images at multiple scales and the first image; the target feature images at multiple scales are multi-scale decoded using the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image. This solves the technical problem in related technologies where image segmentation based on local features results in low accuracy.

[0028] In this scheme, the first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales. Then, the first target image is encoded using the global feature extraction module and the local feature extraction module in the target model to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and represented, avoiding the information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the fusion of rich multi-scale information and the effective fusion between global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1This is a hardware structure block diagram of a computer terminal provided according to Embodiment 1 of this application;

[0031] Figure 2 This is a flowchart of the image segmentation method provided according to Embodiment 1 of this application;

[0032] Figure 3 This is a schematic diagram of the embedded module provided according to Embodiment 1 of this application;

[0033] Figure 4 This is a schematic diagram of the global feature extraction module provided according to Embodiment 1 of this application;

[0034] Figure 5 This is a schematic diagram of the local feature extraction module provided according to Embodiment 1 of this application;

[0035] Figure 6 This is a schematic diagram of feature fusion according to Embodiment 1 of this application;

[0036] Figure 7 This is a schematic diagram of the target model provided in Embodiment 1 of this application. Figure 1 ;

[0037] Figure 8 This is a schematic diagram of the target model provided in Embodiment 1 of this application. Figure 2 ;

[0038] Figure 9 This is a schematic diagram of the image segmentation method provided according to Embodiment 1 of this application;

[0039] Figure 10 This is a flowchart of the image segmentation method provided according to Embodiment 2 of this application;

[0040] Figure 11 This is a flowchart of the image segmentation method provided according to Embodiment 3 of this application;

[0041] Figure 12 This is a schematic diagram of an image segmentation apparatus according to Embodiment 4 of this application;

[0042] Figure 13 This is a structural block diagram of a computing device provided according to Embodiment 5 of this application. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0044] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0045] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0046] Example 1

[0047] According to an embodiment of this application, an image segmentation method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0048] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing an image segmentation method is shown. Figure 1 As shown, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, and the processor set 102 may include a processor set, Figure 1(Illustrated as 102a, 102b, ..., 102n), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0049] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image segmentation method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the image segmentation method described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0052] The display may be a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0053] Under the aforementioned operating environment, this application provides the following: Figure 2 The image segmentation method shown. Figure 2 This is a flowchart of an image segmentation method according to Embodiment 1 of this application. The image segmentation method includes:

[0054] Step S201: The first image to be segmented is downsampled through the embedding module of the target model to obtain a second image at multiple scales.

[0055] Optionally, the target model consists of an embedding module, a global feature extraction module, and a local feature extraction module. The embedding module can be an Order Preserving Embedding (OPE) module, the global feature extraction module can be a Global Spatial Block (GSB) module, and the local feature extraction module can be a Local Spatial Block (LSB) module.

[0056] The first image to be segmented is identified. It should be noted that the first image can be a medical image, a remote sensing image, etc. The first image to be segmented is input into the target model. The embedding module of the target model performs downsampling processing on the first image to be segmented. The embedding module OPE converts the first image into a low-dimensional embedding vector, which is the second image with multiple scales mentioned above.

[0057] For example, the embedding module OPE performs three levels of downsampling on the first image, which is H*W*C1. H (Height) represents the height, i.e., the number of pixels in the vertical direction of the image or feature map. W (Width) represents the width, i.e., the number of pixels in the horizontal direction of the image or feature map. C (Channels) represents the number of channels; for color images, there are typically three channels (red, green, and blue), while for grayscale images, there is one channel. Downsampling H*W*C1 by the embedding module OPE yields H / 2*W / 2*C2, downsampling H / 2*W / 2*C2 yields H / 4*W / 4*C3, and downsampling H / 4*W / 4*C3 yields H / 8*W / 8*C4. These H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4 are the second images at multiple scales mentioned above.

[0058] In an optional embodiment, the embedding module OPE can also perform downsampling processing on the first image at three or more levels. For example, the embedding module OPE can obtain second images at multiple scales, such as H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4, and H / 16*W / 16*C5.

[0059] In an optional embodiment, the embedding module OPE may consist of convolutional layers, batch normalization layers, nonlinear activation functions, and downsampling, such as... Figure 3 As shown in the diagram, the embedding module OPE processes the first image using 3*3 Conv, s1 (a 3x3 kernel (or filter) performing convolution with a stride of 1), BN (Batch Normalization), GELU (Gaussian Error Linear Unit), and Down (the downsampling mentioned above) to obtain the second image at multiple scales.

[0060] Step S202: The first target image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales. The first target image is obtained from the second image and the first image at multiple scales.

[0061] Optionally, after obtaining the second images at multiple scales, the first target image is encoded using a global feature extraction module and a local feature extraction module to obtain target feature images at multiple scales. It should be noted that the first target image is determined from the second images at multiple scales and the first image. For example, the first target image can be the first image itself, a second image at a specific scale among the multiple second images, or a combination of the second images at multiple scales and the first image.

[0062] The global features of the first target image are extracted by the global feature extraction module. In an optional embodiment, the structural diagram of the global feature extraction module is shown below. Figure 4 As shown, the global feature extraction module can consist of Layer Normalization (LN), a Token Mixer, and a Multilayer Perceptron (MLP). In image processing, an image can be divided into multiple fixed-size regions, each treated as a token. The role of the Token Mixer is to mix and interact information between these tokens, typically achieved through self-attention mechanisms, to enhance the model's understanding of the global structure of the image. Figure 4 As shown, the global feature extraction module consists of two LNs, one Token-Mixer, and one MLP. The global feature extraction module can accurately extract features of the entire image or a large area, such as the overall layout of the image, background information, and the positional relationships of objects.

[0063] The local feature extraction module extracts local features from the first target image. In an optional embodiment, the structural diagram of the local feature extraction module is shown below. Figure 5 As shown, the local feature extraction module consists of convolutional layers (e.g., 3*3 Conv, s1), batch normalization layers, non-linear activation functions, and downsampling / subsampling, as follows. Figure 5 As shown, the local feature extraction module consists of two convolutional layers (e.g., 3*3 Conv, s1), two batch normalization (BN) layers, two non-linear activation functions, and down / up (i.e., downsampling / downsampling as described above). The local feature extraction module can accurately extract local details of an image, such as the specific texture, edges, and shape of an object.

[0064] It should be noted that the global feature extraction module and the local feature extraction module perform multi-scale encoding operations on the first target image, including downsampling and upsampling operations. These operations adjust the resolution of the image or feature map. Downsampling helps the model focus on more abstract features by reducing the dimensionality of the feature map, while upsampling helps the target model recover details in segmentation or reconstruction tasks by increasing the dimensionality of the feature map. Sampling processing is the foundation for constructing multi-scale representations, enabling the target model to process both macroscopic and microscopic information of the image simultaneously.

[0065] Step S203: Multi-scale decoding processing is performed on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image.

[0066] Optionally, after obtaining the target feature images at multiple scales, the target feature images at multiple scales can be further processed by the global feature extraction module and the local feature extraction module to perform multi-scale decoding, generating a classification label for each pixel in the first image, thereby obtaining the image segmentation result. For example, an upsampling operation can be performed by the global feature extraction module and the local feature extraction module to restore the feature map to the same resolution as the first image, thereby achieving accurate segmentation of the first image.

[0067] In summary, the first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales. Then, the first target image is encoded at multiple scales using the global feature extraction module and the local feature extraction module in the target model to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and represented, avoiding the information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the fusion of rich multi-scale information and the effective fusion between global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0068] To improve the accuracy of feature extraction from the first target image, the image segmentation method provided in Embodiment 1 of this application performs multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales. This includes: upsampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; and downsampling the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

[0069] Optionally, the first target image is first upsampled using a global feature extraction module and a local feature extraction module. This upsampling process extracts both global and local features of the first target image, resulting in a third image at multiple scales. The third image has higher resolution, facilitating the model's recovery and utilization of local details for more refined analysis in subsequent steps. It should be noted that the first target image can be a combination of a second image and a first image at multiple scales. For example, two paths can be branched: one path inputs the second and first images at multiple scales into the GSB (Global Subsampler) to obtain an upsampled global feature map; the other path inputs the second and first images at multiple scales into the LSB (Local Subsampler) to obtain an upsampled local feature map. By fusing the global and local feature maps, the aforementioned third image at multiple scales is obtained.

[0070] Then, the second target image is downsampled through the global feature extraction module and the local feature extraction module. The second target image can be a second image at multiple scales and a third image at multiple scales. For example, two paths are branched out: one path enters the GSB processing to obtain the downsampled global feature map, and the other path enters the LSB processing to obtain the downsampled local feature map. By fusing the downsampled global feature map and the local feature map, the target feature image at multiple scales mentioned above is obtained.

[0071] For example, the embedding module OPE obtains second images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4. Then, the global feature extraction module and the local feature extraction module upsample the first image H*W*C1 and the second images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4 to obtain third images H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1. Then, the global feature extraction module and the local feature extraction module downsample the second image H / 8*W / 8*C4 and the third image H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1 to obtain target feature images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4.

[0072] Upsampling and downsampling operations further enhance the target model's ability to fuse and utilize multi-scale information, enabling the target model to understand both macroscopic structure and microscopic details when processing images, thereby improving the accuracy of image segmentation.

[0073] To improve the accuracy of the third image, the image segmentation method provided in Embodiment 1 of this application upsamples the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales. This includes: performing global feature extraction on the first image and the fourth image through the global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a part of the second image at multiple scales; performing local feature extraction on the second image at the first scale in the second image at multiple scales through the local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and obtaining a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales.

[0074] Optionally, the upsampling process of the global feature extraction module and the local feature extraction module for the first target image includes the following steps: First, the global feature extraction module performs global feature extraction on the first image and the fourth image to obtain first initial feature images at multiple scales. It should be noted that the fourth image is a partial image of the second image at multiple scales. For example, the fourth image is a second image of H / 2*W / 2*C2 and H / 4*W / 4*C3. For example, the global feature extraction module performs global feature extraction on the first image of H*W*C1, the second image of H / 2*W / 2*C2 and H / 4*W / 4*C3 to obtain first initial feature images at multiple scales. For example, the first initial feature images of H*W*C1, H / 2*W / 2*C2 and H / 4*W / 4*C3 are obtained.

[0075] It should be noted that if OPE performs four levels of downsampling, then the fourth image is the second image of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, and the first initial feature images of multiple scales are the first initial feature images of H*W*C1, H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4.

[0076] While performing global feature extraction through the global feature extraction module, the local feature extraction module performs local feature extraction on the second image at the first scale among the second images at multiple scales to obtain the second initial feature image at the second scale. For example, the local feature extraction module performs local feature extraction on the second image of H / 8*W / 8*C4 to obtain the second initial feature image of H / 4*W / 4*C3.

[0077] It should be noted that if OPE performs four-level downsampling processing, the local feature extraction module will perform local feature extraction on the second image of H / 16*W / 16*C5 to obtain the second initial feature image of H / 8*W / 8*C3.

[0078] Finally, based on the second initial feature image at the second scale and the first initial feature images at multiple scales, a third image at multiple scales is obtained. For example, the target model will perform multi-scale feature fusion, fusing the second initial feature image at the second scale with the first initial feature images at multiple scales to obtain a third image at multiple scales.

[0079] For example, feature alignment is required before fusing features at different scales. After alignment, the aligned feature maps can be directly added element-wise to obtain a third image at multiple scales, or they can be multiplied element-wise to obtain a third image at multiple scales.

[0080] By combining global and local feature extraction modules with multi-scale feature fusion, the system can comprehensively capture the macroscopic structure and microscopic details of an image, thereby improving the accuracy of image segmentation.

[0081] To further improve the accuracy of the third image containing features, the image segmentation method provided in Embodiment 1 of this application, based on a second initial feature image at a second scale and first initial feature images at multiple scales, obtains a third image at multiple scales by: fusing features of the second initial feature image at the second scale and the first initial feature images at multiple scales using a feature fusion module in the target model to obtain a third image at the second scale; extracting local features from the third image at the second scale using a local feature extraction module to obtain a second initial feature image at the third scale; fusing features of the second initial feature image at the third scale and the second initial feature images at the third scale using a feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; and repeatedly performing the step of extracting local features from the third image at the third scale using a local feature extraction module until all images in the first initial feature images at multiple scales have been processed, and the currently obtained third image is determined as a third image at multiple scales.

[0082] Optionally, obtaining a third image at multiple scales based on a second initial feature image at a second scale and first initial feature images at multiple scales includes the following steps: First, the feature fusion module performs feature fusion processing on the second initial feature image at the second scale and the first initial feature images at the same scale (second scale) to obtain a third image at the second scale. For example, if the second scale is H / 4*W / 4*C3, the feature fusion module performs feature fusion processing on the second initial feature image of H / 4*W / 4*C3 and the first initial feature image of H / 4*W / 4*C3 to obtain a third image of H / 4*W / 4*C3.

[0083] Then, the local feature extraction module will perform local feature extraction on the third image at the second scale again to obtain the second initial feature image at the third scale. For example, the local feature extraction module performs local feature extraction on the third image of H / 4*W / 4*C3 to obtain the second initial feature image of H / 2*W / 2*C2.

[0084] After obtaining the second initial feature image at the third scale, the feature fusion module performs feature fusion processing on the second initial feature image at the third scale and the second initial feature image at the third scale from multiple first initial feature images at different scales to obtain the third image at the third scale. For example, if the third scale is H / 2*W / 2*C2, the feature fusion module performs feature fusion on the second initial feature image of H / 2*W / 2*C2 and the first initial feature image of H / 2*W / 2*C2 to obtain the third image of H / 2*W / 2*C2.

[0085] The aforementioned local feature extraction and feature fusion will be repeatedly executed. In each loop, the local feature extraction module will extract local features based on the third image at the current scale and generate the second initial feature image at the next scale. Then, it will be fused with the first initial feature image at the corresponding scale again through the feature fusion module until all images in the first initial feature images at multiple scales have been processed, and the currently obtained third image will be determined as the third image at multiple scales.

[0086] In an optional embodiment, a schematic diagram of the feature fusion module performing feature fusion is shown below. Figure 6 As shown, through the global feature extraction module and the local feature extraction module, global features (e.g., the first initial feature image) and local features (e.g., the second initial feature image) are obtained. Then, the global features and local features are convolved to obtain the fused features (e.g., the third image).

[0087] Through progressive feature fusion and local feature extraction, the target model can effectively integrate feature information at different scales. Furthermore, through the aforementioned local feature extraction and feature fusion, it can ensure that global and local features can be continuously transmitted without confusion, thereby improving the accuracy of the features contained in the third image.

[0088] To improve the accuracy of the target feature image, the image segmentation method provided in Embodiment 1 of this application performs downsampling processing on the second target image through a global feature extraction module and a local feature extraction module to obtain target feature images at multiple scales. This includes: performing global feature extraction on the fifth image and the second image at the first target scale in the multiple scales of the second image through the global feature extraction module to obtain a third initial feature image at multiple scales, wherein the fifth image is a partial image in the multiple scales of the third image, and the scale of the second image at the first target scale is smaller than the scale of the remaining images in the multiple scales of the second image; performing local feature extraction on the third image at the Nth scale in the multiple scales of the third image through the local feature extraction module to obtain a fourth initial feature image at the N-1th scale; and obtaining target feature images at multiple scales based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales.

[0089] Optionally, downsampling the second target image using a global feature extraction module and a local feature extraction module includes the following steps: First, the global feature extraction module performs global feature extraction on the fifth image and the second image at the first target scale from multiple scales to obtain a third initial feature image at multiple scales. It should be noted that the fifth image can be a portion of the third images at multiple scales. For example, if OPE performs four-level downsampling, the fifth image can be a third image of size H / 2*W / 2*C2, H / 4*W / 4*C3, or H / 8*W / 8*C4. If OPE performs three-level downsampling, the fifth image can be a third image of size H / 2*W / 2*C2 or H / 4*W / 4*C3. The second image at the first target scale is smaller than the scale of the remaining images at multiple scales. For example, if OPE performs four-level downsampling, the first target scale is H / 16*W / 16*C5; if OPE performs three-level downsampling, the first target scale is H / 8*W / 8*C4.

[0090] In an optional embodiment, the global feature extraction module performs global feature extraction on the third images of H / 2*W / 2*C2 and H / 4*W / 4*C3 and the second image of H / 8*W / 8*C4 to obtain the third initial feature images of H / 2*W / 2*C2, H / 4*W / 4*C3 and H / 8*W / 8*C4.

[0091] Then, the local feature extraction module performs local feature extraction on the third image at the Nth scale to obtain the fourth initial feature image at the (N-1)th scale. For example, the scale of the third image at the Nth scale is larger than the scale of the other images in the third image. For example, if the third image includes H*W*C1, H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, then the Nth scale is H*W*C1, and the corresponding (N-1)th scale is H / 2*W / 2*C2.

[0092] Finally, based on the fourth initial feature image at the (N-1)th scale and the third initial feature images at multiple scales, target feature images at multiple scales are obtained. For example, the target model performs multi-scale feature fusion, fusing the fourth initial feature image at the (N-1)th scale with the third initial feature images at multiple scales to obtain target feature images at multiple scales.

[0093] For example, feature alignment is required before fusing features at different scales. After alignment, the aligned feature maps can be directly added element-wise to obtain target feature images at multiple scales, or they can be multiplied element-wise to obtain target feature images at multiple scales.

[0094] By employing complementary global and local feature extraction strategies, as well as multi-scale feature fusion, the accuracy of target feature images can be effectively improved, thereby enhancing the technical effect of improving image segmentation accuracy.

[0095] To further improve the accuracy of the target feature image, the image segmentation method provided in Embodiment 1 of this application, based on the fourth initial feature image at the N-1 scale and the third initial feature images at multiple scales, obtains the target feature image at multiple scales by: fusing features of the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale from the third initial feature images at multiple scales through the feature fusion module in the target model to obtain the target feature image at the N-1 scale; extracting local features from the target feature image at the N-1 scale through the local feature extraction module to obtain the fourth initial feature image at the N-2 scale; fusing features of the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale from the third initial feature images at multiple scales through the feature fusion module to obtain the target feature image at the N-2 scale; and repeatedly performing the step of extracting local features from the target feature image at the N-2 scale through the local feature extraction module until all images in the third initial feature images at multiple scales have been processed, and determining the currently obtained target feature image as the target feature image at multiple scales.

[0096] Optionally, the feature fusion module performs feature fusion on the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale. For example, feature fusion is achieved by performing convolution calculation on the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale to obtain the target feature image at the N-1 scale.

[0097] After obtaining the target feature image at the (N-1)th scale, the local feature extraction module performs local feature extraction on the target feature image at the (N-1)th scale to obtain the fourth initial feature image at the (N-2)th scale. That is, the local feature extraction module gradually downsamples. For example, if the (N-1)th scale is H / 2*W / 2*C2, then the (N-2)th scale is H / 4*W / 4*C3.

[0098] The aforementioned local feature extraction and feature fusion are repeatedly performed. In each loop, the local feature extraction module extracts local features based on the target feature image at the current scale and generates the fourth initial feature image at the next scale. Then, it is fused with the third initial feature image at the corresponding scale by the feature fusion module again, until all images in the third initial feature images at multiple scales have been processed, generating a target feature image covering all scales.

[0099] In an optional embodiment, the second images of multiple scales obtained by downsampling the first image H*W*C1 using OPE are the second images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4. The second images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4 and the first image H*W*C1 are progressively processed using GSB and LSB to obtain the third images H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1. The third images H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1 and the second image H / 8*W / 8*C4 are progressively processed using GSB and LSB to obtain the target feature images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4.

[0100] In an optional embodiment, the second images of multiple scales obtained by downsampling the first image H*W*C1 using OPE are second images of H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4, and H / 16*W / 16*C5. The second images H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4, and H / 16*W / 16*C5, along with the first image H*W*C1, are then progressively upsampled using GSB and LSB to obtain H... The third image of H / 8*W / 8*C4, H / 4*W / 4*C3, H / 2*W / 2*C2, H*W*C1 is subjected to progressive downsampling processing using GSB and LSB to obtain the target feature images of H / 2*W / 2*C2, H / 4*W / 4*C3, H / 2*W / 2*C2, H*W*C1 and the second image of H / 16*W / 16*C5.

[0101] It should be noted that the second image at multiple scales is not limited to the third-level or fourth-level scale second image mentioned above. The downsampling level of the first image can be set according to actual needs, for example, five-level downsampling.

[0102] By using recursive feature fusion and local feature refinement, the details and boundary clarity of image segmentation are enhanced, thereby improving the robustness and generalization ability of the model in complex image analysis tasks.

[0103] To improve the accuracy of image segmentation, the image segmentation method provided in Embodiment 1 of this application performs multi-scale decoding processing on target feature images at multiple scales through a global feature extraction module and a local feature extraction module to obtain the image segmentation result corresponding to the first image. This includes: performing global feature extraction on the sixth and seventh images through the global feature extraction module to obtain first feature maps at multiple scales, wherein the sixth image is a partial image among the target feature images at multiple scales, and the seventh image is an image at the second target scale among the third images at multiple scales; performing local feature extraction on the target feature images at the first scale in the target feature images at multiple scales through the local feature extraction module to obtain a second feature map at the second scale; performing feature fusion on the second feature map at the second scale and the first feature map at the second scale in the first feature images at multiple scales to obtain a feature fusion map at the second scale; and repeating the step of performing local feature extraction on the feature fusion map at the second scale through the local feature extraction module until a feature fusion map at the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map at the Nth scale.

[0104] Optionally, obtaining the image segmentation result corresponding to the first image based on target feature images at multiple scales includes the following steps: determining a sixth image from a subset of the target feature images at multiple scales. For example, if the target feature images at multiple scales are H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, then the sixth image is the target feature image of H / 2*W / 2*C2 and H / 4*W / 4*C3. The seventh image is the image at the second target scale among the third images at multiple scales. For example, if the third images at multiple scales are the third images of H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1, then the seventh image is the third image of H*W*C1. The global feature extraction module performs global feature extraction on the target feature images of H / 2*W / 2*C2 and H / 4*W / 4*C3 and the third image of H*W*C1 to obtain the first feature maps of H*W*C1, H / 2*W / 2*C2 and H / 4*W / 4*C3.

[0105] Then, the local feature extraction module processes the target feature image at the first scale to generate a second feature map at the second scale. For example, if the target feature images at multiple scales are H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, then the target feature image at the first scale is the target feature image at H / 8*W / 8*C4, and the target feature image at the second scale is H / 4*W / 4*C3.

[0106] The second feature map at the second scale is then fused with the first feature map at the same scale using the feature fusion module to generate a feature fusion map at the second scale. For example, feature fusion of the second feature map of H / 4*W / 4*C3 and the first feature map of H / 4*W / 4*C3 yields a feature fusion map of H / 4*W / 4*C3.

[0107] After obtaining the feature fusion map at the second scale, the local feature extraction and feature fusion steps will be repeated until the feature fusion map at the Nth scale is obtained. It should be noted that the feature fusion map at the Nth scale contains image segmentation information and can be directly used as the image segmentation result mentioned above.

[0108] It should be noted that the Nth scale is the scale corresponding to the first image. That is, when the target model segments the first image, it first obtains second images at multiple scales by downsampling through OPE, and then performs upsampling and downsampling processing on the first image and the second images at multiple scales through GSB and LSB to realize the encoding process. Then, it performs downsampling on the encoded target feature image through GSB and LSB to realize the decoding process, thereby obtaining the final image segmentation result.

[0109] By downsampling the input image using OPE, constructing multi-scale encoding and decoding layers using LSB, and introducing GSB between the encoding and decoding layers, we can ensure that global and local features can be continuously transmitted without confusion, thereby achieving the technical effect of generating higher quality image segmentation results.

[0110] To improve the accuracy of image segmentation, in the image segmentation method provided in Embodiment 1 of this application, the target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module. The embedding module and the local feature extraction module are connected by the global feature extraction module, and the local feature extraction modules are connected by the global feature extraction module.

[0111] Optionally, the target model consists of an embedding module (OPE), a local feature extraction module (LSB), a global feature extraction module (GSB), and a feature fusion module. GSBs connect the OPE and LSBs, and GSBs also connect the LSBs to each other. That is, the LSBs construct multi-scale encoding and decoding layers, and GSBs are introduced between the encoding and decoding layers, such as... Figure 7 As shown, through the above connection relationships, the network structure of the target model presents a W-shape.

[0112] In an optional embodiment, the target model consists of 3 OPEs, 9 LSBs, 9 GSBs, and 9 Fus (feature fusion modules), as follows: Figure 7 As shown, the network structure of the target model is W-shaped.

[0113] In an optional embodiment, the target model may also consist of 4 OPEs, 12 LSBs, 12 GSBs, and 12 Fus, as shown below. Figure 8 As shown.

[0114] In an optional embodiment, where the target model consists of 3 OPEs, 9 LSBs, and 9 GSBs, a schematic diagram of image segmentation is shown below. Figure 9 As shown, the specific steps include: downsampling the first image H*W*C1 using three OPEs to obtain second images H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4; processing the second image H / 4*W / 4*C3 using GSB and processing the second image H / 8*W / 8*C4 using LSB to obtain the third image H / 4*W / 4*C3; processing the second image H / 2*W / 2*C2 using GSB and processing the third image H / 4*W / 4*C3 using LSB to obtain the third image H / 2*W / 2*C2; and processing the first image H*W*C1 using GSB and processing the third image H / 2*W / 2*C2 using LSB to obtain the third image H*W*C1.

[0115] Then, the third image of H / 2*W / 2*C2 is processed by GSB, and the third image of H*W*C1 is processed by LSB to obtain the target feature image of H / 2*W / 2*C2; the third image of H / 4*W / 4*C3 is processed by GSB, and the target feature image of H / 2*W / 2*C2 is processed by LSB to obtain the target feature image of H / 4*W / 4*C3; the second image of H / 8*W / 8*C4 is processed by GSB, and the target feature image of H / 4*W / 4*C3 is processed by LSB to obtain the target feature image of H / 8*W / 8*C4.

[0116] Next, the target feature image of H / 4*W / 4*C3 is processed using GSB, and the target feature image of H / 8*W / 8*C4 is processed using LSB to obtain the feature fusion map of H / 4*W / 4*C3; the target feature image of H / 2*W / 2*C2 is processed using GSB, and the feature fusion map of H / 4*W / 4*C3 is processed using LSB to obtain the feature fusion map of H / 2*W / 2*C2; the third image of H*W*C1 is processed using GSB, and the feature fusion map of H / 2*W / 2*C2 is processed using LSB to obtain the feature fusion map of H*W*C1, which is the final image segmentation result.

[0117] Multi-scale image feature representations were constructed by downsampling the original image at different levels using three OPEs. The combined use of GSB and LSB modules ensured information transmission and fusion between features at different scales. GSB can handle global features, maintaining the integrity of the macroscopic structure, while LSB focuses on extracting local features. The interaction between the two not only enhances feature representation but also avoids information loss during scale transformation, thereby improving the technical effect of image segmentation accuracy.

[0118] In an optional embodiment, the target model was validated using multiple datasets (e.g., retinal vessel segmentation datasets, skin lesion segmentation datasets, etc.), and the Jaccard index was used as a performance metric to evaluate the segmentation results. Regarding the Jaccard index, the target model achieved a significance level below 0.05 in 28 out of 32 t-tests, reaching a significance rate of 87.50%. This demonstrates that the target model has a statistically significant advantage and improvement.

[0119] In the image segmentation method provided in Embodiment 1 of this application, the first image to be segmented is downsampled by the embedding module of the target model to obtain a second image at multiple scales; the first target image is encoded at multiple scales by the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image; the target feature images at multiple scales are decoded at multiple scales by the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image, thereby solving the technical problem in related technologies that segment images based on local features, resulting in relatively low accuracy of image segmentation.

[0120] In this scheme, the first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales. Then, the first target image is encoded using the global feature extraction module and the local feature extraction module in the target model to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and represented, avoiding the information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the fusion of rich multi-scale information and the effective fusion between global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0121] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0123] Example 2

[0124] According to embodiments of this application, an image segmentation method is also provided, such as... Figure 10 As shown, the image segmentation method includes:

[0125] Step S1001: The first medical image is downsampled using the embedding module of the target model to obtain a second medical image at multiple scales.

[0126] Step S1002: The first target medical image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales. The first target medical image is obtained from the second medical image at multiple scales and the first medical image.

[0127] Step S1003: Multi-scale decoding processing is performed on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first medical image.

[0128] It should be noted that the specific method for segmenting medical images is the same as in Example 1, and will not be repeated here.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0131] Example 3

[0132] According to embodiments of this application, an image segmentation method is also provided, such as... Figure 11 As shown, the image segmentation method includes:

[0133] Step S1101: Obtain the first image to be segmented uploaded by the client;

[0134] Step S1102: In the cloud server, the first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales; the first target image is encoded at multiple scales using the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second images at multiple scales and the first image; the target feature images at multiple scales are decoded at multiple scales using the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image.

[0135] Step S1103: Return the image segmentation result to the client.

[0136] It should be noted that the specific method for image segmentation in the cloud server is the same as in Example 1, and will not be repeated here.

[0137] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0139] Example 4

[0140] According to embodiments of this application, an image segmentation apparatus for implementing the above-described image segmentation method is also provided, such as... Figure 12 As shown, the device includes: a first processing unit 1201, a second processing unit 1202, and a determination unit 1203.

[0141] The first processing unit 1201 is used to perform downsampling processing on the first image to be segmented through the embedding module of the target model to obtain a second image at multiple scales;

[0142] The second processing unit 1202 is used to perform multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image;

[0143] The determining unit 1203 is used to perform multi-scale decoding processing on target feature images of multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image.

[0144] In the image segmentation apparatus provided in Embodiment 4 of this application, the first processing unit 1201 performs downsampling processing on the first image to be segmented through the embedding module of the target model to obtain second images at multiple scales; the second processing unit 1202 performs multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second images at multiple scales and the first image; the determination unit 1203 performs multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image, which solves the technical problem in related technologies that segmenting images based on local features of the image results in relatively low accuracy of image segmentation.

[0145] In this scheme, the first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales. Then, the first target image is encoded using the global feature extraction module and the local feature extraction module in the target model to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and represented, avoiding the information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the fusion of rich multi-scale information and the effective fusion between global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0146] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the second processing unit includes: a first processing subunit, used to upsample the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; and a second processing subunit, used to downsample the second target image through the global feature extraction module and the local feature extraction module to obtain a target feature image at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

[0147] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the first processing subunit includes: a first extraction module, configured to perform global feature extraction on the first image and the fourth image through a global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a partial image in the second images at multiple scales; a second extraction module, configured to perform local feature extraction on the second image at the first scale in the second images at multiple scales through a local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and a first determination module, configured to obtain a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales.

[0148] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the first determining module includes: a first fusion submodule, used to perform feature fusion on the second initial feature image at the second scale and the first initial feature image at the second scale among multiple scales of the first initial feature image through the feature fusion module in the target model to obtain a third image at the second scale; a first extraction submodule, used to perform local feature extraction on the third image at the second scale through the local feature extraction module to obtain a second initial feature image at the third scale; a second fusion submodule, used to perform feature fusion on the second initial feature image at the third scale and the second initial feature image at the third scale among multiple scales of the first initial feature image through the feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; and a first determining submodule, used to repeatedly execute the step of performing local feature extraction on the third image at the third scale through the local feature extraction module until all images in the first initial feature images at multiple scales have been processed, and to determine the currently obtained third image as the third image at multiple scales.

[0149] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the second processing subunit includes: a third extraction module, configured to perform global feature extraction on the fifth image and the second image at a first target scale in the second images at multiple scales through a global feature extraction module to obtain a third initial feature image at multiple scales, wherein the fifth image is a partial image in the third images at multiple scales, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the second images at multiple scales; a fourth extraction module, configured to perform local feature extraction on the third image at the Nth scale in the third images at multiple scales through a local feature extraction module to obtain a fourth initial feature image at the N-1th scale; and a second determination module, configured to obtain a target feature image at multiple scales based on the fourth initial feature image at the N-1th scale and the third initial feature image at multiple scales.

[0150] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the second determining module includes: a third fusion submodule, used to perform feature fusion on the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale among multiple scales of third initial feature images through the feature fusion module in the target model to obtain a target feature image at the N-1 scale; a second extraction submodule, used to perform local feature extraction on the target feature image at the N-1 scale through the local feature extraction module to obtain a fourth initial feature image at the N-2 scale; a fourth fusion submodule, used to perform feature fusion on the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale among multiple scales of third initial feature images through the feature fusion module to obtain a target feature image at the N-2 scale; and a second determining submodule, used to repeatedly execute the step of performing local feature extraction on the target feature image at the N-2 scale through the local feature extraction module until all images in the third initial feature images at multiple scales have been processed, and to determine the currently obtained target feature image as the target feature image at multiple scales.

[0151] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the determining unit includes: a first extraction subunit, used to perform global feature extraction on the sixth image and the seventh image through a global feature extraction module to obtain a first feature map at multiple scales, wherein the sixth image is a partial image among the target feature images at multiple scales, and the seventh image is an image at the second target scale among the third images at multiple scales; a second extraction subunit, used to perform local feature extraction on the target feature image at the first scale among the target feature images at multiple scales through a local feature extraction module to obtain a second feature map at the second scale; a fusion subunit, used to perform feature fusion on the second feature map at the second scale and the first feature map at the second scale among the first feature images at multiple scales to obtain a feature fusion map at the second scale; and a determining subunit, used to repeatedly execute the step of performing local feature extraction on the feature fusion map at the second scale through the local feature extraction module until a feature fusion map at the Nth scale is obtained, and to obtain an image segmentation result based on the feature fusion map at the Nth scale.

[0152] Optionally, in the image segmentation apparatus provided in Embodiment 4 of this application, the target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module, wherein the embedding module and the local feature extraction module are connected by a global feature extraction module, and the local feature extraction modules are connected by a global feature extraction module.

[0153] It should be noted that the first processing unit 1201, the second processing unit 1202, and the determining unit 1203 mentioned above correspond to steps S201 to S203 in Embodiment 1. The three units and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run on the computer terminal 10 provided in Embodiment 1.

[0154] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0155] Example 5

[0156] Embodiments of this application may provide an electronic device, which may be any one of a group of electronic device terminals. Optionally, in this embodiment, the aforementioned electronic device may also be replaced by a terminal device such as a mobile terminal.

[0157] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0158] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: downsampling the first image to be segmented through the embedding module of the target model to obtain a second image at multiple scales; multi-scale encoding processing of the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image; multi-scale decoding processing of the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image.

[0159] The aforementioned electronic device can execute the following steps in the image segmentation method: performing multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, including: performing upsampling processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; performing downsampling processing on the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

[0160] The aforementioned electronic device can execute the following steps in the image segmentation method: Upsampling the first target image using the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales, including: performing global feature extraction on the first and fourth images using the global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a portion of the second images at multiple scales; performing local feature extraction on the second image at the first scale using the local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and obtaining a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales.

[0161] The aforementioned electronic device can execute the following steps in the image segmentation method: obtaining a third image at multiple scales based on a second initial feature image at a second scale and first initial feature images at multiple scales includes: fusing features of the second initial feature image at the second scale and the first initial feature images at multiple scales from the second initial feature image at the second scale using a feature fusion module in the target model to obtain a third image at the second scale; extracting local features from the third image at the second scale using a local feature extraction module to obtain a second initial feature image at the third scale; fusing features of the second initial feature image at the third scale and the second initial feature image at the third scale from the first initial feature images at multiple scales using a feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; and repeatedly executing the step of extracting local features from the third image at the third scale using a local feature extraction module until all images in the first initial feature images at multiple scales have been processed, and determining the currently obtained third image as the third image at multiple scales.

[0162] The aforementioned electronic device can execute the following steps in the image segmentation method: Downsampling the second target image using a global feature extraction module and a local feature extraction module to obtain target feature images at multiple scales, including: performing global feature extraction on the fifth image and the second image at the first target scale from the second images at multiple scales using the global feature extraction module to obtain third initial feature images at multiple scales, wherein the fifth image is a partial image from the third images at multiple scales, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the second images at multiple scales; performing local feature extraction on the third image at the Nth scale from the third images at multiple scales using the local feature extraction module to obtain a fourth initial feature image at the N-1th scale; and obtaining target feature images at multiple scales based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales.

[0163] The aforementioned electronic device can execute the following steps in the image segmentation method: obtaining target feature images at multiple scales based on the fourth initial feature image at the N-1 scale and the third initial feature images at multiple scales, including: fusing features of the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale from the third initial feature images at multiple scales through the feature fusion module in the target model to obtain the target feature image at the N-1 scale; extracting local features from the target feature image at the N-1 scale through the local feature extraction module to obtain the fourth initial feature image at the N-2 scale; fusing features of the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale from the third initial feature images at multiple scales through the feature fusion module to obtain the target feature image at the N-2 scale; repeatedly executing the step of extracting local features from the target feature image at the N-2 scale through the local feature extraction module until all images in the third initial feature images at multiple scales have been processed, and determining the currently obtained target feature image as the target feature image at multiple scales.

[0164] The aforementioned electronic device can execute the following steps in the image segmentation method: Multi-scale decoding of target feature images at multiple scales using a global feature extraction module and a local feature extraction module to obtain the image segmentation result corresponding to the first image, including: global feature extraction of the sixth and seventh images using the global feature extraction module to obtain first feature maps at multiple scales, wherein the sixth image is a partial image among the target feature images at multiple scales, and the seventh image is the second target scale image among the third images at multiple scales; local feature extraction of the first scale target feature image in the target feature images at multiple scales using the local feature extraction module to obtain a second feature map at the second scale; feature fusion of the second feature map at the second scale and the first feature map at the second scale in the first feature images at multiple scales to obtain a feature fusion map at the second scale; repeating the step of local feature extraction of the feature fusion map at the second scale using the local feature extraction module until a feature fusion map at the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map at the Nth scale.

[0165] The aforementioned electronic device can execute the following steps in the image segmentation method: The target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0166] Optionally, Figure 13 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 13 As shown, the electronic device 130 may include: one or more ( Figure 13 (Only one is shown in the image) Processor 1302 and memory 1304. The electronic device 130 may also include a memory controller to control and manage the memory 1304; the electronic device 130 may also include a peripheral interface to connect to a radio frequency module, an audio module, and a display screen, etc.

[0167] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image segmentation method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned image segmentation method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the electronic device 130 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] The processor can access the information and application program stored in the memory via the transmission device to perform the following steps: downsampling the first image to be segmented through the embedding module of the target model to obtain a second image at multiple scales; multi-scale encoding of the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at multiple scales and the first image; multi-scale decoding of the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image.

[0169] Optionally, the processor may also execute program code for the following steps: performing multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, including: performing upsampling processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; performing downsampling processing on the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

[0170] Optionally, the processor may also execute program code for the following steps: upsampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales, including: performing global feature extraction on the first image and the fourth image through the global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a partial image in the second image at multiple scales; performing local feature extraction on the second image at the first scale in the second image at multiple scales through the local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and obtaining a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales.

[0171] Optionally, the processor may also execute program code for the following steps: obtaining a third image at multiple scales based on a second initial feature image at a second scale and first initial feature images at multiple scales includes: fusing features of the second initial feature image at the second scale and the first initial feature images at multiple scales from the second initial feature image at the second scale using a feature fusion module in the target model to obtain a third image at the second scale; extracting local features from the third image at the second scale using a local feature extraction module to obtain a second initial feature image at the third scale; fusing features of the second initial feature image at the third scale and the second initial feature image at the third scale from the first initial feature images at multiple scales using a feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; and repeatedly executing the step of extracting local features from the third image at the third scale using a local feature extraction module until all images in the first initial feature images at multiple scales have been processed, and determining the currently obtained third image as the third image at multiple scales.

[0172] Optionally, the processor may also execute program code for the following steps: Downsampling the second target image using a global feature extraction module and a local feature extraction module to obtain target feature images at multiple scales, including: performing global feature extraction on the fifth image and the second image at the first target scale from the second images at multiple scales using the global feature extraction module to obtain third initial feature images at multiple scales, wherein the fifth image is a partial image from the third images at multiple scales, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the second images at multiple scales; performing local feature extraction on the third image at the Nth scale from the third images at multiple scales using the local feature extraction module to obtain a fourth initial feature image at the N-1th scale; and obtaining target feature images at multiple scales based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales.

[0173] Optionally, the processor may also execute program code for the following steps: obtaining target feature images at multiple scales based on the fourth initial feature image at the N-1 scale and the third initial feature images at multiple scales includes: fusing features of the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale from the third initial feature images at multiple scales through the feature fusion module in the target model to obtain the target feature image at the N-1 scale; extracting local features from the target feature image at the N-1 scale through the local feature extraction module to obtain the fourth initial feature image at the N-2 scale; fusing features of the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale from the third initial feature images at multiple scales through the feature fusion module to obtain the target feature image at the N-2 scale; and repeatedly executing the step of extracting local features from the target feature image at the N-2 scale through the local feature extraction module until all images in the third initial feature images at multiple scales have been processed, and determining the currently obtained target feature image as the target feature image at multiple scales.

[0174] Optionally, the processor may also execute program code for the following steps: performing multi-scale decoding processing on target feature images at multiple scales through a global feature extraction module and a local feature extraction module to obtain the image segmentation result corresponding to the first image, including: performing global feature extraction on the sixth and seventh images through the global feature extraction module to obtain first feature maps at multiple scales, wherein the sixth image is a partial image in the target feature images at multiple scales, and the seventh image is the second target scale image in the third images at multiple scales; performing local feature extraction on the target feature images at the first scale in the target feature images at multiple scales through the local feature extraction module to obtain a second feature map at the second scale; performing feature fusion on the second feature map at the second scale and the first feature map at the second scale in the first feature images at multiple scales to obtain a feature fusion map at the second scale; repeatedly executing the step of performing local feature extraction on the feature fusion map at the second scale through the local feature extraction module until the feature fusion map at the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map at the Nth scale.

[0175] Optionally, the processor may also execute program code with the following steps: the target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module, wherein the embedding module and the local feature extraction module are connected by the global feature extraction module, and the local feature extraction modules are connected by the global feature extraction module.

[0176] Those skilled in the art will understand that Figure 13The structure shown is for illustrative purposes only. Electronic device 130 can also be a smartphone, tablet computer, handheld computer, mobile internet device (MID), PAD and other terminal devices. Figure 13 This does not limit the structure of the aforementioned electronic device. For example, electronic device 130 may also include components that are more... Figure 13 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 13 The different configurations shown.

[0177] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0178] Example 6

[0179] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product can be used to store the program code executed by the image segmentation method provided in Embodiment 1.

[0180] Optionally, in this embodiment, the computer program product may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0181] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0182] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0184] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0186] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0187] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image segmentation method, characterized in that, include: The first image to be segmented is downsampled using the embedding module of the target model to obtain a second image at multiple scales; The first target image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at the multiple scales and the first image; The target feature images at multiple scales are decoded using the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image. Specifically, the first target image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, including: The global feature extraction module performs global feature extraction on the first image and the fourth image to obtain a first initial feature image at multiple scales, wherein the fourth image is a portion of the second image at multiple scales; The local feature extraction module extracts local features from the second image at the first scale among the second images at multiple scales to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale. The feature fusion module in the target model performs feature fusion on the second initial feature image at the second scale and the first initial feature image at the second scale among the multiple scales to obtain the third image at the second scale. The local feature extraction module extracts local features from the third image at the second scale to obtain a second initial feature image at the third scale. The feature fusion module performs feature fusion on the second initial feature image at the third scale and the first initial feature image at the third scale from the multiple scales of first initial feature images to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; Repeat the step of extracting local features from the third image at the third scale using the local feature extraction module until all images in the first initial feature images at the multiple scales have been processed, and the currently obtained third image is determined as the third image at the multiple scales; The second target image is downsampled by the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

2. The method according to claim 1, characterized in that, The second target image is downsampled using the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, including: The global feature extraction module performs global feature extraction on the fifth image and the second image at the first target scale among the multiple scales of the second images to obtain a third initial feature image at multiple scales. The fifth image is a partial image among the multiple scales of the third images, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the multiple scales of the second images. The local feature extraction module extracts local features from the third image at the Nth scale among the multiple scales of the third image to obtain the fourth initial feature image at the N-1th scale. Based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales, the target feature images at multiple scales are obtained.

3. The method according to claim 2, characterized in that, Based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales, the target feature images at the multiple scales are obtained as follows: The feature fusion module in the target model performs feature fusion on the fourth initial feature image at the N-1 scale and the third initial feature image at the N-1 scale among the multiple scales of third initial feature images to obtain the target feature image at the N-1 scale. The local feature extraction module extracts local features from the target feature image at the N-1 scale to obtain the fourth initial feature image at the N-2 scale. The feature fusion module performs feature fusion on the fourth initial feature image at the N-2 scale and the third initial feature image at the N-2 scale from among the multiple scales of third initial feature images to obtain the target feature image at the N-2 scale. Repeat the step of extracting local features from the target feature image at the N-2 scale through the local feature extraction module until all images in the third initial feature image at the multiple scales have been processed, and determine the currently obtained target feature image as the target feature image at the multiple scales.

4. The method according to claim 1, characterized in that, The image segmentation result corresponding to the first image is obtained by performing multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module, including: The global feature extraction module performs global feature extraction on the sixth and seventh images to obtain first feature maps at multiple scales. The sixth image is a partial image among the target feature maps at multiple scales, and the seventh image is an image at the second target scale among the third images at multiple scales. The local feature extraction module extracts local features from the target feature image at the first scale among the target feature images at multiple scales to obtain a second feature map at the second scale. The second feature map at the second scale and the first feature map at the second scale in the first feature maps of the plurality of scales are fused to obtain a feature fused map at the second scale. The step of extracting local features from the feature fusion map at the second scale through the local feature extraction module is repeated until the feature fusion map at the Nth scale is obtained, and the image segmentation result is obtained based on the feature fusion map at the Nth scale.

5. The method according to claim 1, characterized in that, The target model consists of an embedding module, a local feature extraction module, a global feature extraction module, and a feature fusion module. The global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

6. An image segmentation method, characterized in that, include: The first medical image is downsampled by the embedding module of the target model to obtain a second medical image at multiple scales. The first target medical image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales. The first target medical image is obtained from the second medical image at multiple scales and the first medical image. The target feature images at multiple scales are decoded by the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first medical image; Specifically, the first target medical image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, including: The global feature extraction module performs global feature extraction on the first medical image and the fourth medical image to obtain a first initial feature image at multiple scales, wherein the fourth medical image is a portion of the second medical image at multiple scales; The local feature extraction module extracts local features from the second medical image at the first scale of the second medical image at multiple scales to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale. The feature fusion module in the target model performs feature fusion on the second initial feature image at the second scale and the first initial feature image at the second scale among the multiple scales to obtain the third medical image at the second scale. The local feature extraction module extracts local features from the third medical image at the second scale to obtain a second initial feature image at the third scale. The feature fusion module performs feature fusion on the second initial feature image at the third scale and the first initial feature image at the third scale from the multiple scales of first initial feature images to obtain a third medical image at the third scale, wherein the second scale is smaller than the third scale; Repeat the step of extracting local features from the third medical image at the third scale through the local feature extraction module until all images in the first initial feature images at multiple scales have been processed, and the currently obtained third medical image is determined as the third medical image at multiple scales. The second target image is downsampled by the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second medical image at multiple scales and the third medical image at multiple scales.

7. An image segmentation method, characterized in that, include: Obtain the first image to be segmented uploaded by the client; In a cloud server, the first image to be segmented is downsampled using the embedding module of the target model to obtain second images at multiple scales. The first target image is then encoded at multiple scales using the global and local feature extraction modules of the target model to obtain target feature images at multiple scales, wherein the first target image is derived from the second images at multiple scales and the first image. Finally, the target feature images at multiple scales are decoded using the global and local feature extraction modules to obtain the image segmentation result corresponding to the first image. The image segmentation result is returned to the client. Specifically, the first target image is processed by multi-scale encoding through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, including: The global feature extraction module performs global feature extraction on the first image and the fourth image to obtain a first initial feature image at multiple scales, wherein the fourth image is a portion of the second image at multiple scales; The local feature extraction module extracts local features from the second image at the first scale among the second images at multiple scales to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale. The feature fusion module in the target model performs feature fusion on the second initial feature image at the second scale and the first initial feature image at the second scale among the multiple scales to obtain the third image at the second scale. The local feature extraction module extracts local features from the third image at the second scale to obtain a second initial feature image at the third scale. The feature fusion module performs feature fusion on the second initial feature image at the third scale and the first initial feature image at the third scale from the multiple scales of first initial feature images to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; Repeat the step of extracting local features from the third image at the third scale using the local feature extraction module until all images in the first initial feature images at the multiple scales have been processed, and the currently obtained third image is determined as the third image at the multiple scales; The second target image is downsampled by the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales.

8. An image segmentation apparatus, characterized in that, include: The first processing unit is used to downsample the first image to be segmented through the embedding module of the target model to obtain a second image at multiple scales; The second processing unit is used to perform multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at the multiple scales and the first image; The determining unit is used to perform multi-scale decoding processing on the target feature images of multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image; The second processing unit includes: a first processing subunit, used to upsample the first target image using the global feature extraction module and the local feature extraction module of the target model to obtain a third image at multiple scales; and a second processing subunit, used to downsample the second target image using the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales, wherein the second target image is obtained from the second image at multiple scales and the third image at multiple scales. The first processing subunit includes: a first extraction module, configured to perform global feature extraction on the first image and the fourth image through the global feature extraction module to obtain a first initial feature image at multiple scales, wherein the fourth image is a portion of the second images at multiple scales; a second extraction module, configured to perform local feature extraction on the second image at the first scale in the second images at multiple scales through the local feature extraction module to obtain a second initial feature image at the second scale, wherein the first scale is smaller than the second scale; and a first determination module, configured to obtain a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales. The first determining module includes: a first fusion submodule, used to perform feature fusion on the second initial feature image at the second scale and the first initial feature image at the second scale among the multiple scales of the first initial feature images through the feature fusion module in the target model to obtain a third image at the second scale; a first extraction submodule, used to perform local feature extraction on the third image at the second scale through the local feature extraction module to obtain a second initial feature image at the third scale; a second fusion submodule, used to perform feature fusion on the second initial feature image at the third scale and the first initial feature image at the third scale among the multiple scales of the first initial feature images through the feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; and a first determining submodule, used to repeatedly execute the step of performing local feature extraction on the third image at the third scale through the local feature extraction module until all images among the multiple scales of the first initial feature images have been processed, and to determine the currently obtained third image as the third image at the multiple scales.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the storage medium is located to perform the image segmentation method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the image segmentation method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program or instructions that, when executed by a processor, implement the image segmentation method according to any one of claims 1 to 7.

Images