Image segmentation method and device, storage medium and electronic equipment

Abstract

The invention discloses an image segmentation method and device, a storage medium and electronic equipment. Relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the down-sampling processing of a to-be-segmented first image through an embedding module of a target model, and obtaining second images of a plurality of scales; multi-scale coding processing is carried out on a first target image through a global feature extraction module and a local feature extraction module of the target model, target feature images of multiple scales are obtained, and the first target image is obtained from second images of multiple scales and the first image; and performing multi-scale decoding processing on the target feature images of the multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image. The image segmentation method and device solve the technical problem that the accuracy of image segmentation is low due to the fact that the image is segmented according to the local features of the image in the prior art.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image segmentation method and device, a storage medium, and an electronic device. Background Art

[0002] Image segmentation, a core task in image processing and analysis, particularly in the development of computer vision technology, is crucial for precision medical diagnosis, autonomous driving systems, and remote sensing image analysis. While traditional methods, including threshold-based segmentation, edge detection, and early machine learning models, perform well in specific scenarios, they often face challenges when processing complex images, such as those with diverse textures, varying lighting, and blurred boundaries.

[0003] Medical imaging, such as pathology image analysis, relies heavily on the precise identification and segmentation of skin lesions and tissue structures. However, existing technologies often focus too much on local image features and overlook the importance of global context. While local features can capture detailed information, they struggle to accurately identify lesions with blurred boundaries and variable morphology.

[0004] With respect to the problem in the above-mentioned related technologies that image segmentation is performed based on local features of the image, resulting in relatively low accuracy of image segmentation, no effective solution has been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide an image segmentation method and device, a storage medium, and an electronic device to at least solve the technical problem in the related art that image segmentation is performed based on local features of the image, resulting in relatively low accuracy of image segmentation.

[0006] According to one aspect of an embodiment of the present application, an image segmentation method is provided, comprising: performing downsampling processing on a first image to be segmented by an embedding module of a target model to obtain second images of multiple scales; performing multi-scale encoding processing on the first target image by a global feature extraction module and a local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images of the multiple scales and the first image; performing multi-scale decoding processing on the target feature images of the multiple scales by the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0007] Furthermore, multi-scale encoding processing is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, including: up-sampling processing is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales; down-sampling processing is performed on the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of the multiple scales, wherein the second target image is obtained from the second image of the multiple scales and the third image of the multiple scales.

[0008] Furthermore, upsampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain third images of multiple scales includes: performing global feature extraction on the first image and the fourth image through the global feature extraction module to obtain first initial feature images of multiple scales, wherein the fourth image is a partial image of the second images of the multiple scales; performing local feature extraction on the second image of the first scale among the second images of the multiple scales through the local feature extraction module to obtain a second initial feature image of the second scale, wherein the first scale is smaller than the second scale; and obtaining the third images of the multiple scales based on the second initial feature image of the second scale and the first initial feature images of the multiple scales.

[0009] Furthermore, based on the second initial feature image of the second scale and the first initial feature images of the multiple scales, obtaining the third images of the multiple scales includes: performing feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale in the first initial feature images of the multiple scales through the feature fusion module in the target model to obtain a third image of the second scale; performing local feature extraction on the third image of the second scale through the local feature extraction module to obtain a second initial feature image of the third scale; performing feature fusion on the second initial feature image of the third scale and the second initial feature image of the third scale in the first initial feature images of the multiple scales through the feature fusion module to obtain a third image of the third scale, wherein the second scale is smaller than the third scale; repeatedly performing the step of performing local feature extraction on the third image of the third scale through the local feature extraction module until all images in the first initial feature images of the multiple scales have been processed, and determining the currently obtained third image as the third image of the multiple scales.

[0010] Furthermore, downsampling the second target image by the global feature extraction module and the local feature extraction module to obtain the target feature images of the multiple scales includes: performing global feature extraction on the fifth image and the second image of the first target scale among the second images of the multiple scales by the global feature extraction module to obtain third initial feature images of multiple scales, wherein the fifth image is a partial image of the third images of the multiple scales, and the scale of the second image of the first target scale is smaller than the scale of the remaining images of the second images of the multiple scales; performing local feature extraction on the third image of the Nth scale among the third images of the multiple scales by the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; and obtaining the target feature images of the multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of the multiple scales.

[0011] Furthermore, obtaining the target feature images of the multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature images of the multiple scales includes: performing feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale among the third initial feature images of the multiple scales by a feature fusion module in the target model to obtain the target feature image of the N-1th scale; performing local feature extraction on the target feature image of the N-1th scale by the local feature extraction module to obtain the fourth initial feature image of the N-2th scale; performing feature fusion on the fourth initial feature image of the N-2th scale and the third initial feature image of the N-2th scale among the third initial feature images of the multiple scales by the feature fusion module to obtain the target feature image of the N-2th scale; repeatedly performing the step of performing local feature extraction on the target feature image of the N-2th scale by the local feature extraction module until all images in the third initial feature images of the multiple scales have been processed, and determining the currently obtained target feature image as the target feature image of the multiple scales.

[0012] Furthermore, multi-scale decoding processing is performed on the target feature images of the multiple scales by the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image, including: performing global feature extraction on the sixth image and the seventh image by the global feature extraction module to obtain first feature maps of multiple scales, wherein the sixth image is a partial image in the target feature images of the multiple scales, and the seventh image is an image of the second target scale in the third image of the multiple scales; performing local feature extraction on the target feature image of the first scale in the target feature images of the multiple scales by the local feature extraction module to obtain a second feature map of the second scale; performing feature fusion on the second feature map of the second scale and the first feature map of the second scale in the first feature maps of the multiple scales to obtain a feature fusion map of the second scale; repeatedly performing the step of performing local feature extraction on the feature fusion map of the second scale by the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map of the Nth scale.

[0013] Furthermore, the target model is composed of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0014] According to another aspect of an embodiment of the present application, an image segmentation method is also provided, including: downsampling a first medical image through an embedding module of a target model to obtain a second medical image at multiple scales; performing multi-scale encoding processing on the first target medical image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target medical image is obtained from the second medical image images at multiple scales and the first medical image; performing multi-scale decoding processing on the target feature images at multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first medical image.

[0015] According to another aspect of an embodiment of the present application, an image segmentation method is also provided, including: obtaining a first image to be segmented uploaded by a client; downsampling the first image to be segmented through an embedded module of a target model in a cloud server to obtain second images of multiple scales; performing multi-scale encoding processing on the first target image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images of the multiple scales and the first image; performing multi-scale decoding processing on the target feature images of the multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image; and returning the image segmentation result to the client.

[0016] According to another aspect of an embodiment of the present application, an image segmentation device is also provided, including: a first processing unit, used to downsample the first image to be segmented through an embedding module of a target model to obtain second images of multiple scales; a second processing unit, used to perform multi-scale encoding processing on the first target image through a global feature extraction module and a local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images of the multiple scales and the first image; a determination unit, used to perform multi-scale decoding processing on the target feature images of the multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0017] Furthermore, the second processing unit includes: a first processing subunit, used to upsample the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales; a second processing subunit, used to downsample the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of the multiple scales, wherein the second target image is obtained from the second image of the multiple scales and the third image of the multiple scales.

[0018] Furthermore, the first processing subunit includes: a first extraction module, used to perform global feature extraction on the first image and the fourth image through the global feature extraction module to obtain first initial feature images of multiple scales, wherein the fourth image is a partial image of the second images of the multiple scales; a second extraction module, used to perform local feature extraction on the second image of the first scale among the second images of the multiple scales through the local feature extraction module to obtain a second initial feature image of the second scale, wherein the first scale is smaller than the second scale; and a first determination module, used to obtain a third image of the multiple scales based on the second initial feature image of the second scale and the first initial feature images of the multiple scales.

[0019] Furthermore, the first determination module includes: a first fusion submodule, used to perform feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale among the first initial feature images of the multiple scales through the feature fusion module in the target model to obtain a third image of the second scale; a first extraction submodule, used to perform local feature extraction on the third image of the second scale through the local feature extraction module to obtain a second initial feature image of the third scale; a second fusion submodule, used to perform feature fusion on the second initial feature image of the third scale and the second initial feature image of the third scale among the first initial feature images of the multiple scales through the feature fusion module to obtain a third image of the third scale, wherein the second scale is smaller than the third scale; and a first determination submodule, used to repeatedly perform the step of performing local feature extraction on the third image of the third scale through the local feature extraction module until all images in the first initial feature images of the multiple scales have been processed, and the currently obtained third image is determined as the third image of the multiple scales.

[0020] Furthermore, the second processing subunit includes: a third extraction module, used to perform global feature extraction on the fifth image and the second image of the first target scale among the second images of the multiple scales through the global feature extraction module to obtain third initial feature images of multiple scales, wherein the fifth image is a partial image of the third images of the multiple scales, and the scale of the second image of the first target scale is smaller than the scale of the remaining images of the second images of the multiple scales; a fourth extraction module, used to perform local feature extraction on the third image of the Nth scale among the third images of the multiple scales through the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; a second determination module, used to obtain the target feature images of the multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of the multiple scales.

[0021] Furthermore, the second determination module includes: a third fusion submodule, used to perform feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale among the third initial feature images of the multiple scales through the feature fusion module in the target model to obtain a target feature image of the N-1th scale; a second extraction submodule, used to perform local feature extraction on the target feature image of the N-1th scale through the local feature extraction module to obtain a fourth initial feature image of the N-2th scale; a fourth fusion submodule, used to perform feature fusion on the fourth initial feature image of the N-2th scale and the third initial feature image of the N-2th scale among the third initial feature images of the multiple scales through the feature fusion module to obtain a target feature image of the N-2th scale; and a second determination submodule, used to repeatedly perform the step of performing local feature extraction on the target feature image of the N-2th scale through the local feature extraction module until all images in the third initial feature images of the multiple scales have been processed, and the currently obtained target feature image is determined as the target feature image of the multiple scales.

[0022] Furthermore, the determination unit includes: a first extraction subunit, used to perform global feature extraction on the sixth image and the seventh image through the global feature extraction module to obtain first feature maps of multiple scales, wherein the sixth image is a partial image in the target feature images of the multiple scales, and the seventh image is an image of the second target scale in the third image of the multiple scales; a second extraction subunit, used to perform local feature extraction on the target feature image of the first scale in the target feature images of the multiple scales through the local feature extraction module to obtain a second feature map of the second scale; a fusion subunit, used to perform feature fusion on the second feature map of the second scale and the first feature map of the second scale in the first feature maps of the multiple scales to obtain a feature fusion map of the second scale; a determination subunit, used to repeatedly perform the step of performing local feature extraction on the feature fusion map of the second scale through the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtain the image segmentation result based on the feature fusion map of the Nth scale.

[0023] Furthermore, the target model is composed of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0024] According to another aspect of an embodiment of the present invention, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the above-mentioned image segmentation methods when running.

[0025] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The storage medium stores a program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned image segmentation methods.

[0026] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program or instructions, which implements any one of the above-mentioned image segmentation methods when executed by a processor.

[0027] In an embodiment of the present application, the following steps are adopted: downsampling the first image to be segmented is performed through the embedding module of the target model to obtain second images of multiple scales; multi-scale encoding is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images of the multiple scales and the first image; multi-scale decoding is performed on the target feature images of the multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image, thereby solving the technical problem in the related art that the image is segmented according to the local features of the image, resulting in relatively low accuracy of image segmentation.

[0028] In this solution, the first image to be segmented is downsampled using the embedding module of the target model to obtain a second image at multiple scales. The global feature extraction module and the local feature extraction module in the target model are then used to perform multi-scale encoding processing on the first target image to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and expressed, avoiding the problems of information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the integration of rich multi-scale information and the effective fusion of global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0030] Figure 1This is a hardware structure block diagram of a computer terminal provided according to the first embodiment of the present application;

[0031] Figure 2 is a flowchart of the image segmentation method provided according to the first embodiment of the present application;

[0032] Figure 3 is a schematic diagram of an embedded module provided according to Example 1 of the present application;

[0033] Figure 4 is a schematic diagram of a global feature extraction module provided according to Example 1 of the present application;

[0034] Figure 5 is a schematic diagram of a local feature extraction module provided according to the first embodiment of the present application;

[0035] Figure 6 is a schematic diagram of feature fusion provided according to Example 1 of the present application;

[0036] Figure 7 This is a schematic diagram of the target model provided in Example 1 of this application. Figure 1 ;

[0037] Figure 8 This is a schematic diagram of the target model provided in Example 1 of this application. Figure 2 ;

[0038] Figure 9 is a schematic diagram of an image segmentation method provided according to Example 1 of the present application;

[0039] Figure 10 is a flowchart of the image segmentation method provided in Example 2 of the present application;

[0040] Figure 11 is a flowchart of the image segmentation method provided in Example 3 of the present application;

[0041] Figure 12 is a schematic diagram of an image segmentation device provided according to the fourth embodiment of the present application;

[0042] Figure 13 This is a structural block diagram of a computing device provided according to Example 5 of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0046] Example 1

[0047] According to an embodiment of the present application, an image segmentation method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image segmentation method. Figure 1 As shown, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA, and the processor set 102 may include a processor set, Figure 1102a, 102b, ..., 102n are used to illustrate), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0049] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image segmentation method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned image segmentation method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0051] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0052] The display may be a touch screen liquid crystal display that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0053] Under the above operating environment, this application provides Figure 2 The image segmentation method shown. Figure 2 : is a flowchart of an image segmentation method according to embodiment 1 of the present application. The image segmentation method includes:

[0054] Step S201 : down-sampling the first image to be segmented by using the embedding module of the target model to obtain second images of multiple scales.

[0055] Optionally, the target model is composed of an embedding module, a global feature extraction module, and a local feature extraction module. The embedding module may be an Order Preserving Embedding (OPE) module, the global feature extraction module may be a Global Spatial Block (GSB) module, and the local feature extraction module may be a Local Spatial Block (LSB) module.

[0056] A first image to be segmented is identified. It should be noted that the first image can be a medical image or a remote sensing image. The first image to be segmented is input into the target model. The target model's embedding module performs downsampling on the first image to be segmented. The embedding module OPE converts the first image into a low-dimensional embedding vector, i.e., the aforementioned second images at multiple scales.

[0057] For example, the embedding module OPE performs three-level downsampling processing on the first image. The first image is H*W*C1, where H (Height) represents the height, that is, the number of pixels in the vertical direction of the image or feature map. W (Width) represents the width, that is, the number of pixels in the horizontal direction of the image or feature map. C (Channels) represents the number of channels. For color images, there are usually 3 channels (red, green, and blue), while for grayscale images, the number of channels is 1. H*W*C1 is downsampled by the embedding module OPE to obtain H / 2*W / 2*C2, and H / 2*W / 2*C2 is downsampled by the embedding module OPE to obtain H / 4*W / 4*C3. H / 4*W / 4*C3 is downsampled by the embedding module OPE to obtain H / 8*W / 8*C4. The above-mentioned H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4 are the second images of the above-mentioned multiple scales.

[0058] In an optional embodiment, the embedding module OPE can also perform three or more levels of downsampling processing on the first image. For example, the embedding module OPE can obtain second images of multiple scales of H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4 and H / 16*W / 16*C5.

[0059] In an optional embodiment, the embedding module OPE may be composed of a convolutional layer, a batch normalization layer, a nonlinear activation function and downsampling, such as Figure 3 As shown in FIG, a schematic diagram of the embedding module OPE is shown. The embedding module OPE processes the first image through 3*3Conv, s1 (a 3x3 convolution kernel (or filter) performs a convolution operation with a stride of 1), BN (Batch Normalization), GELU (Gaussian Error Linear Unit, nonlinear activation function) and Down (the above-mentioned downsampling) to obtain the above-mentioned second images of multiple scales.

[0060] In step S202 , a global feature extraction module and a local feature extraction module of a target model are used to perform multi-scale encoding processing on the first target image to obtain target feature images of multiple scales, wherein the first target image is obtained from the second image and the first image of multiple scales.

[0061] Optionally, after obtaining the second images at multiple scales, the first target image is subjected to multi-scale encoding processing using the global feature extraction module and the local feature extraction module to obtain target feature images at multiple scales. It should be noted that the first target image is determined from the second images at multiple scales and the first image. For example, the first target image may be the first image, a second image at any scale among the second images at multiple scales, or a combination of the second images at multiple scales and the first image.

[0062] The global features of the first target image are extracted by the global feature extraction module. In an optional embodiment, the structural diagram of the global feature extraction module is as follows: Figure 4 As shown in the figure, the global feature extraction module can be composed of layer normalization (LN), token mixer (Token-Mixer) and multilayer perceptron (MLP). Token-Mixer, in image processing, an image can be divided into multiple fixed-size regions, each of which is regarded as a token. The role of Token-Mixer is to mix and interact information between these tokens, usually through the self-attention mechanism to enhance the model's ability to understand the global structure of the image. Figure 4 As shown in the figure, the global feature extraction module consists of 2 LNs, 1 Token-Mixer and 1 MLP. Through the global feature extraction module, the features of the entire image or a large area, such as the overall layout of the image, background information, and the positional relationship of objects, can be accurately extracted.

[0063] The local features of the first target image are extracted from the first target image by the local feature extraction module. In an optional embodiment, the structural diagram of the local feature extraction module is as follows: Figure 5 As shown, the local feature extraction module consists of a convolutional layer (e.g., 3*3Conv, s1), a batch normalization layer, a nonlinear activation function, and downsampling / downsampling, as shown in Figure 5 As shown in the figure, the local feature extraction module consists of two convolutional layers (e.g., 3*3Conv, s1), two batch normalization layers, two nonlinear activation functions, and down / up (i.e., the above-mentioned downsampling / downsampling). The local feature extraction module can accurately extract local details of the image, such as the specific texture, edges, and shape of the object.

[0064] It should be noted that the multi-scale encoding operations performed by the global and local feature extraction modules on the first target image include downsampling and upsampling operations, which adjust the resolution of the image or feature map. Downsampling helps the model focus on more abstract features by reducing the feature map dimension, while upsampling helps the target model recover details in segmentation or reconstruction tasks by increasing the feature map dimension. Sampling is the foundation for building multi-scale representations, enabling the target model to simultaneously process both macroscopic and microscopic information in the image.

[0065] Step S203 : performing multi-scale decoding processing on the target feature images of multiple scales by using the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0066] Optionally, after obtaining the target feature images at multiple scales, the target feature images at multiple scales can be further subjected to multi-scale decoding processing using the global feature extraction module and the local feature extraction module to generate a classification label for the pixels in the first image, thereby obtaining an image segmentation result. For example, upsampling can be performed using the global feature extraction module and the local feature extraction module to restore the feature map to the same resolution as the first image, thereby achieving accurate segmentation of the first image.

[0067] In summary, the first image to be segmented is downsampled through the embedding module of the target model to obtain a second image of multiple scales. Then, the global feature extraction module and the local feature extraction module in the target model are used to perform multi-scale encoding processing on the first target image to obtain target feature images of multiple scales. The global feature extraction module can capture the macro information of the image, and the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and expressed, avoiding the problems of information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the fusion of rich multi-scale information and the effective fusion of global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0068] In order to improve the accuracy of feature extraction of the first target image, in the image segmentation method provided in Example 1 of the present application, the first target image is multi-scale encoded by the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, including: up-sampling the first target image by the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales; down-sampling the second target image by the global feature extraction module and the local feature extraction module to obtain target feature images of multiple scales, wherein the second target image is obtained from the second image of multiple scales and the third image of multiple scales.

[0069] Optionally, the first target image is first upsampled by a global feature extraction module and a local feature extraction module, that is, the global features and local features of the first target image are extracted by upsampling the first target image to obtain a third image of multiple scales. The third image has a higher resolution, which is convenient for the model to recover in subsequent steps and use the local details of the image for more detailed analysis. It should be noted that the first target image can be a second image and a first image of multiple scales. For example, two paths are branched out. On one path, the second image and the first image of multiple scales enter the GSB processing to obtain a global feature map of the upsampled processing, and on the other path, the second image and the first image of multiple scales enter the LSB processing to obtain a local feature map of the upsampled processing. By fusing the global feature map and the local feature map, the above-mentioned third image of multiple scales is obtained.

[0070] Then, the second target image is downsampled through the global feature extraction module and the local feature extraction module. The second target image can be a second image of multiple scales and a third image of multiple scales. For example, two paths are branched out. In one path, the second target image enters the GSB processing to obtain a downsampled global feature map, and in the other path, the second target image enters the LSB processing to obtain a downsampled local feature map. By fusing the downsampled global feature map and the local feature map, the above-mentioned target feature images of multiple scales are obtained.

[0071] For example, the embedding module OPE is used to obtain the second image of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4. Then, the global feature extraction module and the local feature extraction module are used to upsample the first image of H*W*C1 and the second image of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4 to obtain the third image of H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1. Then, the global feature extraction module and the local feature extraction module are used to downsample the second image of H / 8*W / 8*C4 and the third image of H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1 to obtain the target feature images of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4.

[0072] The upsampling and downsampling operations further enhance the fusion and utilization of multi-scale information by the target model, so that the target model can understand both macroscopic structures and microscopic details when processing images, thereby improving the accuracy of image segmentation.

[0073] In order to improve the accuracy of the third image, in the image segmentation method provided in Example 1 of the present application, the first target image is upsampled by the global feature extraction module and the local feature extraction module of the target model to obtain third images of multiple scales, including: performing global feature extraction on the first image and the fourth image by the global feature extraction module to obtain first initial feature images of multiple scales, wherein the fourth image is a partial image of the second image of multiple scales; performing local feature extraction on the second image of the first scale in the second images of multiple scales by the local feature extraction module to obtain a second initial feature image of the second scale, wherein the first scale is smaller than the second scale; and obtaining third images of multiple scales based on the second initial feature image of the second scale and the first initial feature images of multiple scales.

[0074] Optionally, the global feature extraction module and the local feature extraction module perform upsampling processing on the first target image, including the following steps: first, the global feature extraction module performs global feature extraction on the first image and the fourth image to obtain first initial feature images of multiple scales. It should be noted that the fourth image is a partial image of the second image of multiple scales. For example, the fourth image is the second image of H / 2*W / 2*C2, H / 4*W / 4*C3. For example, the global feature extraction module performs global feature extraction on the first image of H*W*C1, the second image of H / 2*W / 2*C2, H / 4*W / 4*C3 to obtain first initial feature images of multiple scales, for example, the first initial feature images of H*W*C1, H / 2*W / 2*C2, H / 4*W / 4*C3 are obtained.

[0075] It should be noted that if OPE performs four-level downsampling processing, then the fourth image is the second image of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, and the first initial feature images of multiple scales are the first initial feature images of H*W*C1, H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4.

[0076] While the global feature extraction module is performing global feature extraction, the local feature extraction module is performing local feature extraction on the second image of the first scale among the second images of multiple scales to obtain a second initial feature image of the second scale. For example, the local feature extraction module is performing local feature extraction on the second image of H / 8*W / 8*C4 to obtain a second initial feature image of H / 4*W / 4*C3.

[0077] It should be noted that if OPE performs four-level downsampling processing, the local feature extraction module performs local feature extraction on the second image of H / 16*W / 16*C5 to obtain the second initial feature image of H / 8*W / 8*C3.

[0078] Finally, third images of multiple scales are obtained based on the second initial feature image of the second scale and the first initial feature images of multiple scales. For example, the target model will perform multi-scale feature fusion, and the second initial feature image of the second scale is respectively fused with the first initial feature images of multiple scales to obtain third images of multiple scales.

[0079] For example, before fusing features of different scales, feature alignment is required. After alignment, the aligned feature maps can be directly added element-by-element to obtain a third image of multiple scales, or they can be multiplied element-by-element to obtain a third image of multiple scales.

[0080] Through the global feature extraction module and the local feature extraction module, combined with multi-scale feature fusion, it is possible to fully capture the macro structure and micro details of the image and improve the accuracy of image segmentation.

[0081] In order to further improve the accuracy of the features contained in the third image, in the image segmentation method provided in the first embodiment of the present application, based on the second initial feature image of the second scale and the first initial feature images of multiple scales, obtaining the third images of multiple scales includes: performing feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale in the first initial feature images of multiple scales through a feature fusion module in the target model to obtain a third image of the second scale; performing local feature extraction on the third image of the second scale through a local feature extraction module to obtain a second initial feature image of the third scale; performing feature fusion on the second initial feature image of the third scale and the second initial feature image of the third scale in the first initial feature images of multiple scales through a feature fusion module to obtain a third image of the third scale, wherein the second scale is smaller than the third scale; repeatedly performing the step of performing local feature extraction on the third image of the third scale through the local feature extraction module until all images in the first initial feature images of multiple scales have been processed, and the currently obtained third image is determined as the third image of multiple scales.

[0082] Optionally, obtaining a third image at multiple scales based on the second initial feature image at the second scale and the first initial feature images at multiple scales includes the following steps: First, a feature fusion module performs feature fusion processing on the second initial feature image at the second scale and the first initial feature image at the same scale (the second scale) to obtain a third image at the second scale. For example, if the second scale is H / 4*W / 4*C3, the feature fusion module performs feature fusion processing on the second initial feature image at H / 4*W / 4*C3 and the first initial feature image at H / 4*W / 4*C3 to obtain a third image at H / 4*W / 4*C3.

[0083] Then, the local feature extraction module will again perform local feature extraction on the third image of the second scale to obtain a second initial feature image of the third scale. For example, the local feature extraction module performs local feature extraction on the third image of H / 4*W / 4*C3 to obtain a second initial feature image of H / 2*W / 2*C2.

[0084] After obtaining the second initial feature image at the third scale, the feature fusion module performs feature fusion processing on the second initial feature image at the third scale and the second initial feature image at the third scale among the first initial feature images at multiple scales to obtain a third image at the third scale. For example, if the third scale is H / 2*W / 2*C2, the feature fusion module performs feature fusion on the second initial feature image at H / 2*W / 2*C2 and the first initial feature image at H / 2*W / 2*C2 to obtain a third image at H / 2*W / 2*C2.

[0085] The above-mentioned local feature extraction and feature fusion will be repeatedly executed. In one cycle, the local feature extraction module will refine local features based on the third image of the current scale and generate a second initial feature image of the next scale. Then, it will be fused with the first initial feature image of the corresponding scale through the feature fusion module again until all images in the first initial feature images of multiple scales have been processed and the currently obtained third image is determined as the third image of multiple scales.

[0086] In an optional embodiment, the schematic diagram of the feature fusion module performing feature fusion is as follows: Figure 6 As shown, through the global feature extraction module and the local feature extraction module, the global feature Global (for example, the first initial feature image) and the local feature Local (for example, the second initial feature image) are obtained, and then the global feature Global and the local feature Local are convolved to obtain the fused feature Fusion Feature (for example, the third image).

[0087] Through progressive feature fusion and local feature extraction, the target model can effectively integrate feature information of different scales. Moreover, through the above-mentioned local feature extraction and feature fusion, it can ensure that global and local features can be transmitted continuously without confusion, thereby improving the accuracy of the features contained in the third image.

[0088] In order to improve the accuracy of the target feature image, in the image segmentation method provided in Example 1 of the present application, the second target image is down-sampled by a global feature extraction module and a local feature extraction module to obtain target feature images of multiple scales, including: performing global feature extraction on the fifth image and the second image of the first target scale in the second images of multiple scales by the global feature extraction module to obtain third initial feature images of multiple scales, wherein the fifth image is a partial image in the third images of multiple scales, and the scale of the second image of the first target scale is smaller than the scale of the remaining images of the second images of multiple scales; performing local feature extraction on the third image of the Nth scale in the third images of multiple scales by the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; and obtaining target feature images of multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of multiple scales.

[0089] Optionally, downsampling the second target image using the global feature extraction module and the local feature extraction module includes the following steps: First, the global feature extraction module performs global feature extraction on the fifth image and the second image at the first target scale among the second images at multiple scales to obtain a third initial feature image at multiple scales. It should be noted that the fifth image may be a portion of the third image at multiple scales. For example, if OPE performs four-level downsampling, the fifth image may be a third image at H / 2*W / 2*C2, H / 4*W / 4*C3, or H / 8*W / 8*C4. If OPE performs three-level downsampling, the fifth image may be a third image at H / 2*W / 2*C2 or H / 4*W / 4*C3. The second image at the first target scale is smaller than the remaining scale of the second images at the multiple scales. For example, if OPE performs four-level downsampling, the first target scale is H / 16*W / 16*C5, and if OPE performs three-level downsampling, the first target scale is H / 8*W / 8*C4.

[0090] In an optional embodiment, the global feature extraction module performs global feature extraction on the third image of H / 2*W / 2*C2, H / 4*W / 4*C3 and the second image of H / 8*W / 8*C4 to obtain the third initial feature image of H / 2*W / 2*C2, H / 4*W / 4*C3 and H / 8*W / 8*C4.

[0091] Then, the local feature extraction module performs local feature extraction on the third image at the Nth scale to obtain a fourth initial feature image at the N-1th scale. For example, the scale of the third image at the Nth scale is larger than the scales of the remaining images in the third image. For example, if the third image includes H*W*C1, H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, then the Nth scale is H*W*C1, and the corresponding N-1th scale is H / 2*W / 2*C2.

[0092] Finally, target feature images at multiple scales are obtained based on the fourth initial feature image at the N-1th scale and the third initial feature images at multiple scales. For example, the target model may perform multi-scale feature fusion, fusing the fourth initial feature image at the N-1th scale with the third initial feature images at multiple scales to obtain target feature images at multiple scales.

[0093] For example, before fusing features of different scales, feature alignment is required. After alignment, the aligned feature maps can be directly added element-by-element to obtain target feature images of multiple scales, or they can be multiplied element-by-element to obtain target feature images of multiple scales.

[0094] Through the complementarity of global and local feature extraction, as well as the strategy of multi-scale feature fusion, the accuracy of the target feature image can be effectively improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0095] In order to further improve the accuracy of the target feature image, in the image segmentation method provided in the first embodiment of the present application, based on the fourth initial feature image of the N-1th scale and the third initial feature images of multiple scales, obtaining the target feature images of multiple scales includes: performing feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale in the third initial feature images of multiple scales through a feature fusion module in the target model to obtain the target feature image of the N-1th scale; performing local feature extraction on the target feature image of the N-1th scale through a local feature extraction module to obtain the fourth initial feature image of the N-2th scale; performing feature fusion on the fourth initial feature image of the N-2th scale and the third initial feature image of the N-2th scale in the third initial feature images of multiple scales through a feature fusion module to obtain the target feature image of the N-2th scale; repeatedly performing the step of performing local feature extraction on the target feature image of the N-2th scale through the local feature extraction module until all images in the third initial feature images of multiple scales have been processed, and determining the currently obtained target feature image as the target feature image of multiple scales.

[0096] Optionally, the feature fusion module performs feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale, for example, by performing convolution calculation on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale to achieve feature fusion, and obtain a target feature image of the N-1th scale.

[0097] After obtaining the target feature image at the N-1th scale, the local feature extraction module performs local feature extraction on the target feature image at the N-1th scale to obtain the fourth initial feature image at the N-2th scale. This means that the local feature extraction module gradually downsamples. For example, if the N-1th scale is H / 2*W / 2*C2, then the N-2th scale is H / 4*W / 4*C3.

[0098] The above-mentioned local feature extraction and feature fusion will be repeatedly executed. In one cycle, the local feature extraction module will refine local features based on the target feature image of the current scale and generate the fourth initial feature image of the next scale. Then, it will be fused with the third initial feature image of the corresponding scale through the feature fusion module again until all images in the third initial feature images of multiple scales have been processed and the target feature images covering all scales are generated.

[0099] In an optional embodiment, the second images of multiple scales obtained by downsampling the first image H*W*C1 through OPE are second images of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4. The second images of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4 and the first image of H*W*C1 are gradually processed through GSB and LSB to obtain third images of H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1. The third image of H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1 and the second image of H / 8*W / 8*C4 are gradually processed through GSB and LSB to obtain target feature images of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4.

[0100] In an optional embodiment, the second images of multiple scales obtained by downsampling the first image H*W*C1 through OPE are second images of H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4, and H / 16*W / 16*C5. The second images of H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4, and H / 16*W / 16*C5 and the first image of H*W*C1 are gradually upsampled through GSB and LSB to obtain H The third image of H / 8*W / 8*C4, H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1 and the second image of H / 16*W / 16*C5 are gradually downsampled by GSB and LSB to obtain target feature images of H / 2*W / 2*C2, H / 4*W / 4*C3, H / 8*W / 8*C4, and H / 16*W / 16*C5.

[0101] It should be noted that the second images of multiple scales are not limited to the above-mentioned third-level second images or fourth-level second images. The downsampling level of the first image can be set according to actual needs, for example, five-level downsampling.

[0102] Through recursive feature fusion and local feature refinement, the details and boundary clarity of image segmentation are enhanced, and the robustness and generalization ability of the model in complex image analysis tasks are improved.

[0103] In order to improve the accuracy of image segmentation, in the image segmentation method provided in the first embodiment of the present application, multi-scale decoding processing is performed on target feature images of multiple scales by a global feature extraction module and a local feature extraction module to obtain an image segmentation result corresponding to the first image, including: performing global feature extraction on the sixth image and the seventh image by the global feature extraction module to obtain first feature maps of multiple scales, wherein the sixth image is a partial image in the target feature images of multiple scales, and the seventh image is an image of the second target scale in the third image of multiple scales; performing local feature extraction on the target feature image of the first scale in the target feature images of the multiple scales by the local feature extraction module to obtain a second feature map of the second scale; performing feature fusion on the second feature map of the second scale and the first feature map of the second scale in the first feature maps of the multiple scales to obtain a feature fusion map of the second scale; repeatedly performing the step of performing local feature extraction on the feature fusion map of the second scale by the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map of the Nth scale.

[0104] Optionally, obtaining an image segmentation result corresponding to the first image based on target feature images of multiple scales includes the following steps: determining a sixth image from a portion of the images in the target feature images of multiple scales, for example, if the target feature images of the multiple scales are target feature images of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, then the sixth image is a target feature image of H / 2*W / 2*C2 and H / 4*W / 4*C3. The seventh image is an image of the second target scale in the third image of multiple scales, for example, if the third image of the multiple scales is a third image of H / 4*W / 4*C3, H / 2*W / 2*C2, and H*W*C1, then the seventh image is a third image of H*W*C1. The global feature extraction module is used to perform global feature extraction on the target feature images of H / 2*W / 2*C2, H / 4*W / 4*C3 and the third image of H*W*C1 to obtain the first feature maps of H*W*C1, H / 2*W / 2*C2, and H / 4*W / 4*C3.

[0105] Then, the local feature extraction module processes the target feature image at the first scale to generate a second feature map at the second scale. For example, if the target feature images at multiple scales are H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, then the target feature image at the first scale is H / 8*W / 8*C4, and the second scale is H / 4*W / 4*C3.

[0106] The feature fusion module then fuses the second feature map of the second scale with the first feature map of the same scale to generate a fused feature map of the second scale. For example, the second feature map of H / 4*W / 4*C3 is fused with the first feature map of H / 4*W / 4*C3 to generate a fused feature map of H / 4*W / 4*C3.

[0107] After obtaining the feature fusion map of the second scale, the local feature extraction and feature fusion steps will be repeated until the feature fusion map of the Nth scale is obtained. It should be noted that the feature fusion map of the Nth scale contains image segmentation information, and the feature fusion map of the Nth scale can be directly used as the above-mentioned image segmentation result.

[0108] It should be noted that the Nth scale is the scale corresponding to the first image, that is, when the target model segments the first image, it first downsamples through OPE to obtain second images of multiple scales, and then performs up and down sampling on the first image and the second images of multiple scales through GSB and LSB to realize the encoding process, and then downsamples the encoded target feature image through GSB and LSB to realize the decoding process and obtain the final image segmentation result.

[0109] The input image is downsampled through OPE, and multi-scale encoding and decoding layers are constructed using LSB. At the same time, GSB is introduced between the encoding and decoding layers to ensure that global and local features can be transmitted continuously without confusion, thereby achieving the technical effect of generating higher quality image segmentation results.

[0110] In order to improve the accuracy of image segmentation, in the image segmentation method provided in Example 1 of the present application, the target model is composed of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0111] Optionally, the target model consists of an embedding module OPE, a local feature extraction module LSB, a global feature extraction module GSB, and a feature fusion module. GSB is connected between OPE and LSB, and GSB is also connected between LSB. That is, LSB constructs multi-scale encoding and decoding layers, and GSB is introduced between the encoding and decoding layers, such as Figure 7 As shown in Figure 2, through the above connection relationship, the network structure of the target model presents a W shape.

[0112] In an optional embodiment, the target model consists of 3 OPEs, 9 LSBs, 9 GSBs and 9 Fus (feature fusion modules), such as Figure 7 As shown in Figure 3, the network structure of the target model is W-shaped.

[0113] In an optional embodiment, the target model may also be composed of 4 OPEs, 12 LSBs, 12 GSBs and 12 Fuss, such as Figure 8 shown.

[0114] In an optional embodiment, when the target model consists of 3 OPEs, 9 LSBs and 9 GSBs, the image segmentation diagram is as follows: Figure 9 As shown, it specifically includes: downsampling the first image of H*W*C1 through 3 OPEs to obtain second images of H / 2*W / 2*C2, H / 4*W / 4*C3, and H / 8*W / 8*C4, processing the second image of H / 4*W / 4*C3 through GSB, and processing the second image of H / 8*W / 8*C4 through LSB to obtain the third image of H / 4*W / 4*C3; processing the second image of H / 2*W / 2*C2 through GSB, and processing the third image of H / 4*W / 4*C3 through LSB to obtain the third image of H / 2*W / 2*C2; processing the first image of H*W*C1 through GSB, and processing the third image of H / 2*W / 2*C2 through LSB to obtain the third image of H*W*C1.

[0115] Then, the third image of H / 2*W / 2*C2 is processed by GSB, and the third image of H*W*C1 is processed by LSB to obtain the target feature image of H / 2*W / 2*C2; the third image of H / 4*W / 4*C3 is processed by GSB, and the target feature image of H / 2*W / 2*C2 is processed by LSB to obtain the target feature image of H / 4*W / 4*C3; the second image of H / 8*W / 8*C4 is processed by GSB, and the target feature image of H / 4*W / 4*C3 is processed by LSB to obtain the target feature image of H / 8*W / 8*C4.

[0116] Then, the target feature image of H / 4*W / 4*C3 is processed by GSB, and the target feature image of H / 8*W / 8*C4 is processed by LSB to obtain the feature fusion map of H / 4*W / 4*C3; the target feature image of H / 2*W / 2*C2 is processed by GSB, and the feature fusion map of H / 4*W / 4*C3 is processed by LSB to obtain the feature fusion map of H / 2*W / 2*C2; the third image of H*W*C1 is processed by GSB, and the feature fusion map of H / 2*W / 2*C2 is processed by LSB to obtain the feature fusion map of H*W*C1, that is, the final image segmentation result.

[0117] By downsampling the original image at different levels using three OPEs, a multi-scale image feature representation is constructed. The combined use of GSB and LSB modules ensures information transfer and fusion between features at different scales. GSB processes global features and maintains the integrity of macroscopic structures, while LSB focuses on extracting local features. The interaction between the two not only enhances feature representation but also avoids information loss during scale conversion, thereby achieving the technical effect of improving image segmentation accuracy.

[0118] In an optional embodiment, the target model was validated using multiple datasets (e.g., retinal vessel segmentation datasets, skin lesion segmentation datasets, etc.), and the Jaccard index (Jaccard) was used as a performance metric to evaluate the segmentation results. The target model achieved a significance level below 0.05 in 28 of 32 t-tests, achieving a significance rate of 87.50%. This demonstrates that the target model has statistically significant advantages and improvements.

[0119] In the image segmentation method provided in Example 1 of the present application, the first image to be segmented is down-sampled by the embedding module of the target model to obtain second images of multiple scales; the first target image is multi-scale encoded by the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images and the first image of multiple scales; the target feature images of multiple scales are multi-scale decoded by the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image, thereby solving the technical problem in the related art that the image is segmented according to the local features of the image, resulting in relatively low accuracy of image segmentation.

[0120] In this solution, the first image to be segmented is downsampled using the embedding module of the target model to obtain a second image at multiple scales. The global feature extraction module and the local feature extraction module in the target model are then used to perform multi-scale encoding processing on the first target image to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and expressed, avoiding the problems of information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the integration of rich multi-scale information and the effective fusion of global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0121] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0123] Example 2

[0124] According to an embodiment of the present application, an image segmentation method is also provided, such as Figure 10 As shown, the image segmentation method includes:

[0125] Step S1001: downsampling a first medical image using an embedding module of a target model to obtain second medical image images at multiple scales;

[0126] Step S1002: performing multi-scale encoding processing on a first target medical image using a global feature extraction module and a local feature extraction module of a target model to obtain target feature images at multiple scales, wherein the first target medical image is obtained from a second medical image at multiple scales and the first medical image;

[0127] Step S1003 : performing multi-scale decoding processing on target feature images of multiple scales by using a global feature extraction module and a local feature extraction module to obtain an image segmentation result corresponding to the first medical image.

[0128] It should be noted that the specific method for segmenting the medical image is the same as that in the first embodiment and will not be described in detail here.

[0129] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0131] Example 3

[0132] According to an embodiment of the present application, an image segmentation method is also provided, such as Figure 11 As shown, the image segmentation method includes:

[0133] Step S1101, obtaining a first image to be segmented uploaded by a client;

[0134] Step S1102: Downsampling the first image to be segmented in the cloud server using the embedding module of the target model to obtain second images at multiple scales; multi-scale encoding is performed on the first target image using the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second images at multiple scales and the first image; multi-scale decoding is performed on the target feature images at multiple scales using the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image;

[0135] Step S1103: Return the image segmentation result to the client.

[0136] It should be noted that the specific method of segmenting the image in the cloud server is the same as that in the first embodiment and will not be described in detail here.

[0137] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0139] Example 4

[0140] According to an embodiment of the present application, an image segmentation device for implementing the above image segmentation method is also provided. Figure 12 As shown, the device includes: a first processing unit 1201, a second processing unit 1202 and a determining unit 1203.

[0141] A first processing unit 1201 is configured to perform downsampling processing on the first image to be segmented by using an embedding module of a target model to obtain second images of multiple scales;

[0142] a second processing unit 1202 configured to perform multi-scale encoding processing on the first target image using a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image and the first image at multiple scales;

[0143] The determining unit 1203 is configured to perform multi-scale decoding processing on the target feature images of multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0144] In the image segmentation device provided in Example 4 of the present application, the first processing unit 1201 downsamples the first image to be segmented through the embedding module of the target model to obtain second images of multiple scales; the second processing unit 1202 performs multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images and the first image of multiple scales; the determination unit 1203 performs multi-scale decoding processing on the target feature images of multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image, which solves the technical problem in the related art that the image is segmented according to the local features of the image, resulting in relatively low accuracy of image segmentation.

[0145] In this solution, the first image to be segmented is downsampled using the embedding module of the target model to obtain a second image at multiple scales. The global feature extraction module and the local feature extraction module in the target model are then used to perform multi-scale encoding processing on the first target image to obtain target feature images at multiple scales. The global feature extraction module can capture the macroscopic information of the image, while the local feature extraction module can capture the detailed features of the image. In this way, both global and local features can be fully extracted and expressed, avoiding the problems of information loss and blurred segmentation boundaries that may be caused by single-scale features. Due to the integration of rich multi-scale information and the effective fusion of global and local features, the overall performance of image segmentation is improved, thereby achieving the technical effect of improving the accuracy of image segmentation.

[0146] Optionally, in the image segmentation device provided in Example 4 of the present application, the second processing unit includes: a first processing subunit, used to upsample the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales; a second processing subunit, used to downsample the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of multiple scales, wherein the second target image is obtained from the second image of multiple scales and the third image of multiple scales.

[0147] Optionally, in the image segmentation device provided in Example 4 of the present application, the first processing subunit includes: a first extraction module, used to perform global feature extraction on the first image and the fourth image through a global feature extraction module to obtain first initial feature images of multiple scales, wherein the fourth image is a partial image of the second image of multiple scales; a second extraction module, used to perform local feature extraction on the second image of the first scale among the second images of multiple scales through a local feature extraction module to obtain a second initial feature image of the second scale, wherein the first scale is smaller than the second scale; and a first determination module, used to obtain third images of multiple scales based on the second initial feature image of the second scale and the first initial feature images of multiple scales.

[0148] Optionally, in the image segmentation device provided in Example 4 of the present application, the first determination module includes: a first fusion submodule, used to perform feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale in the first initial feature images of multiple scales through the feature fusion module in the target model to obtain a third image of the second scale; a first extraction submodule, used to perform local feature extraction on the third image of the second scale through the local feature extraction module to obtain a second initial feature image of the third scale; a second fusion submodule, used to perform feature fusion on the second initial feature image of the third scale and the second initial feature image of the third scale in the first initial feature images of multiple scales through the feature fusion module to obtain a third image of the third scale, wherein the second scale is smaller than the third scale; the first determination submodule, used to repeatedly perform the step of performing local feature extraction on the third image of the third scale through the local feature extraction module until all images in the first initial feature images of multiple scales have been processed, and the currently obtained third image is determined as the third image of multiple scales.

[0149] Optionally, in the image segmentation device provided in Example 4 of the present application, the second processing subunit includes: a third extraction module, used to perform global feature extraction on the fifth image and the second image of the first target scale in the second images of multiple scales through the global feature extraction module to obtain third initial feature images of multiple scales, wherein the fifth image is a partial image in the third images of multiple scales, and the scale of the second image of the first target scale is smaller than the scale of the remaining images of the second images of multiple scales; a fourth extraction module, used to perform local feature extraction on the third image of the Nth scale in the third images of multiple scales through the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; and a second determination module, used to obtain target feature images of multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of multiple scales.

[0150] Optionally, in the image segmentation device provided in Example 4 of the present application, the second determination module includes: a third fusion submodule, used to perform feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale in the third initial feature images of multiple scales through the feature fusion module in the target model to obtain a target feature image of the N-1th scale; a second extraction submodule, used to perform local feature extraction on the target feature image of the N-1th scale through the local feature extraction module to obtain a fourth initial feature image of the N-2th scale; a fourth fusion submodule, used to perform feature fusion on the fourth initial feature image of the N-2th scale and the third initial feature image of the N-2th scale in the third initial feature images of multiple scales through the feature fusion module to obtain a target feature image of the N-2th scale; and a second determination submodule, used to repeatedly perform the step of performing local feature extraction on the target feature image of the N-2th scale through the local feature extraction module until all images in the third initial feature images of multiple scales have been processed, and the currently obtained target feature image is determined as the target feature image of multiple scales.

[0151] Optionally, in the image segmentation device provided in Example 4 of the present application, the determination unit includes: a first extraction subunit, used to perform global feature extraction on the sixth image and the seventh image through a global feature extraction module to obtain first feature maps of multiple scales, wherein the sixth image is a partial image in the target feature images of multiple scales, and the seventh image is an image of the second target scale in the third image of multiple scales; a second extraction subunit, used to perform local feature extraction on the target feature image of the first scale in the target feature images of multiple scales through a local feature extraction module to obtain a second feature map of the second scale; a fusion subunit, used to perform feature fusion on the second feature map of the second scale and the first feature map of the second scale in the first feature maps of multiple scales to obtain a feature fusion map of the second scale; a determination subunit, used to repeatedly perform the step of performing local feature extraction on the feature fusion map of the second scale through the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtain an image segmentation result based on the feature fusion map of the Nth scale.

[0152] Optionally, in the image segmentation device provided in Example 4 of the present application, the target model is composed of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0153] It should be noted that the first processing unit 1201, the second processing unit 1202, and the determination unit 1203 described above correspond to steps S201 to S203 in the first embodiment. The examples and application scenarios implemented by the three units and the corresponding steps are the same, but are not limited to the contents disclosed in the first embodiment. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in the first embodiment.

[0154] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0155] Example 5

[0156] The embodiment of the present application may provide an electronic device, which may be any electronic device in a group of electronic device terminals. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0157] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0158] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: down-sampling the first image to be segmented through the embedding module of the target model to obtain second images of multiple scales; multi-scale encoding processing is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images and the first image of multiple scales; multi-scale decoding processing is performed on the target feature images of multiple scales through the global feature extraction module and the local feature extraction module to obtain the image segmentation result corresponding to the first image.

[0159] The above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: multi-scale encoding processing is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, including: up-sampling processing is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales; down-sampling processing is performed on the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of multiple scales, wherein the second target image is obtained from the second image of multiple scales and the third image of multiple scales.

[0160] The above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: upsampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales, including: performing global feature extraction on the first image and the fourth image through the global feature extraction module to obtain a first initial feature image of multiple scales, wherein the fourth image is a partial image of the second image of multiple scales; performing local feature extraction on the second image of the first scale in the second images of multiple scales through the local feature extraction module to obtain a second initial feature image of the second scale, wherein the first scale is smaller than the second scale; obtaining a third image of multiple scales based on the second initial feature image of the second scale and the first initial feature images of multiple scales.

[0161] The above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: based on the second initial feature image of the second scale and the first initial feature images of multiple scales, obtaining the third image of multiple scales includes: performing feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale in the first initial feature images of multiple scales through the feature fusion module in the target model to obtain the third image of the second scale; performing local feature extraction on the third image of the second scale through the local feature extraction module to obtain the second initial feature image of the third scale; performing feature fusion on the second initial feature image of the third scale and the second initial feature image of the third scale in the first initial feature images of multiple scales through the feature fusion module to obtain the third image of the third scale, wherein the second scale is smaller than the third scale; repeatedly performing the step of performing local feature extraction on the third image of the third scale through the local feature extraction module until all images in the first initial feature images of multiple scales have been processed, and the currently obtained third image is determined as the third image of multiple scales.

[0162] The above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: downsampling the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of multiple scales, including: performing global feature extraction on the fifth image and the second image of the first target scale in the second images of multiple scales through the global feature extraction module to obtain third initial feature images of multiple scales, wherein the fifth image is a partial image in the third images of multiple scales, and the scale of the second image of the first target scale is smaller than the scale of the remaining images of the second images of multiple scales; performing local feature extraction on the third image of the Nth scale in the third images of multiple scales through the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; obtaining target feature images of multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of multiple scales.

[0163] The electronic device can execute the program code of the following steps in the image segmentation method: based on the fourth initial feature image of the N-1th scale and the third initial feature images of multiple scales, obtaining the target feature images of multiple scales includes: performing feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale in the third initial feature images of multiple scales through a feature fusion module in the target model to obtain the target feature image of the N-1th scale; performing local feature extraction on the target feature image of the N-1th scale through a local feature extraction module to obtain the fourth initial feature image of the N-2th scale; performing feature fusion on the fourth initial feature image of the N-2th scale and the third initial feature image of the N-2th scale in the third initial feature images of multiple scales through a feature fusion module to obtain the target feature image of the N-2th scale; repeatedly performing the step of performing local feature extraction on the target feature image of the N-2th scale through the local feature extraction module until all images in the third initial feature images of multiple scales have been processed, and the currently obtained target feature image is determined as the target feature image of multiple scales.

[0164] The electronic device can execute the program code of the following steps in the image segmentation method: performing multi-scale decoding processing on target feature images of multiple scales through a global feature extraction module and a local feature extraction module to obtain an image segmentation result corresponding to the first image, including: performing global feature extraction on the sixth image and the seventh image through the global feature extraction module to obtain first feature maps of multiple scales, wherein the sixth image is a partial image in the target feature images of multiple scales, and the seventh image is an image of the second target scale in the third image of multiple scales; performing local feature extraction on the target feature image of the first scale in the target feature images of the multiple scales through the local feature extraction module to obtain a second feature map of the second scale; performing feature fusion on the second feature map of the second scale and the first feature map of the second scale in the first feature maps of the multiple scales to obtain a feature fusion map of the second scale; repeatedly performing the step of performing local feature extraction on the feature fusion map of the second scale through the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map of the Nth scale.

[0165] The above-mentioned electronic device can execute the program code of the following steps in the image segmentation method: the target model is composed of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0166] Optionally, Figure 13 This is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 13 As shown, the electronic device 130 may include: one or more ( Figure 13 (only one is shown) processor 1302, memory 1304. The electronic device 130 may further include a memory controller to control and manage the memory 1304; the electronic device 130 may further include a peripheral interface to connect to a radio frequency module, an audio module, and a display screen, etc.

[0167] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image segmentation method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image segmentation method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the electronic device 130 via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0168] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: down-sampling the first image to be segmented through the embedding module of the target model to obtain second images of multiple scales; multi-scale encoding processing is performed on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, wherein the first target image is obtained from the second images of multiple scales and the first image; multi-scale decoding processing is performed on the target feature images of multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

[0169] Optionally, the above-mentioned processor can also execute the program code of the following steps: performing multi-scale encoding processing on the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales, including: up-sampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales; down-sampling the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of multiple scales, wherein the second target image is obtained from the second image of multiple scales and the third image of multiple scales.

[0170] Optionally, the processor may also execute the program code of the following steps: upsampling the first target image through the global feature extraction module and the local feature extraction module of the target model to obtain a third image of multiple scales, including: performing global feature extraction on the first image and the fourth image through the global feature extraction module to obtain a first initial feature image of multiple scales, wherein the fourth image is a partial image of the second image of multiple scales; performing local feature extraction on the second image of the first scale in the second images of multiple scales through the local feature extraction module to obtain a second initial feature image of the second scale, wherein the first scale is smaller than the second scale; and obtaining a third image of multiple scales based on the second initial feature image of the second scale and the first initial feature images of multiple scales.

[0171] Optionally, the processor may also execute the program code of the following steps: obtaining a third image of multiple scales based on the second initial feature image of the second scale and the first initial feature images of multiple scales includes: performing feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale in the first initial feature images of multiple scales through a feature fusion module in the target model to obtain a third image of the second scale; performing local feature extraction on the third image of the second scale through a local feature extraction module to obtain a second initial feature image of the third scale; performing feature fusion on the second initial feature image of the third scale and the second initial feature image of the third scale in the first initial feature images of multiple scales through a feature fusion module to obtain a third image of the third scale, wherein the second scale is smaller than the third scale; repeatedly performing the step of performing local feature extraction on the third image of the third scale through the local feature extraction module until all images in the first initial feature images of multiple scales have been processed, and the currently obtained third image is determined as the third image of multiple scales.

[0172] Optionally, the processor may also execute the program code of the following steps: downsampling the second target image through the global feature extraction module and the local feature extraction module to obtain target feature images of multiple scales, including: performing global feature extraction on the fifth image and the second image of the first target scale in the second images of multiple scales through the global feature extraction module to obtain third initial feature images of multiple scales, wherein the fifth image is a partial image in the third images of multiple scales, and the scale of the second image of the first target scale is smaller than the scale of the remaining images of the second images of multiple scales; performing local feature extraction on the third image of the Nth scale in the third images of multiple scales through the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; and obtaining target feature images of multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of multiple scales.

[0173] Optionally, the processor may also execute the program code of the following steps: obtaining target feature images of multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature images of multiple scales includes: performing feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale in the third initial feature images of multiple scales through a feature fusion module in the target model to obtain the target feature image of the N-1th scale; performing local feature extraction on the target feature image of the N-1th scale through a local feature extraction module to obtain the fourth initial feature image of the N-2th scale; performing feature fusion on the fourth initial feature image of the N-2th scale and the third initial feature image of the N-2th scale in the third initial feature images of multiple scales through a feature fusion module to obtain the target feature image of the N-2th scale; repeatedly performing the step of performing local feature extraction on the target feature image of the N-2th scale through the local feature extraction module until all images in the third initial feature images of multiple scales have been processed, and the currently obtained target feature image is determined as the target feature image of multiple scales.

[0174] Optionally, the processor may also execute the program code of the following steps: performing multi-scale decoding processing on target feature images of multiple scales through a global feature extraction module and a local feature extraction module to obtain an image segmentation result corresponding to the first image, including: performing global feature extraction on the sixth image and the seventh image through the global feature extraction module to obtain first feature maps of multiple scales, wherein the sixth image is a partial image in the target feature images of multiple scales, and the seventh image is an image of the second target scale in the third image of multiple scales; performing local feature extraction on the target feature image of the first scale in the target feature images of multiple scales through the local feature extraction module to obtain a second feature map of the second scale; performing feature fusion on the second feature map of the second scale and the first feature map of the second scale in the first feature maps of multiple scales to obtain a feature fusion map of the second scale; repeatedly performing the step of performing local feature extraction on the feature fusion map of the second scale through the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map of the Nth scale.

[0175] Optionally, the above-mentioned processor can also execute the program code of the following steps: the target model is composed of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

[0176] It can be understood by those skilled in the art that Figure 13The structure shown is for illustration only, and the electronic device 130 may also be a terminal device such as a smart phone, a tablet computer, a PDA, a mobile Internet device (MID), or a PAD. Figure 13 It does not limit the structure of the above electronic device. For example, the electronic device 130 may also include Figure 13 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 13 Different configurations shown.

[0177] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0178] Example 6

[0179] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product can be used to store the program code executed by the image segmentation method provided in the first embodiment.

[0180] Optionally, in this embodiment, the computer program product may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0181] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0182] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0184] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0186] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0187] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image segmentation method, characterized in that: include: Downsampling the first image to be segmented by using an embedding module of the target model to obtain second images at multiple scales; Performing multi-scale encoding processing on the first target image by using the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at the multiple scales and the first image; Multi-scale decoding processing is performed on the target feature images of the multiple scales by the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

2. The method according to claim 1, characterized in that Performing multi-scale encoding processing on the first target image by the global feature extraction module and the local feature extraction module of the target model to obtain target feature images of multiple scales includes: Upsampling the first target image using the global feature extraction module and the local feature extraction module of the target model to obtain third images of multiple scales; The second target image is downsampled by the global feature extraction module and the local feature extraction module to obtain target feature images of the multiple scales, wherein the second target image is obtained from the second images of the multiple scales and the third images of the multiple scales.

3. The method according to claim 2, characterized in that Upsampling the first target image by the global feature extraction module and the local feature extraction module of the target model to obtain third images of multiple scales includes: Performing global feature extraction on the first image and the fourth image by the global feature extraction module to obtain first initial feature images at multiple scales, wherein the fourth image is a partial image of the second image at the multiple scales; Performing local feature extraction on a second image at a first scale among the second images at multiple scales by the local feature extraction module to obtain a second initial feature image at a second scale, wherein the first scale is smaller than the second scale; Based on the second initial feature image at the second scale and the first initial feature images at the multiple scales, third images at the multiple scales are obtained.

4. The method according to claim 3, characterized in that Obtaining third images at the multiple scales based on the second initial feature image at the second scale and the first initial feature images at the multiple scales includes: performing feature fusion on the second initial feature image of the second scale and the first initial feature image of the second scale among the first initial feature images of the multiple scales through a feature fusion module in the target model to obtain a third image of the second scale; Performing local feature extraction on the third image at the second scale by the local feature extraction module to obtain a second initial feature image at the third scale; performing feature fusion on the second initial feature image at the third scale and the second initial feature image at the third scale among the first initial feature images at the multiple scales by the feature fusion module to obtain a third image at the third scale, wherein the second scale is smaller than the third scale; Repeating the step of performing local feature extraction on the third image at the third scale by the local feature extraction module until all images in the first initial feature images at the multiple scales have been processed, and determining the currently obtained third image as the third image at the multiple scales.

5. The method according to claim 2, characterized in that Downsampling the second target image by the global feature extraction module and the local feature extraction module to obtain the target feature images of the multiple scales includes: performing global feature extraction on the fifth image and the second image at a first target scale among the second images at the multiple scales by the global feature extraction module to obtain third initial feature images at multiple scales, wherein the fifth image is a partial image among the third images at the multiple scales, and the scale of the second image at the first target scale is smaller than the scale of the remaining images of the second images at the multiple scales; Performing local feature extraction on the third image of the Nth scale among the third images of the multiple scales by the local feature extraction module to obtain a fourth initial feature image of the N-1th scale; The target feature images of the multiple scales are obtained based on the fourth initial feature image of the N-1th scale and the third initial feature images of the multiple scales.

6. The method according to claim 5, characterized in that Obtaining target feature images of the multiple scales based on the fourth initial feature image of the N-1th scale and the third initial feature image of the multiple scales includes: performing feature fusion on the fourth initial feature image of the N-1th scale and the third initial feature image of the N-1th scale among the third initial feature images of the multiple scales through a feature fusion module in the target model to obtain a target feature image of the N-1th scale; Performing local feature extraction on the target feature image at the N-1th scale by the local feature extraction module to obtain a fourth initial feature image at the N-2th scale; performing feature fusion on the fourth initial feature image at the N-2th scale and the third initial feature image at the N-2th scale among the third initial feature images at the multiple scales by the feature fusion module to obtain a target feature image at the N-2th scale; Repeating the step of performing local feature extraction on the target feature image at the N-2th scale by the local feature extraction module until all images in the third initial feature images at the multiple scales have been processed, and determining the currently obtained target feature image as the target feature image at the multiple scales.

7. The method according to claim 2, characterized in that Performing multi-scale decoding processing on the target feature images of the multiple scales by the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image includes: Performing global feature extraction on the sixth image and the seventh image by the global feature extraction module to obtain first feature maps at multiple scales, wherein the sixth image is a partial image of the target feature images at the multiple scales, and the seventh image is an image of the second target scale in the third image at the multiple scales; Performing local feature extraction on a target feature image of a first scale among the target feature images of the multiple scales by the local feature extraction module to obtain a second feature map of a second scale; Performing feature fusion on the second feature map of the second scale and the first feature map of the second scale among the first feature maps of the multiple scales to obtain a feature fusion map of the second scale; Repeating the step of performing local feature extraction on the feature fusion map of the second scale by the local feature extraction module until a feature fusion map of the Nth scale is obtained, and obtaining the image segmentation result based on the feature fusion map of the Nth scale.

8. The method according to claim 1, characterized in that The target model consists of an embedding module, a local feature extraction module, a global feature extraction module and a feature fusion module, wherein the global feature extraction module is connected between the embedding module and the local feature extraction module, and the global feature extraction module is connected between the local feature extraction modules.

9. An image segmentation method, characterized in that: include: Downsampling the first medical image through an embedding module of the target model to obtain second medical image images at multiple scales; performing multi-scale encoding processing on a first target medical image using a global feature extraction module and a local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target medical image is obtained from a second medical image at the multiple scales and the first medical image; Multi-scale decoding processing is performed on the target feature images of the multiple scales by the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first medical image.

10. An image segmentation method, characterized in that: include: Obtaining a first image to be segmented uploaded by a client; In the cloud server, a first image to be segmented is downsampled using an embedding module of a target model to obtain second images at multiple scales; a global feature extraction module and a local feature extraction module of the target model are used to perform multi-scale encoding on the first target image to obtain target feature images at multiple scales, wherein the first target image is obtained from the second images at multiple scales and the first image; a multi-scale decoding process is performed on the target feature images at multiple scales using the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image; The image segmentation result is returned to the client.

11. An image segmentation device, characterized in that: include: A first processing unit is configured to perform downsampling processing on the first image to be segmented by using an embedding module of a target model to obtain second images of multiple scales; a second processing unit, configured to perform multi-scale encoding processing on the first target image using the global feature extraction module and the local feature extraction module of the target model to obtain target feature images at multiple scales, wherein the first target image is obtained from the second image at the multiple scales and the first image; The determining unit is configured to perform multi-scale decoding processing on the target feature images of the multiple scales through the global feature extraction module and the local feature extraction module to obtain an image segmentation result corresponding to the first image.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the image segmentation method according to any one of claims 1 to 10.

13. An electronic device, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program, when running, executes the image segmentation method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The method comprises a computer program or an instruction, which implements the image segmentation method according to any one of claims 1 to 10 when executed by a processor.

Images