Image segmentation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image segmentation method and device, electronic equipment and a storage medium, and belongs to the technical field of image segmentation. The method comprises the following steps: acquiring a target pathological image; performing three-way section feature modeling on the target pathological image to obtain pathological three-way fusion features; performing down-sampling processing on the pathological three-dimensional fusion features to obtain down-sampled pathological image features; performing jump connection on the pathological three-way fusion features and the down-sampling pathological image features to obtain pathological fusion features; performing up-sampling processing on the pathological fusion features to obtain pathological feature image pixels containing pixel pathological probability information; based on the pixel pathology probability information, carrying out pixel classification on the pathology feature image pixels to obtain target pathology pixels; and carrying out pixel splicing on the target pathological pixels to obtain a pathological segmentation image. According to the embodiment of the invention, the accuracy of image segmentation can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to an image segmentation method and device, electronic equipment and storage medium. Background Art

[0002] Image segmentation refers to the cutting and separation of objects in an image. For example, in a tumor pathology image containing tumor tissue, by performing image segmentation on the tumor pathology image, a target image containing only the tumor tissue and a background layer of the tumor pathology image can be obtained.

[0003] At present, the pathological image segmentation method usually uses a trained convolutional neural network to identify pathological tissues on the input pathological images, obtain the probability that each part of the pathological image contains pathological tissue, and then segment the pathological image according to the probability. However, the convolutional neural network has local receptive field limitations, resulting in poor global learning ability and easy to cause image segmentation errors. Therefore, how to improve the accuracy of pathological image segmentation has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an image segmentation method and device, an electronic device and a storage medium, aiming to improve the accuracy of pathological image segmentation.

[0005] To achieve the above object, a first aspect of an embodiment of the present application provides an image segmentation method, the method comprising:

[0006] Acquire a target pathological image, wherein the target pathological image includes a pathological tissue image, and the pathological tissue image is used to represent an image portion of the target pathological image that includes pathological tissue;

[0007] Acquire an image segmentation model, wherein the image segmentation model includes a coding layer and a decoding layer;

[0008] Based on the coding layer, three-dimensional slice feature modeling is performed on the target pathological image to obtain a three-dimensional fusion feature of the pathology;

[0009] Based on the coding layer, down-sampling the pathological three-way fusion feature is performed to obtain a down-sampled pathological image feature;

[0010] Based on the decoding layer, performing jump connection on the pathology three-way fusion feature and the downsampled pathology image feature to obtain a pathology fusion feature;

[0011] Based on the decoding layer, upsampling the pathological fusion feature to obtain an upsampled pathological feature image, wherein the upsampled pathological feature image includes a plurality of pathological feature map pixels, and the pathological feature map pixels include pixel pathological probability information, and the pixel pathological probability information is used to indicate the probability that the pathological feature map pixel belongs to the pathological tissue image;

[0012] Based on the pixel pathology probability information, pixel classification is performed on the pathology feature image pixels to obtain target pathology pixels;

[0013] The target pathological pixels are pixel-stitched to obtain a pathological segmentation image.

[0014] In some embodiments, the three-way slice feature modeling of the target pathology image based on the coding layer to obtain the three-way pathology fusion feature includes:

[0015] Based on the coding layer, performing three-dimensional anatomical processing on the target pathological image to obtain a three-dimensional pathological image;

[0016] Based on the coding layer, flattening the pathology three-dimensional map to obtain a pathology three-dimensional flattened map;

[0017] Based on the coding layer, feature modeling is performed on the pathology three-way flattened image to obtain the pathology three-way fusion feature.

[0018] In some embodiments, the flattening the pathology three-dimensional map based on the coding layer to obtain the pathology three-dimensional flattened map includes:

[0019] Based on the coding layer, extracting geometric features of the pathological three-dimensional image to obtain geometric features of the three-dimensional image;

[0020] Based on the coding layer and the geometric features of the three-dimensional image, the pathology three-dimensional image is dimensionally compressed to obtain the pathology three-dimensional flattened image.

[0021] In some embodiments, the performing feature modeling on the pathology three-way flattened image based on the coding layer to obtain the pathology three-way fusion feature includes:

[0022] Based on the coding layer, feature learning is performed on the pathological three-dimensional flattened image to obtain pathological three-dimensional features;

[0023] Based on the coding layer, information interaction is performed on the pathology three-way flattened image to obtain three-way feature image context information;

[0024] Based on the context information of the three-way feature graph, the pathology three-way features are fused to obtain the pathology three-way fused features.

[0025] In some embodiments, the performing feature learning on the pathology three-way flattened image based on the coding layer to obtain the pathology three-way feature includes:

[0026] Based on the coding layer, performing sequence conversion on the pathological three-dimensional flattened image to obtain a pathological three-dimensional flattened vector;

[0027] Based on the coding layer, position coding and integration are performed on the pathological three-dimensional flattened vector to obtain a target three-dimensional flattened vector;

[0028] Based on the coding layer, feature extraction is performed on the target three-dimensional flattened vector to obtain the pathological three-dimensional feature.

[0029] In some embodiments, the information interaction of the pathology three-way flattened image based on the coding layer to obtain the three-way feature image context information includes:

[0030] Based on the encoding layer, convolution processing is performed on the pathological three-way flattened image to obtain an attention feature map;

[0031] Based on the encoding layer and the attention feature map, calculating the attention weight of the pathological three-way flattened image to obtain the three-way image attention weight;

[0032] Based on the encoding layer and the three-way graph attention weight, feature enhancement is performed on the attention feature map to obtain an enhanced feature representation;

[0033] Based on the coding layer and the enhanced feature representation, the long-range dependency relationship of the pathological three-way flattened image is captured to obtain the context information of the three-way feature image.

[0034] In some embodiments, the downsampling of the pathological three-way fusion feature based on the coding layer to obtain the downsampled pathological image feature includes:

[0035] Based on the encoding layer, convolution processing is performed on the pathological three-way fusion feature to obtain a target three-way fusion feature;

[0036] Based on the coding layer, function activation processing is performed on the target three-way fusion feature to obtain a nonlinear fusion feature representation;

[0037] Based on the coding layer, performing a depth-separable convolution process on the nonlinear fusion feature representation to obtain a convolution fusion feature representation;

[0038] Based on the coding layer, performing quadratic function activation processing on the convolution fusion feature representation to obtain a quadratic nonlinear feature representation;

[0039] Based on the coding layer, the feature dimension of the secondary nonlinear feature representation is adjusted to obtain the downsampled pathological image feature.

[0040] To achieve the above objective, a second aspect of an embodiment of the present application provides an image segmentation device, the device comprising:

[0041] An image acquisition module, used for acquiring a target pathological image, wherein the target pathological image includes a pathological tissue image, and the pathological tissue image is used for representing an image portion containing pathology in the target pathological image;

[0042] A model acquisition module, used to acquire an image segmentation model, wherein the image segmentation model includes a coding layer and a decoding layer;

[0043] A feature modeling module, used for performing three-way slice feature modeling on the target pathological image based on the coding layer to obtain a three-way fusion feature of the pathology;

[0044] A feature downsampling module, used for downsampling the pathological three-way fusion feature based on the coding layer to obtain a downsampled pathological image feature;

[0045] A skip connection module, used for performing skip connection on the pathology three-way fusion feature and the downsampled pathology image feature based on the decoding layer to obtain a pathology fusion feature;

[0046] A feature upsampling module, configured to perform upsampling processing on the pathology fusion feature based on the decoding layer to obtain an upsampled pathology feature image, wherein the upsampled pathology feature image includes a plurality of pathology feature map pixels, and the pathology feature map pixels include pixel pathology probability information, and the pixel pathology probability information is used to indicate the probability that the pathology feature map pixels belong to the pathology tissue image;

[0047] A pixel classification module, used to classify the pixels of the pathological feature map based on the pixel pathological probability information to obtain target pathological pixels;

[0048] The image segmentation module is used to perform pixel splicing on the target pathological pixels to obtain a pathological segmentation image.

[0049] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0050] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0051] The image segmentation method and device, electronic device and storage medium proposed in the present application utilize the encoding layer of the image segmentation model to perform three-way slice feature modeling on the target pathological image to obtain the pathological three-way fusion feature, thereby enhancing the image segmentation model's ability to capture the multi-view features of the target pathological image. Furthermore, by downsampling and skipping the pathological three-way fusion feature, the pathological fusion feature is obtained, which can effectively integrate and refine the pathological three-way fusion feature. Finally, by utilizing the decoding layer of the image segmentation model, the pathological fusion feature is upsampled and transposed convolutionally processed to achieve the recovery of the spatial resolution of the target pathological image, thereby achieving accurate segmentation of the pathological tissue image in the target pathological image and improving the accuracy of pathological image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of the image segmentation method provided in an embodiment of the present application;

[0053] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.

[0054] Figure 3 yes Figure 2 Flow chart of step S202 in FIG.

[0055] Figure 4 yes Figure 2 Flow chart of step S203 in FIG.

[0056] Figure 5 yes Figure 4 Flow chart of step S401 in FIG.

[0057] Figure 6 yes Figure 4 Flow chart of step S402 in FIG.

[0058] Figure 7 yes Figure 1 Flow chart of step S104 in FIG.

[0059] Figure 8 is a schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application;

[0060] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0064] First, some nouns involved in this application are analyzed:

[0065] Convolutional Neural Network: Convolutional Neural Network is a deep learning model that processes and analyzes data by simulating the working principle of the human visual system. It is particularly good at processing data with grid structures, such as images. CNN extracts local features of input data through convolutional layers, reduces the spatial dimension of features through pooling layers, and performs final classification or regression tasks through fully connected layers. It is widely used in image recognition, classification, segmentation, and object detection.

[0066] Axial self-attention mechanism: Axial self-attention mechanism is a method to reduce the amount of computation when processing high-dimensional data. It achieves global modeling of the data by applying self-attention in two orthogonal directions (such as rows and columns of an image) respectively. This mechanism first performs self-attention operations on one dimension of the image and then on the other dimension, effectively reducing the computational complexity. The axial self-attention mechanism is particularly suitable for processing large inputs, such as high-resolution images, because it can capture global dependencies of the data while maintaining computational efficiency.

[0067] 2D Selective Scanning Mechanism: 2D selective scanning mechanism refers to selectively scanning and processing on 2D image data to extract and utilize key feature information in the image. This mechanism captures global context information by scanning the image in different directions while maintaining computational efficiency.

[0068] Image segmentation refers to the cutting and separation of objects in an image. For example, in a pathological image containing pathology, by performing image segmentation on the pathological image, a target image containing only the pathology and the background layer of the pathological image can be obtained.

[0069] At present, the pathological image segmentation method usually uses a trained convolutional neural network to perform pathological recognition on the input pathological image, obtain the probability that each part of the pathological image contains pathology, and then segment the pathological image according to the probability. However, the convolutional neural network has local receptive field limitations, resulting in poor global learning ability and easy to cause image segmentation errors. Therefore, how to improve the accuracy of pathological image segmentation has become a technical problem that needs to be solved urgently.

[0070] Based on this, the embodiments of the present application provide an image segmentation method and device, an electronic device and a storage medium, aiming to improve the accuracy of pathological image segmentation.

[0071] The image segmentation method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the image segmentation method in the embodiments of the present application is described.

[0072] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0074] The image segmentation method provided in the embodiment of the present application relates to the field of image segmentation technology. The image segmentation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the image segmentation method, etc., but is not limited to the above forms.

[0075] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0076] Figure 1 is an optional flowchart of the image segmentation method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S108.

[0077] Step S101, acquiring a target pathological image, wherein the target pathological image includes a pathological tissue image, and the pathological tissue image is used to represent an image portion of the target pathological image that includes pathological tissue;

[0078] Step S102, obtaining an image segmentation model, wherein the image segmentation model includes a coding layer and a decoding layer;

[0079] Step S103, based on the coding layer, three-dimensional slice feature modeling is performed on the target pathological image to obtain a three-dimensional fusion feature of the pathology;

[0080] Step S104, based on the coding layer, down-sampling the pathological three-way fusion features to obtain down-sampled pathological image features;

[0081] Step S105, based on the decoding layer, performing jump connection on the pathology three-way fusion features and the down-sampled pathology image features to obtain the pathology fusion features;

[0082] Step S106, based on the decoding layer, upsampling the pathology fusion feature to obtain an upsampled pathology feature image, wherein the upsampled pathology feature image includes a plurality of pathology feature map pixels, the pathology feature map pixels include pixel pathology probability information, and the pixel pathology probability information is used to indicate the probability that the pathology feature map pixel belongs to the pathology tissue image;

[0083] Step S107, based on the pixel pathology probability information, pixel classification is performed on the pathology feature image pixels to obtain target pathology pixels;

[0084] Step S108, performing pixel stitching on the target pathological pixels to obtain a pathological segmentation image.

[0085] Steps S101 to S108 shown in the embodiment of the present application can perform image segmentation processing on the acquired target pathological image through an image segmentation model including a coding layer and a decoding layer to segment the pathological tissue image in the target pathological image, specifically including: first, using the coding layer of the image segmentation model, three-way slice feature modeling is performed on the target pathological image to obtain the pathological three-way fusion feature, and then using the coding layer of the image segmentation model, the pathological three-way fusion feature is downsampled to obtain the downsampled pathological image feature, further, using the decoding layer of the image segmentation model, the pathological three-way fusion feature and the downsampled pathological image feature are jump-connected to obtain the pathological fusion feature, and then, using the decoding layer of the image segmentation model, the pathological fusion feature is upsampled to obtain an upsampled pathological feature image composed of multiple pathological feature map pixels containing pixel pathology probability information, and finally, according to the above-mentioned pixel pathology probability information, the pathological feature map pixels are pixel classified to obtain the target pathological pixels, and then the target pathological pixels are pixel spliced ​​to obtain the pathological segmented image. Therefore, the present application utilizes the encoding layer of the image segmentation model to perform three-way slice feature modeling on the target pathological image to obtain the pathological three-way fusion feature, thereby enhancing the image segmentation model's ability to capture the multi-perspective features of the target pathological image. Furthermore, by downsampling and jump-connecting the pathological three-way fusion feature, the pathological fusion feature is obtained, which can effectively integrate and refine the pathological three-way fusion feature. Finally, the decoding layer of the image segmentation model is used to upsample the pathological fusion feature to achieve the recovery of the spatial resolution of the target pathological image, thereby achieving accurate segmentation of the pathological tissue image in the target pathological image and improving the accuracy of pathological image segmentation.

[0086] In step S101 of some embodiments, the target pathological image refers to an image containing pathological tissue and background layers, for example, a breast cancer tumor image containing a breast cancer tumor. It should be noted that the target pathological image includes a pathological tissue image and a non-pathological tissue image, wherein the pathological tissue image is used to represent the image layer portion containing the pathological tissue in the target pathological image, and conversely, the non-pathological tissue image is used to represent the background layer portion in the target pathological image.

[0087] In an embodiment of the present application, in order to improve the resolution and contrast of soft tissue and reduce ionizing radiation damage, a dynamic enhanced magnetic resonance imaging method can be used to obtain an image containing pathological tissue, thereby obtaining a target pathological image. In addition, through dynamic enhanced magnetic resonance imaging, multi-directional imaging of pathological tissue can also be achieved, so that the target pathological image can include images in multiple directions.

[0088] In step S102 of some embodiments, the image segmentation model is a lightweight hybrid convolutional neural network - Mamba and a U-type network of axial self-attention. The image segmentation model is used to perform image segmentation on images containing pathological tissue to achieve image separation of the pathological tissue part and the background part. It should be noted that the image segmentation model can be divided into two parts: a coding layer and a decoding layer, wherein the coding layer includes multiple residual blocks and multiple mixed blocks, the mixed blocks include multi-view axial self-attention blocks and dense blocks, and the decoding layer includes multiple convolution blocks and multiple deconvolution blocks.

[0089] In an embodiment of the present application, a pathological sample image can be obtained by acquiring a pathological image with annotated pathological tissue parts, and then the pathological sample image can be used to train a pre-constructed initial image segmentation model so that the model parameters in the initial image segmentation model can handle the segmentation problem of the pathological image, thereby completing the model training of the initial image segmentation model and obtaining an image segmentation model.

[0090] In step S103 of some embodiments, the pathology three-way fusion feature is a feature representation obtained by fusing the features of the target pathology image in the cross-sectional orientation, the coronal orientation, and the sagittal orientation.

[0091] In an embodiment of the present application, the encoding layer of the image segmentation model can be used to perform anatomical processing on the input target pathological image in three orientations: cross-section, coronal and sagittal planes, to obtain a pathological three-dimensional image including a pathological cross-sectional image, a pathological coronal image and a pathological sagittal image. Furthermore, the pathological three-dimensional image is converted into a two-dimensional pathological three-dimensional flattened image, and feature modeling is performed on the pathological three-dimensional flattened image to obtain a pathological three-dimensional fusion feature.

[0092] For details, see Figure 2In some embodiments, step S103 may include but is not limited to steps S201 to S203:

[0093] Step S201, based on the coding layer, performing three-dimensional anatomical processing on the target pathological image to obtain a three-dimensional pathological image;

[0094] Step S202, based on the coding layer, flattening the pathology three-dimensional map to obtain a pathology three-dimensional flattened map;

[0095] Step S203: Based on the coding layer, feature modeling is performed on the pathology three-way flattened image to obtain the pathology three-way fusion feature.

[0096] In step S201 of some embodiments, the pathological three-dimensional image refers to an image representation of the target pathological image in three orientations: cross-sectional, coronal, and sagittal planes. The pathological three-dimensional image includes a pathological cross-sectional image, a pathological coronal image, and a pathological sagittal image.

[0097] In an embodiment of the present application, the above-mentioned multi-view axial attention module can be used to perform pixel orientation recognition on each pathological image voxel in the target pathological image to obtain the imaging orientation of the pathological image voxel. Furthermore, by performing pixel aggregation on the pathological image voxels with the same imaging orientation, a pathological cross-sectional image, a pathological coronal image or a pathological sagittal image can be obtained.

[0098] It should be noted that since the observation clarity of pathological tissue lesions in each orientation is inconsistent, the channel usage ratio of the pathological three-dimensional image is also different. For example, because the cross-section provides the clearest observation of bilateral breast lesions, 50% of the model channels in the image segmentation model can be allocated to model operations in the cross-sectional dimension, and the remaining 50% of the model channels can be evenly distributed to the coronal and sagittal dimensions.

[0099] In step S202 of some embodiments, the pathology three-dimensional flattened image refers to an image representation of the above pathology three-dimensional image in a two-dimensional layer.

[0100] In an embodiment of the present application, based on the geometric features of each pathological three-dimensional image, the above-mentioned multi-perspective axial attention module can be used to compress the pathological three-dimensional image in height, width or depth to achieve flattening of the pathological three-dimensional image and obtain a pathological three-dimensional flattened image.

[0101] For details, see Figure 3 In some embodiments, step S202 may include but is not limited to steps S301 to S302:

[0102] Step S301, based on the coding layer, extracting geometric features of the pathology three-dimensional image to obtain geometric features of the three-dimensional image;

[0103] Step S302: based on the coding layer and the geometric features of the three-dimensional image, the pathology three-dimensional image is dimensionally compressed to obtain a pathology three-dimensional flattened image.

[0104] In step S301 of some embodiments, the three-dimensional image geometric features refer to the features of the three dimensions of height, width and depth of the pathology three-dimensional image.

[0105] In the embodiment of the present application, the geometric features of the pathology three-dimensional map, that is, the three-dimensional map geometric features, can be obtained by calculating the number of pixels in the three dimensions of height, width and depth in the pathology three-dimensional map.

[0106] In step S302 of some embodiments, dimensional compression of the pathology three-dimensional map can be achieved by merging two dimensions in the pathology three-dimensional map, and then the pathology three-dimensional map is mapped from three-dimensional space to two-dimensional space to obtain a pathology three-dimensional flattened map. For example, dimensional compression of the pathology cross-sectional map can be achieved by dimensionally merging the voxels in the depth dimension and the voxels in the width dimension of the pathology cross-sectional map.

[0107] It should be noted that in order to ensure that the pathological three-way flattened map can adapt to the subsequent processing of the encoding layer, after obtaining the pathological three-way flattened map, the pathological three-way flattened map needs to be dimensionally adjusted so that the pathological three-way flattened map can adapt to a specific network structure.

[0108] In steps S301 to S302 shown in this embodiment, the coding layer can be used to extract the geometric features of the pathology three-dimensional map to obtain the geometric features of the three-dimensional map. Then, based on the geometric features of the three-dimensional map, the pathology three-dimensional map is dimensionally compressed to obtain the pathology three-dimensional flattened map, so as to map the pathology three-dimensional map from three-dimensional space to two-dimensional space, thereby reducing the computational complexity of three-dimensional slice feature modeling of the target pathology image and improving the efficiency of three-dimensional slice feature modeling of the target pathology image.

[0109] In step S203 of some embodiments, the bidirectional state space model in the above-mentioned multi-view axial attention module can be used to learn the image features of the pathological three-way flattened image to obtain the pathological three-way features, and then the axial self-attention mechanism is used to interact with the information in the pathological three-way flattened image to obtain the three-way feature map context information. Finally, according to the obtained three-way feature map context information, the pathological three-way features are feature fused to obtain the pathological three-way fused features.

[0110] For details, see Figure 4 In some embodiments, step S203 may include but is not limited to steps S401 to S403:

[0111] Step S401, based on the coding layer, feature learning is performed on the pathology three-dimensional flattened image to obtain the pathology three-dimensional features;

[0112] Step S402, based on the coding layer, information interaction is performed on the pathology three-way flattened image to obtain three-way feature image context information;

[0113] Step S403, based on the context information of the three-way feature graph, feature fusion is performed on the pathology three-way features to obtain the pathology three-way fused features.

[0114] In step S401 of some embodiments, the pathological three-dimensional flattened image can be converted into a vector identifier in a sequence form, i.e., a pathological three-dimensional flattened vector, and combined with a bidirectional scanning mechanism, the visual information of the pathological three-dimensional flattened vector in the horizontal and vertical directions can be captured. Furthermore, in order to enhance the spatial perception ability of the pathological three-dimensional flattened vector, position coding is integrated in the pathological three-dimensional flattened vector to obtain a target three-dimensional flattened vector. Finally, feature extraction is performed on the target three-dimensional flattened vector to obtain a pathological three-dimensional feature.

[0115] For details, see Figure 5 In some embodiments, step S401 may include but is not limited to steps S501 to S503:

[0116] Step S501, based on the coding layer, performing sequence conversion on the pathology three-dimensional flattened image to obtain the pathology three-dimensional flattened vector;

[0117] Step S502, based on the coding layer, position coding and integrating the pathological three-dimensional flattened vector to obtain a target three-dimensional flattened vector;

[0118] Step S503: Based on the coding layer, feature extraction is performed on the target three-dimensional flattened vector to obtain a pathological three-dimensional feature.

[0119] In step S501 of some embodiments, the pathological three-dimensional flattened image can be converted into a pathological three-dimensional flattened sequence in sequence form through a two-dimensional selective scanning mechanism, and then a pre-trained state space model combined with a bidirectional scanning mechanism can be used to capture the visual information of the pathological three-dimensional flattened sequence in the horizontal and vertical directions to obtain horizontal visual information and vertical visual information. Furthermore, the horizontal visual information and the vertical visual information are integrated into the pathological three-dimensional flattened sequence to obtain a pathological three-dimensional flattened vector.

[0120] In step S502 of some embodiments, position coding can be added to the pathology three-way flattened vector to provide position information of each element in the pathology three-way flattened vector, thereby improving the spatial perception ability of the image segmentation model for the pathology three-way flattened vector.

[0121] In step S503 of some embodiments, the pathological three-dimensional feature may be obtained by extracting the feature vector from the target three-dimensional flattened vector.

[0122] In steps S501 to S503 shown in the present embodiment, the pathology three-way flattened image is sequence-converted using a coding layer to obtain a pathology three-way flattened vector, and then the pathology three-way flattened vector is position-coded and integrated to obtain a target three-way flattened vector. Furthermore, feature extraction is performed on the target three-way flattened vector to obtain a pathology three-way feature. This can realize state transfer of the pathology three-way flattened image, thereby realizing long-range dependency modeling of the pathology three-way flattened image, improving the feature extraction capability of the image segmentation model, and reducing the computational complexity of feature extraction, thereby improving the efficiency of pathology image segmentation.

[0123] In step S403 of some embodiments, a convolution operation can be performed on the pathological three-way flattened image according to the axial self-attention mechanism in the coding layer to generate an attention feature map, and then the three-way graph attention weight of the pathological three-way flattened image is calculated using the coding layer according to the attention feature map. Then, the attention feature map is feature enhanced by combining the coding layer and the calculated three-way graph attention weight to enhance the feature representation of the attention feature map to obtain an enhanced feature representation. Finally, the long-range dependencies in the pathological three-way flattened image are captured through the coding layer and the enhanced feature representation, thereby obtaining the three-way feature graph context information between the pathological three-way flattened images.

[0124] For details, see Figure 6 In some embodiments, step S402 may include but is not limited to steps S601 to S604:

[0125] Step S601, based on the coding layer, convolution processing is performed on the pathology three-way flattened image to obtain an attention feature map;

[0126] Step S602, based on the coding layer and the attention feature map, the attention weight is calculated for the pathology three-way flattened image to obtain the three-way image attention weight;

[0127] Step S603, based on the coding layer and the three-way graph attention weight, feature enhancement is performed on the attention feature map to obtain an enhanced feature representation;

[0128] Step S604: Based on the coding layer and the enhanced feature representation, the long-range dependency relationship of the pathology three-way flattened image is captured to obtain the three-way feature image context information.

[0129] In step S601 of some embodiments, the attention feature map is the core part of the self-attention mechanism in the image segmentation model, and the attention feature map includes a query feature map, a key feature map and a value feature map, wherein the query feature map represents the information that needs to be paid attention to at the current position of the pathology three-way flattened image, the key feature map represents the features of the pathology three-way flattened image at each position, and the value feature map represents the actual content of the pathology three-way flattened image at each position.

[0130] In an embodiment of the present application, a query feature map, a key feature map, and a value feature map of the pathology three-way flattened image can be generated by performing a 1×1×1 convolution operation on the pathology three-way flattened image and combining different pre-set parameters.

[0131] In step S602 of some embodiments, the attention weight matrix between the pathological three-way flattened graphs can be calculated using a scaled dot product attention method to obtain the three-way graph attention weight, wherein the scaled dot product attention method is to calculate the dot product of the query feature map and the key feature map, and then divide it by the relevant numerical value of the dimension of the key feature map to scale it, and finally calculate it through the softmax function to obtain the attention weight.

[0132] In step S603 of some embodiments, feature enhancement of the attention feature map can be achieved by performing matrix multiplication of the above-mentioned three-way graph attention weights and the value feature map, thereby obtaining an enhanced feature representation.

[0133] In step S604 of some embodiments, the axial self-attention mechanism in the encoding layer can capture the enhanced feature representation in different spatial dimensions to achieve long-range dependency capture of the pathological three-way flattened image, thereby obtaining the three-way feature map context information.

[0134] It is also necessary to know that the shape feature expression of the pathological three-way flattened image before convolution processing is (B, C, l1, l2, l3), where B is the batch size of the pathological three-way flattened image, C is the number of channels of the pathological three-way flattened image, l1, l2, l3 are the three-dimensional spatial dimensions of the pathological three-way flattened image, and in order to facilitate the calculation of the three-way image attention weight of the pathological three-way flattened image, after obtaining the attention feature mapping, the shape feature expression of the pathological three-way flattened image needs to be reshaped into a reshaped shape feature expression in the style of (B×l1×l2, l3, C), and after obtaining the enhanced feature representation, the reshaped shape feature expression is reshaped back to the shape feature expression.

[0135] In steps S601 to S604 shown in the present embodiment, the pathology three-way flattened image is convolved through the coding layer to obtain an attention feature map, and then the attention weights are calculated for the pathology three-way flattened image based on the attention feature map to obtain the three-way image attention weights. Furthermore, based on the three-way image attention weights, the attention feature maps are feature enhanced to obtain an enhanced feature representation. Finally, based on the enhanced feature representation, the long-range dependencies of the pathology three-way flattened image are captured to obtain the contextual information of the three-way feature map, which can enhance the expressive power of the pathology three-way features, thereby improving the accuracy of the image segmentation model in segmenting the target pathology image.

[0136] In step S403 of some embodiments, the pathological three-way features of the pathological three-way flattened image may be channel-fused through 1×1×1 convolution according to the three-way feature image context information to obtain the pathological three-way fused features.

[0137] In steps S401 to S403 shown in the present embodiment, feature learning is performed on the pathological three-way flattened image according to the coding layer in the image segmentation model to obtain the pathological three-way feature, and then information interaction is performed on the pathological three-way flattened image to obtain the three-way feature map context information. Finally, based on the three-way feature map context information, feature fusion is performed on the pathological three-way feature to obtain the pathological three-way fused feature. This can ensure that the complexity of three-way slice feature modeling of the target pathological image is always maintained in the dimension of linear calculation, thereby improving the efficiency of three-way slice feature modeling of the target pathological image. In addition, by performing feature fusion on the image features of the pathological three-way flattened image, the accuracy of the image features is improved, thereby improving the accuracy of pathological image segmentation.

[0138] In steps S201 to S203 shown in the present embodiment, the target pathological image is subjected to three-dimensional anatomical processing through the encoding layer of the image segmentation model to obtain a pathological three-dimensional map, which is then flattened into a pathological three-dimensional flattened map, and then feature modeling is performed on the pathological three-dimensional flattened map. This can reduce the complexity of three-dimensional slice feature modeling of the target pathological image and improve the efficiency of three-dimensional slice feature modeling of the target pathological image.

[0139] In step S104 of some embodiments, the downsampled pathological image features refer to image features that have been subjected to convolution processing.

[0140] In an embodiment of the present application, the coding layer can be used to perform preliminary convolution processing on the pathological three-way fusion features to obtain target three-way fusion features, so that the dimensions of the target three-way fusion features can adapt to the model operation. Furthermore, by performing function activation processing on the target three-way fusion features, a nonlinear fusion feature representation is obtained to complete the preliminary downsampling processing of the pathological three-way fusion features. Secondly, by performing deep separable convolution processing and quadratic function activation processing on the nonlinear fusion feature representation, a secondary nonlinear feature representation is obtained to complete the secondary downsampling processing of the pathological three-way fusion features. Finally, by adjusting the feature dimension of the secondary nonlinear feature representation, the downsampled pathological image features are obtained.

[0141] For details, see Figure 7 In some embodiments, step S104 may include but is not limited to steps S701 to S705:

[0142] Step S701, based on the coding layer, convolution processing is performed on the pathological three-way fusion feature to obtain the target three-way fusion feature;

[0143] Step S702, based on the coding layer, function activation processing is performed on the target three-way fusion feature to obtain a nonlinear fusion feature representation;

[0144] Step S703, based on the coding layer, performing a depth-separable convolution process on the nonlinear fusion feature representation to obtain a convolution fusion feature representation;

[0145] Step S704, based on the coding layer, performing quadratic function activation processing on the convolution fusion feature representation to obtain a quadratic nonlinear feature representation;

[0146] Step S705: Based on the coding layer, the feature dimension of the quadratic nonlinear feature representation is adjusted to obtain downsampled pathological image features.

[0147] In steps S701 and S702 of some embodiments, the pathological three-way fusion features can be input into the convolution layer of the 1×1×1 convolution in the encoding layer for convolution processing to obtain the target three-way fusion features, and then the target three-way fusion features are instance normalized and activated by the LeakyRelu activation function to obtain a nonlinear fusion feature representation.

[0148] In step S703 and step S704 of some embodiments, a convolutional fusion feature representation can be obtained by inputting the above-mentioned nonlinear fusion feature representation into a 3×3×3 group of convolutional layers with a step size of 2 in the encoding layer for depth-separable convolution processing. Furthermore, the convolutional fusion feature representation is instance normalized and activated by the LeakyRelu activation function to obtain a quadratic nonlinear feature representation.

[0149] In step S705 of some embodiments, the feature dimension of the quadratic nonlinear feature representation may be adjusted using a convolutional layer of 1×1×1 convolution in the encoding layer, thereby obtaining downsampled pathological image features that are adapted to model channel transport.

[0150] In steps S701 to S705 shown in the present embodiment, the pathological three-way fusion features are subjected to convolution processing and function activation processing through the coding layer to obtain a nonlinear fusion feature representation, thereby achieving preliminary downsampling of the pathological three-way fusion features. The nonlinear fusion feature representation is then subjected to deep separable convolution processing and quadratic function activation processing to achieve secondary downsampling of the pathological three-way fusion features, thereby improving the accuracy of the downsampled pathological image features, thereby improving the accuracy of the image segmentation model in segmenting the target pathological image.

[0151] In step S105 of some embodiments, the decoding layer can be used to perform an average pooling operation on the pathology three-way fusion features to obtain pooled pathology image features, and then the pooled pathology image features are added to the above-mentioned downsampled pathology image features to achieve a jump connection between the pathology three-way fusion features and the downsampled pathology image features to obtain the pathology fusion features, which helps the pathology three-way fusion features flow in the image segmentation model, so that the image segmentation model can learn more effective feature representations.

[0152] It should be noted that the above-mentioned three-way slice feature modeling, downsampling processing and jump connection operations are cyclic operations, wherein the three-way slice feature modeling operation is performed in the mixed block of the above-mentioned encoding layer, the downsampling processing operation is performed in the residual block of the above-mentioned encoding layer, and the jump connection operation is performed in the above-mentioned decoding layer.

[0153] It is also important to know that in order to reduce the number of channels in the image segmentation model and avoid errors when the pathological three-way fusion features are jump-connected with the downsampled pathological image features, the present application can use the convolution block in the decoding layer to perform feature fusion on the feature representations between channels before the pathological three-way fusion features and the downsampled pathological image features are jump-connected, thereby reducing the number of channels and ensuring the normal operation of the image segmentation model.

[0154] In step S106 of some embodiments, in order to restore the size of the pathological fusion feature, the pathological fusion feature can be upsampled to enlarge the size of the pathological fusion feature. Specifically, a convolution operation can be performed on the upsampled pathological feature image using a learnable convolution kernel in the decoding layer to refine the upsampled pathological feature image, thereby obtaining a plurality of pathological feature map pixels containing pixel pathology probability information, and forming an upsampled pathological feature map based on the plurality of pathological feature map pixels, thereby ensuring the accuracy of image restoration.

[0155] In step S107 of some embodiments, it can be determined whether the pathology feature map pixel belongs to the pathology tissue image in the target pathology image based on the pixel pathology probability information in each pathology feature map pixel. If the pathology feature map pixel does not belong to the pathology tissue image in the target pathology image, it is classified as a background pathology pixel. If the pathology feature map pixel belongs to the pathology tissue image in the target pathology image, it is classified as a target pathology pixel.

[0156] In step S108 of some embodiments, based on the above-screened target pathological pixels, they are arranged according to their positions in the target pathological image, so as to obtain a pathological segmentation image containing only pathological tissues.

[0157] The present application utilizes the encoding layer of the image segmentation model to perform three-way slice feature modeling on the target pathological image to obtain the pathological three-way fusion feature, thereby enhancing the image segmentation model's ability to capture the multi-view features of the target pathological image. Furthermore, by downsampling and skipping the pathological three-way fusion feature, the pathological fusion feature is obtained, which can effectively integrate and refine the pathological three-way fusion feature. Finally, by utilizing the decoding layer of the image segmentation model, the pathological fusion feature is upsampled and transposed convolutionally processed to achieve the recovery of the spatial resolution of the target pathological image, thereby achieving accurate segmentation of the pathological tissue image in the target pathological image and improving the accuracy of pathological image segmentation.

[0158] See also Figure 8 The embodiment of the present application further provides an image segmentation device, which can implement the above-mentioned image segmentation method, and the device includes:

[0159] An image acquisition module 801 is used to acquire a target pathological image, wherein the target pathological image includes a pathological tissue image, and the pathological tissue image is used to represent an image portion containing pathology in the target pathological image;

[0160] The model acquisition module 802 is used to acquire an image segmentation model, wherein the image segmentation model includes a coding layer and a decoding layer;

[0161] The feature modeling module 803 is used to perform three-dimensional slice feature modeling on the target pathological image based on the coding layer to obtain the pathological three-dimensional fusion feature;

[0162] A feature downsampling module 804 is used to perform downsampling processing on the pathology three-way fusion feature based on the coding layer to obtain a downsampled pathology image feature;

[0163] A skip connection module 805 is used to perform skip connection on the pathology three-way fusion feature and the downsampled pathology image feature based on the decoding layer to obtain the pathology fusion feature;

[0164] A feature upsampling module 806 is used to perform upsampling processing on the pathology fusion feature based on the decoding layer to obtain an upsampled pathology feature image, wherein the upsampled pathology feature image includes a plurality of pathology feature map pixels, and the pathology feature map pixels include pixel pathology probability information, and the pixel pathology probability information is used to indicate the probability that the pathology feature map pixel belongs to the pathology tissue image;

[0165] A pixel classification module 807 is used to classify the pathology feature image pixels based on the pixel pathology probability information to obtain target pathology pixels;

[0166] The image segmentation module 808 is used to perform pixel splicing on the target pathological pixels to obtain a pathological segmentation image.

[0167] The specific implementation of the image segmentation device is basically the same as the specific implementation of the above-mentioned image segmentation method, and will not be repeated here.

[0168] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above image segmentation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0169] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0170] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0171] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the image segmentation method of the embodiment of the present application;

[0172] Input / output interface 903, used to implement information input and output;

[0173] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0174] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0175] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0176] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image segmentation method is implemented.

[0177] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0178] The image segmentation method, image segmentation device, electronic device and storage medium provided in the embodiments of the present application perform image segmentation processing on the acquired target pathological image through an image segmentation model including a coding layer and a decoding layer to segment the pathological tissue image in the target pathological image, specifically comprising: first, using the coding layer of the image segmentation model, three-way slice feature modeling is performed on the target pathological image to obtain the pathological three-way fusion feature; then, using the coding layer of the image segmentation model, the pathological three-way fusion feature is downsampled to obtain the downsampled pathological image feature; further, using the coding layer of the image segmentation model, the pathological three-way fusion feature and the downsampled pathological image feature are jump-connected to obtain the pathological fusion feature; secondly, using the decoding layer of the image segmentation model, the pathological fusion feature is upsampled to obtain an upsampled pathological feature image composed of multiple pathological feature map pixels containing pixel pathology probability information; finally, according to the above-mentioned pixel pathology probability information, the pathological feature map pixels are pixel-classified to obtain the target pathological pixels; and then the target pathological pixels are pixel-spliced ​​to obtain the pathological segmented image.

[0179] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0180] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0181] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0182] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0183] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0184] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0185] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0186] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0188] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0189] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An image segmentation method, characterized in that: The method comprises: Acquire a target pathological image, wherein the target pathological image includes a pathological tissue image, and the pathological tissue image is used to represent an image portion of the target pathological image that includes pathological tissue; Acquire an image segmentation model, wherein the image segmentation model includes a coding layer and a decoding layer; Based on the coding layer, three-dimensional slice feature modeling is performed on the target pathological image to obtain a three-dimensional fusion feature of the pathology; Based on the coding layer, down-sampling the pathological three-way fusion feature is performed to obtain a down-sampled pathological image feature; Based on the decoding layer, performing jump connection on the pathology three-way fusion feature and the downsampled pathology image feature to obtain a pathology fusion feature; Based on the decoding layer, upsampling the pathological fusion feature to obtain an upsampled pathological feature image, wherein the upsampled pathological feature image includes a plurality of pathological feature map pixels, and the pathological feature map pixels include pixel pathological probability information, and the pixel pathological probability information is used to indicate the probability that the pathological feature map pixel belongs to the pathological tissue image; Based on the pixel pathology probability information, pixel classification is performed on the pathology feature image pixels to obtain target pathology pixels; The target pathological pixels are pixel-stitched to obtain a pathological segmentation image.

2. The method according to claim 1, characterized in that: Based on the coding layer, the target pathological image is subjected to three-way slice feature modeling to obtain a pathological three-way fusion feature, including: Based on the coding layer, performing three-dimensional anatomical processing on the target pathological image to obtain a three-dimensional pathological image; Based on the coding layer, flattening the pathology three-dimensional map to obtain a pathology three-dimensional flattened map; Based on the coding layer, feature modeling is performed on the pathology three-way flattened image to obtain the pathology three-way fusion feature.

3. The method according to claim 2, characterized in that The step of flattening the pathology three-dimensional map based on the coding layer to obtain a pathology three-dimensional flattened map includes: Based on the coding layer, extracting geometric features of the pathological three-dimensional image to obtain geometric features of the three-dimensional image; Based on the coding layer and the geometric features of the three-dimensional image, the pathology three-dimensional image is dimensionally compressed to obtain the pathology three-dimensional flattened image.

4. The method according to claim 2, characterized in that: The step of performing feature modeling on the pathology three-way flattened image based on the coding layer to obtain the pathology three-way fusion feature includes: Based on the coding layer, feature learning is performed on the pathological three-dimensional flattened image to obtain pathological three-dimensional features; Based on the coding layer, information interaction is performed on the pathology three-way flattened image to obtain three-way feature image context information; Based on the context information of the three-way feature graph, the pathology three-way features are fused to obtain the pathology three-way fused features.

5. The method according to claim 4, characterized in that The step of performing feature learning on the pathology three-way flattened image based on the coding layer to obtain the pathology three-way features includes: Based on the coding layer, performing sequence conversion on the pathological three-dimensional flattened image to obtain a pathological three-dimensional flattened vector; Based on the coding layer, position coding and integration are performed on the pathological three-dimensional flattened vector to obtain a target three-dimensional flattened vector; Based on the coding layer, feature extraction is performed on the target three-dimensional flattened vector to obtain the pathological three-dimensional feature.

6. The method according to claim 4, characterized in that The step of performing information interaction on the pathology three-way flattened image based on the coding layer to obtain three-way feature image context information includes: Based on the encoding layer, convolution processing is performed on the pathological three-way flattened image to obtain an attention feature map; Based on the encoding layer and the attention feature map, calculating the attention weight of the pathological three-way flattened image to obtain the three-way image attention weight; Based on the encoding layer and the three-way graph attention weight, feature enhancement is performed on the attention feature map to obtain an enhanced feature representation; Based on the coding layer and the enhanced feature representation, the long-range dependency relationship of the pathological three-way flattened image is captured to obtain the context information of the three-way feature image.

7. The method according to any one of claims 1 to 6, characterized in that: The downsampling process is performed on the pathological three-way fusion feature based on the coding layer to obtain the downsampled pathological image feature, including: Based on the encoding layer, convolution processing is performed on the pathological three-way fusion feature to obtain a target three-way fusion feature; Based on the coding layer, function activation processing is performed on the target three-way fusion feature to obtain a nonlinear fusion feature representation; Based on the coding layer, performing a depth-separable convolution process on the nonlinear fusion feature representation to obtain a convolution fusion feature representation; Based on the coding layer, performing quadratic function activation processing on the convolution fusion feature representation to obtain a quadratic nonlinear feature representation; Based on the coding layer, the feature dimension of the secondary nonlinear feature representation is adjusted to obtain the downsampled pathological image feature.

8. An image segmentation device, characterized in that: The device comprises: An image acquisition module, used for acquiring a target pathological image, wherein the target pathological image includes a pathological tissue image, and the pathological tissue image is used for representing an image portion containing pathology in the target pathological image; A model acquisition module, used to acquire an image segmentation model, wherein the image segmentation model includes a coding layer and a decoding layer; A feature modeling module, used for performing three-way slice feature modeling on the target pathological image based on the coding layer to obtain a three-way fusion feature of the pathology; A feature downsampling module, used for downsampling the pathological three-way fusion feature based on the coding layer to obtain a downsampled pathological image feature; A skip connection module, used for performing skip connection on the pathology three-way fusion feature and the downsampled pathology image feature based on the decoded layer to obtain a pathology fusion feature; A feature upsampling module, configured to perform upsampling processing on the pathology fusion feature based on the decoding layer to obtain an upsampled pathology feature image, wherein the upsampled pathology feature image includes a plurality of pathology feature map pixels, and the pathology feature map pixels include pixel pathology probability information, and the pixel pathology probability information is used to indicate the probability that the pathology feature map pixels belong to the pathology tissue image; A pixel classification module, used to classify the pixels of the pathological feature map based on the pixel pathological probability information to obtain target pathological pixels; The image segmentation module is used to perform pixel splicing on the target pathological pixels to obtain a pathological segmentation image.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the image segmentation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image segmentation method according to any one of claims 1 to 7 is implemented.

Images