Medical image segmentation method, system, terminal and storage medium

Abstract

The present application relates to a medical image segmentation method, system, terminal and storage medium. The method includes: extracting superpixels from a first modality image and a second modality image; constructing a first modality graph neural network and a second modality graph neural network based on the superpixels of the first modality image and the second modality image as nodes, respectively, the first modality graph neural network and the second modality graph neural network calculate the adjacency matrix of each superpixel and the neighboring superpixel according to the center position distance and the grayscale mean difference, and calculate the eigenvalue of each superpixel in the current modality according to the adjacency matrix; constructing a multimodal graph neural network based on the superpixels of the first modality image and the second modality image as cross-modality nodes, the multimodal graph neural network fuses multimodal information of different modality images, and segments the first modality image according to the fused feature map. The present invention does not require the pre-fine registration of multimodal images, and improves the segmentation accuracy of medical images.

Description

Technical Field

[0001] The present application belongs to the field of medical image processing technology, and in particular relates to a medical image segmentation method, system, terminal and storage medium. Background Art

[0002] In recent years, the development of artificial intelligence technology in the field of image segmentation has brought new opportunities for the automatic delineation of CT (Computed Tomography) images. Since MR (Magnetic Resonance) images have high soft tissue contrast, when segmenting CT images, doctors often use MR images to delineate the target area in the CT image.

[0003] Since CT and MR images are not registered, the existing technical solutions need to align the two images before image segmentation. The current domain alignment methods are mainly divided into spatial domain alignment and grayscale domain alignment. Among them, spatial domain alignment means that the MR image is registered to the CT image through the classic iterative optimization algorithm or deep learning such as VoxelMorph. After the spatial domains of MR and CT images are aligned, the multimodal images will be input into the deep learning neural network as multi-channel tensors. Then, the multimodal information is adaptively fused through the convolution module and attention module in the neural network. This method can effectively extract features in different modalities that are helpful for the segmentation of the target area, thereby improving the segmentation accuracy. The shortcomings of this method are that the segmentation result is more dependent on the registration result, and in actual operation, it is difficult to achieve global and local precise alignment at the same time through the set optimization target, resulting in the integration of different categories of features in the later stage of multimodal information fusion, which confuses the classifier and reduces the segmentation accuracy; in addition, the classic registration algorithm takes a long time, which reduces the speed of segmentation.

[0004] Grayscale domain alignment means converting CT images into MR image styles through image generation, and then segmenting the MR-style images. The new image generated by this method is a virtual image that lacks certain physical meaning. In addition, while ensuring that the overall image style is close to MR, it is difficult to ensure that the contrast between the target segmentation area and the surrounding tissue is greatly improved. More seriously, this image generation method has a certain risk of changing the subtle structure of the tissue, which will also reduce the accuracy of segmentation. Summary of the invention

[0005] The present application provides a medical image segmentation method, system, terminal and storage medium, aiming to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.

[0006] In order to solve the above problems, this application provides the following technical solutions:

[0007] A medical image segmentation method, comprising:

[0008] Acquire a first modality image and a second modality image, and perform superpixel extraction on the first modality image and the second modality image using a simple linear iterative clustering algorithm;

[0009] Constructing a first modal graph neural network and a second modal graph neural network according to the superpixels of the first modal image and the second modal image as nodes respectively, wherein the first modal graph neural network and the second modal graph neural network calculate the adjacency matrix between each superpixel and the neighboring superpixels according to the center position distance and the grayscale mean difference, and calculate the eigenvalue of each superpixel in the current modality according to the adjacency matrix;

[0010] Constructing a multimodal graph neural network for a cross-modal node according to superpixels of the first modal image and the second modal image, wherein the multimodal graph neural network constructs an adjacency matrix of each cross-modal node according to a position of each cross-modal node in different modal images and a connection pattern with K neighboring nodes;

[0011] Obtaining K neighboring nodes of each cross-modal node in other modalities according to the adjacency matrix of the cross-modal node, fusing the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixels in the current modality, to obtain a multi-modal feature map after feature fusion;

[0012] The multimodal feature map is subjected to image segmentation through a segmentation network.

[0013] The technical solution adopted in the embodiment of the present application also includes: the first modality image is a CT image, and the second modality image is an MR image.

[0014] The technical solution adopted by the embodiment of the present application also includes: the first modal graph neural network and the second modal graph neural network calculate the adjacency matrix of each superpixel and the neighboring superpixel according to the center position distance and the grayscale mean difference, specifically:

[0015]

[0016] Where W k,j represents the adjacency relationship between the kth superpixel and the jth superpixel in the neighborhood, ΔP k,j and ΔG k,j denote the difference between the center position and grayscale mean of the kth superpixel and the jth superpixel, [·||·] denotes vector connection, and f w Represents the mapping relationship between the position grayscale difference and the adjacency matrix, represents the neighborhood of superpixel node k.

[0017] The technical solution adopted by the embodiment of the present application also includes: the characteristic value of each superpixel in the current mode is calculated according to the adjacency matrix as follows:

[0018] Extract the feature vector of each superpixel from the feature map output by the network convolutional layer;

[0019] Determine the K neighboring nodes of each superpixel in the current mode through the adjacency matrix, perform weighted summation on the feature vectors of the K neighboring nodes, and obtain new features of each superpixel in the current mode;

[0020] The new feature is added to the feature values ​​of all pixels in the corresponding superpixel to obtain the feature value of each superpixel in the current mode; the feature vector f of the kth superpixel at the i-th scale is i,k The calculation process includes:

[0021]

[0022]

[0023] Among them, S k represents the kth superpixel, F i,p Represents the feature vector of pixel p in the kth superpixel at the i-th scale; use the adjacency matrix W to transform f i,j Weighted, we get the new feature f′ of the kth superpixel at the i-th scale i,k .

[0024] The technical solution adopted by the embodiment of the present application also includes: the multimodal graph neural network constructs an adjacency matrix of each cross-modal node according to the position of each cross-modal node in the different modal images and the connection mode with K neighboring nodes, specifically:

[0025]

[0026] Where W k,* represents the connection pattern between the kth node and its neighboring nodes in the first modality image, sim(W k,* ,W j,* ) represents the similarity of the connection pattern between the kth node in the first modality image and the jth node in the second modality image, f cross Represents the mapping relationship between position difference, connection pattern similarity and cross-modal adjacency matrix.

[0027] The technical solution adopted by the embodiment of the present application also includes: the fusion of the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixel in the current modality is specifically as follows:

[0028] Perform weighted summation on the feature vectors of the K neighboring nodes of each cross-modal node in other modalities to obtain new features of each cross-modal node in other modalities, and add the new features to the feature values ​​of all pixels in the corresponding superpixel to obtain a fused multi-modal feature map;

[0029] New features of the kth superpixel at the i-th scale in other modalities and the eigenvalue of the pixel p in the kth superpixel in the feature map at the i-th scale The formula for obtaining is:

[0030]

[0031]

[0032] in and denote the feature vectors of the jth neighboring superpixel of the kth superpixel in the current modality image and the other modality images at the i-th scale, respectively, and W k,j and represents the adjacency value of the kth superpixel in the current modality image and the jth neighboring superpixel in the other modality images, respectively, and F i,p Represents the feature vector of pixel p in the kth superpixel at the i-th scale.

[0033] The technical solution adopted by the embodiment of the present application also includes: the image segmentation of the multimodal feature map by the segmentation network is specifically:

[0034] The segmentation network decodes the multimodal feature map, upsamples the multimodal feature map to the size of the first modality image, and outputs a segmentation result of the region of interest in the first modality image.

[0035] Another technical solution adopted by the embodiment of the present application is: a medical image segmentation system, comprising:

[0036] A superpixel extraction module: used for acquiring a first modality image and a second modality image, and performing superpixel extraction on the first modality image and the second modality image using a simple linear iterative clustering algorithm;

[0037] A single-modal network construction module: used to construct a first modal graph neural network and a second modal graph neural network according to the superpixels of the first modal image and the second modal image as nodes, respectively. The first modal graph neural network and the second modal graph neural network calculate the adjacency matrix between each superpixel and the neighboring superpixel according to the center position distance and the grayscale mean difference, and calculate the eigenvalue of each superpixel in the current modality according to the adjacency matrix;

[0038] A multimodal network construction module: used to construct a multimodal graph neural network for a cross-modal node according to superpixels of the first modal image and the second modal image, wherein the multimodal graph neural network constructs an adjacency matrix of each cross-modal node according to the position of each cross-modal node in different modal images and the connection mode with K neighboring nodes;

[0039] Feature fusion module: used to obtain K neighboring nodes of each cross-modal node in other modalities according to the adjacency matrix of the cross-modal node, and fuse the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixels in the current modality to obtain a multi-modal feature map after feature fusion;

[0040] Image segmentation module: used to perform image segmentation on the multimodal feature map through a segmentation network.

[0041] Another technical solution adopted by the embodiment of the present application is: a terminal, the terminal includes a processor and a memory coupled to the processor, wherein:

[0042] The memory stores program instructions for implementing the medical image segmentation method;

[0043] The processor is used to execute the program instructions stored in the memory to control medical image segmentation.

[0044] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the medical image segmentation method.

[0045] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the medical image segmentation method, system, terminal and storage medium of the embodiments of the present application divide images of different modalities into multiple superpixels, and use graph neural networks to establish the correspondence between each superpixel in the images of different modalities and the neighboring superpixels in the current modality and other modalities, thereby fusing multimodal information of images of different modalities, achieving alignment in the spatial domain, solving the problem of information fusion of unregistered multimodal medical images, and improving the overall segmentation speed. The present invention does not require the fine pre-registration of multimodal images, avoids the segmentation error and long time consumption caused by the registration, makes feature fusion more adaptive, and thus improves the segmentation accuracy of medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a medical image segmentation method according to an embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of the structure of a medical image segmentation system according to an embodiment of the present application;

[0048] Figure 3 This is a schematic diagram of the terminal structure of an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] See also Figure 1 , is a flow chart of the medical image segmentation method of the embodiment of the present application. The medical image segmentation method of the embodiment of the present application comprises the following steps:

[0052] S1: acquiring a first modality image and a second modality image, and using a simple linear iterative clustering algorithm (SLIC) to extract superpixels from the first modality image and the second modality image respectively;

[0053] In this step, the first modality image and the second modality image are respectively a CT image and an MR image, and may specifically be medical images of other modalities such as DR.

[0054] S2: constructing a first modality graph neural network and a second modality graph neural network with superpixels of the first modality image and the second modality image as nodes respectively, extracting the adjacency relationship between each node and its neighboring nodes under each modality through the first modality graph neural network and the second modality graph neural network respectively, and calculating the adjacency matrix between each superpixel and its neighboring superpixel according to the center position distance and grayscale mean difference;

[0055] In this step, the first modality graph neural network and the second modality graph neural network are unimodal segmentation networks, which can extract the adjacency relationship between each superpixel (node) and its neighboring nodes in the image of the corresponding modality, and calculate the adjacency matrix of each superpixel by weighting the center position and grayscale mean of the superpixel. The calculation process of the superpixel adjacency matrix W is as follows:

[0056]

[0057] Where ΔP k,j and ΔG k,j denote the difference between the center position and grayscale mean of the kth superpixel and the jth superpixel, [·||·] denotes vector connection, and f w Represents the mapping relationship between the position grayscale difference and the adjacency matrix, represents the neighborhood of superpixel node k, and the calculated W k,jIt represents the adjacency relationship between the kth superpixel and the jth superpixel in the neighborhood.

[0058] S3: Extract the feature vector of each superpixel from the feature map output by the network convolution layer, and determine the K neighboring nodes of each superpixel in the current mode under different modes through the adjacency matrix, perform weighted summation on the feature vectors of the K neighboring nodes, obtain the new features of each superpixel in the current mode, and add the new features to the feature values ​​of all pixels in the corresponding superpixel to obtain the feature value of each superpixel in the current mode;

[0059] In this step, the feature values ​​of all pixels in a superpixel refer to the features of each pixel position in a superpixel. The superpixel index map is scaled to the feature map size, and then the feature values ​​of all pixels in the current superpixel are taken out from the feature map according to the index. Assume that the feature map F of the graph neural network at the i-th scale is i The size is C*H*W. It is necessary to downsample the superpixel index map to H*W and calculate the feature mean of all pixels in each superpixel to finally obtain a C-dimensional feature vector for each superpixel. The adjacency matrix can be used to determine the k-th superpixel S at the i-th scale. k The K neighboring nodes in the current mode are used to identify the superpixel S k The features of the K neighboring nodes are weighted summed to obtain the superpixel S k New feature f′ in the current mode i,k , and then add the new features to the superpixel S k The feature vector f of the kth superpixel at the i-th scale is i,k The calculation process is as follows:

[0060]

[0061]

[0062] In formula (2), S k represents the kth superpixel, F i,p represents the feature vector of pixel p in the kth superpixel at the i-th scale. In formula (3), the adjacency matrix W is used to represent f i,j Weighted, we get the new feature f′ of the kth superpixel at the i-th scale i,k .

[0063] The above calculations are performed at multiple scales to complete the feature extraction of each superpixel under a single modality.

[0064] S4: constructing a multimodal graph neural network for the cross-modal nodes according to the superpixels of the first modal image and the second modal image, and comparing the connection mode between each cross-modal node and the K neighboring nodes, and determining the adjacency matrix of each cross-modal node in the multimodal graph neural network according to the node position and the connection mode;

[0065] In this step, due to the large gap in the grayscale domain of multimodal images, there is a great risk in directly using grayscale values ​​or feature maps in the network to calculate the adjacency matrix of cross-modal nodes. Since the overall tissue and organ structure in the multimodal image is unchanged, the connection mode of the corresponding superpixels and their respective neighborhood nodes is relatively stable. Therefore, the embodiment of the present application determines the adjacency matrix of the cross-modal node by combining the position and connection mode of the superpixels in the two modal images. For example, the connection weight vector of a superpixel p and a neighborhood superpixel in the first modal image can characterize the connection mode of the superpixel p in the first modal image, and the adjacency matrix of the superpixel p is constructed by finding the superpixel in the second modal image that is close to the coordinate position of the superpixel p and is relatively similar to the connection mode of the superpixel p in the first modal image (the similarity of the connection mode is higher than the set threshold) in the second modal image. Specifically, the adjacency matrix construction formula is as follows:

[0066]

[0067] Where W k,* represents the connection pattern between the kth node and its neighboring nodes in the first modality image, sim(W k,* ,W j,* ) represents the similarity of the connection pattern between the kth node in the first modality image and the jth node in the second modality image, f cross Represents the mapping relationship between position difference, connection pattern similarity and cross-modal adjacency matrix.

[0068] Based on the above, the embodiment of the present application compares the connection patterns of each superpixel and K neighboring nodes in different modal images, and constructs an adjacency matrix according to the connection patterns between cross-modal nodes and neighboring nodes in different modal images, thereby providing a solution to the grayscale domain and feature domain gap between different modalities.

[0069] S5: Obtain the K neighboring nodes of each cross-modal node in other modalities through the adjacency matrix of the cross-modal node, perform weighted summation on the feature vectors of the K neighboring nodes of each cross-modal node in other modalities, obtain the new features of each cross-modal node in other modalities, and add the new features to the feature values ​​of all pixels in the corresponding superpixel to obtain a multi-modal feature map with unchanged structure but fused with features of other modalities;

[0070] In this step, the new features of the kth superpixel in other modalities at the i-th scale are and the eigenvalue of the pixel p in the kth superpixel in the feature map at the i-th scale The formula for obtaining is as follows:

[0071]

[0072]

[0073] in and denote the feature vectors of the jth neighboring superpixel of the kth superpixel in the current modality image and the other modality images at the i-th scale, respectively, and W k,j and represents the adjacency value of the kth superpixel in the current modality image and the jth neighboring superpixel in the other modality images, respectively, and F i,p Represents the feature vector of pixel p in the kth superpixel at the i-th scale.

[0074] S6: decoding the multi-scale multimodal feature map through a segmentation network, so that the multimodal feature map is upsampled step by step to the size of the first modal image, and the segmentation result of the region of interest in the first modal image is output;

[0075] In this step, the multimodal feature map is decoded step by step through the segmentation network, and the multimodal feature map is returned to the size of the first modal image to obtain the segmentation result of the region of interest of the first modal image. The segmentation network includes several convolutional layers and deconvolutional layers. Long-range jump connections are added to the decoding stage of each scale to combine the feature map of the encoding stage to increase the detailed information of the segmentation result. Finally, when the size of the decoded feature map is consistent with the first modal image, the network parameters are feedback adjusted by calculating the cross entropy loss between it and the gold standard image to obtain the optimal network parameters.

[0076] Based on the above, the medical image segmentation method of the embodiment of the present application divides the images of different modalities into multiple superpixels, and uses a graph neural network to establish the correspondence between each superpixel in the images of different modalities and the neighboring superpixels in the current modality and other modalities, thereby fusing the multimodal information of the images of different modalities, achieving alignment in the spatial domain, solving the problem of information fusion of unregistered multimodal medical images, and improving the overall segmentation speed. The present invention does not require the fine pre-registration of multimodal images, avoids the segmentation error and long time consumption caused by the registration, makes the feature fusion more adaptive, and thus improves the segmentation accuracy of medical images.

[0077] See also Figure 2 , is a schematic diagram of the structure of a medical image segmentation system according to an embodiment of the present application. The medical image segmentation system 40 according to an embodiment of the present application comprises:

[0078] The superpixel extraction module 41 is used to obtain the first modality image and the second modality image, and perform superpixel extraction on the first modality image and the second modality image using a simple linear iterative clustering algorithm;

[0079] The single-modal network construction module 42 is used to construct a first modal graph neural network and a second modal graph neural network according to the superpixels of the first modal image and the second modal image as nodes, respectively. The first modal graph neural network and the second modal graph neural network calculate the adjacency matrix between each superpixel and the neighboring superpixels according to the center position distance and the grayscale mean difference, and calculate the eigenvalue of each superpixel in the current modality according to the adjacency matrix;

[0080] A multimodal network construction module 43 is used to construct a multimodal graph neural network for a cross-modal node according to superpixels of the first modal image and the second modal image, wherein the multimodal graph neural network constructs an adjacency matrix of each cross-modal node according to the position of each cross-modal node in different modal images and a connection mode with K neighboring nodes;

[0081] Feature fusion module 44: used to obtain K neighboring nodes of each cross-modal node in other modalities according to the adjacency matrix of the cross-modal node, and fuse the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixels in the current modality to obtain a multi-modal feature map after feature fusion;

[0082] Image segmentation module 45: used to perform image segmentation on the multimodal feature map through a segmentation network.

[0083] See also Figure 3 , is a schematic diagram of the terminal structure of an embodiment of the present application. The terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0084] The memory 52 stores program instructions for implementing the above-mentioned medical image segmentation method.

[0085] The processor 51 is used to execute program instructions stored in the memory 52 to control medical image segmentation.

[0086] The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having the ability to process signals. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0087] See also Figure 4 , which is a schematic diagram of the structure of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0088] The above description of the disclosed embodiments enables professionals and technicians in the field to implement or use the present application. Various modifications to these embodiments will be apparent to professionals and technicians in the field, and the general principles defined in this application can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown in the present application, but will conform to the widest range consistent with the principles and novel features disclosed in the present application.

Claims

1. A medical image segmentation method, characterized in that: include: Acquire a first modality image and a second modality image, and perform superpixel extraction on the first modality image and the second modality image using a simple linear iterative clustering algorithm; Constructing a first modal graph neural network and a second modal graph neural network according to the superpixels of the first modal image and the second modal image as nodes respectively, wherein the first modal graph neural network and the second modal graph neural network calculate the adjacency matrix between each superpixel and the neighboring superpixels according to the center position distance and the grayscale mean difference, and calculate the eigenvalue of each superpixel in the current modality according to the adjacency matrix; Constructing a multimodal graph neural network for a cross-modal node according to superpixels of the first modal image and the second modal image, wherein the multimodal graph neural network constructs an adjacency matrix of each cross-modal node according to a position of each cross-modal node in different modal images and a connection pattern with K neighboring nodes; Obtaining K neighboring nodes of each cross-modal node in other modalities according to the adjacency matrix of the cross-modal node, fusing the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixels in the current modality, to obtain a multi-modal feature map after feature fusion; The multimodal feature map is subjected to image segmentation through a segmentation network.

2. The medical image segmentation method according to claim 1, characterized in that: The first modality image is a CT image, and the second modality image is an MR image.

3. The medical image segmentation method according to claim 2, characterized in that: The first modal graph neural network and the second modal graph neural network calculate the adjacency matrix between each superpixel and the neighboring superpixel according to the center position distance and the grayscale mean difference as follows: Where W k，j represents the adjacency relationship between the kth superpixel and the jth superpixel in the neighborhood, ΔP k，j and ΔG k，j denote the difference between the center position and grayscale mean of the kth superpixel and the jth superpixel, [·||·] denotes vector connection, and f w Represents the mapping relationship between the position grayscale difference and the adjacency matrix, represents the neighborhood of superpixel node k.

4. The medical image segmentation method according to claim 3, characterized in that: The specific method of calculating the eigenvalue of each superpixel in the current mode according to the adjacency matrix is: Extract the feature vector of each superpixel from the feature map output by the network convolutional layer; Determine the K neighboring nodes of each superpixel in the current mode through the adjacency matrix, perform weighted summation on the feature vectors of the K neighboring nodes, and obtain new features of each superpixel in the current mode; Add the new feature to the feature values ​​of all pixels in the corresponding superpixel to obtain the feature value of each superpixel in the current mode; The feature vector f of the kth superpixel at the i-th scale is i，k The calculation process includes: Among them, S k represents the kth superpixel, F i，p Represents the feature vector of pixel p in the kth superpixel at the i-th scale; use the adjacency matrix W to transform f i，j Weighted, we get the new feature f′ of the kth superpixel at the i-th scale i，k .

5. The medical image segmentation method according to claim 4, characterized in that: The multimodal graph neural network constructs the adjacency matrix of each cross-modal node according to the position of each cross-modal node in different modal images and the connection mode with K neighboring nodes, specifically: Where W k，* represents the connection pattern between the kth node and its neighboring nodes in the first modality image, sim(W k，* , W j，* ) represents the similarity of the connection pattern between the kth node in the first modality image and the jth node in the second modality image, f cross Represents the mapping relationship between position difference, connection pattern similarity and cross-modal adjacency matrix.

6. The medical image segmentation method according to claim 5, characterized in that: The fusion of the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixel in the current modality is specifically as follows: Perform weighted summation on the feature vectors of the K neighboring nodes of each cross-modal node in other modalities to obtain new features of each cross-modal node in other modalities, and add the new features to the feature values ​​of all pixels in the corresponding superpixel to obtain a fused multi-modal feature map; New features of the kth superpixel at the i-th scale in other modalities and the eigenvalue of the pixel p in the kth superpixel in the feature map at the i-th scale The formula for obtaining is: in and denote the feature vectors of the jth neighboring superpixel of the kth superpixel in the current modality image and the other modality images at the i-th scale, respectively, and W k，j and represents the adjacency value of the kth superpixel in the current modality image and the jth neighboring superpixel in the other modality images, respectively, and F i，p Represents the feature vector of pixel p in the kth superpixel at the i-th scale.

7. The medical image segmentation method according to any one of claims 1 to 6, characterized in that: The image segmentation of the multimodal feature map by using a segmentation network is specifically as follows: The segmentation network decodes the multimodal feature map, upsamples the multimodal feature map to the size of the first modality image, and outputs a segmentation result of the region of interest in the first modality image.

8. A medical image segmentation system, characterized in that: include: A superpixel extraction module: used for acquiring a first modality image and a second modality image, and performing superpixel extraction on the first modality image and the second modality image using a simple linear iterative clustering algorithm; A single-modal network construction module: used to construct a first modal graph neural network and a second modal graph neural network according to the superpixels of the first modal image and the second modal image as nodes, respectively. The first modal graph neural network and the second modal graph neural network calculate the adjacency matrix between each superpixel and the neighboring superpixel according to the center position distance and the grayscale mean difference, and calculate the eigenvalue of each superpixel in the current modality according to the adjacency matrix; A multimodal network construction module: used to construct a multimodal graph neural network for a cross-modal node according to superpixels of the first modal image and the second modal image, wherein the multimodal graph neural network constructs an adjacency matrix of each cross-modal node according to the position of each cross-modal node in different modal images and the connection mode with K neighboring nodes; Feature fusion module: used to obtain K neighboring nodes of each cross-modal node in other modalities according to the adjacency matrix of the cross-modal node, and fuse the feature values ​​of the K neighboring nodes of each cross-modal node in other modalities with the feature values ​​of the corresponding superpixels in the current modality to obtain a multi-modal feature map after feature fusion; Image segmentation module: used to perform image segmentation on the multimodal feature map through a segmentation network.

9. A terminal, characterized in that: The terminal includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the medical image segmentation method according to any one of claims 1 to 7; The processor is used to execute the program instructions stored in the memory to control medical image segmentation.

10. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the medical image segmentation method according to any one of claims 1 to 7.

Images