Medical image segmentation method and device, computer device and storage medium

Abstract

The application relates to the technical field of medical image segmentation, and discloses a medical image segmentation method and device, computer equipment and a storage medium.The method comprises the following steps: performing medical image segmentation based on a T1 weighted image, an anisotropy fraction image and a trained sam large model to obtain a segmentation image, wherein the transformer architecture of the image encoder of the sam large model comprises a multi-modal adapter, the multi-modal adapter comprises a first encoder, a second encoder, a convolution layer, an attention mechanism layer, a deformation conversion layer, a feature splicing layer, an activation function layer and a third decoder.The multi-modal adapter can realize multi-modal data fusion, and the sam large model can realize efficient and accurate image segmentation of the T1 weighted image and the anisotropy fraction image.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a medical image segmentation method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Segmentation of the retinal original visual pathway (RGVP) is crucial for studying and understanding disease progression and developing diagnostic plans. However, the RGVP has a thin and elongated structure, making accurate identification and differentiation of adjacent visual pathways in MRI images a challenging task.

[0003] With the rapid advancement of deep learning technology, deep learning-based visual pathway segmentation methods are gradually being used. Existing visual pathway medical image segmentation methods mainly use small models, which need to be trained from scratch, and the models have weak generalization ability. Furthermore, current larger medical image segmentation models can only perform single-modal segmentation tasks and cannot segment multimodal images. Summary of the Invention

[0004] Therefore, it is necessary to address the technical problem of poor medical image segmentation performance in existing technologies by proposing a medical image segmentation method, device, computer equipment, and storage medium.

[0005] In a first aspect, a medical image segmentation method is provided, the method comprising:

[0006] Obtain the T1-weighted image and the anisotropic score image;

[0007] Medical image segmentation is performed based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder.

[0008] Secondly, a medical image segmentation apparatus is provided, the apparatus comprising:

[0009] The acquisition module is used to acquire the T1 weighted image and the anisotropic score image;

[0010] A medical image segmentation module is used to segment medical images based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain segmented images. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder.

[0011] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described medical image segmentation method.

[0012] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described medical image segmentation method.

[0013] The medical image segmentation method proposed in this invention obtains a T1-weighted image and anisotropic fractional images, and then performs medical image segmentation based on the T1-weighted image, the anisotropic fractional images, and a trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. It can achieve multimodal data fusion through multimodal adaptors and realize efficient and accurate image segmentation of T1 weighted images and anisotropic fractional images by using the SAM large model. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] in:

[0016] Figure 1 This is an application environment diagram of a medical image segmentation method in one embodiment;

[0017] Figure 2 Here is a flowchart of a medical image segmentation method in one embodiment;

[0018] Figure 3 This is a schematic diagram of the transformer architecture of a medical image segmentation method in one embodiment.

[0019] Figure 4 This is a structural block diagram of a medical image segmentation device in one embodiment;

[0020] Figure 5 This is a structural block diagram of a computer device in one embodiment;

[0021] Figure 6 This is a structural block diagram of a computer device in another embodiment. Detailed Implementation

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0023] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The medical image segmentation method provided in this invention can be applied to, for example... Figure 1In the application environment, the client 110 communicates with the server 120 via the network. Server 120 can receive T1-weighted images and anisotropic fractional images from client 110. Then, server 120 performs medical image segmentation based on the T1-weighted images, the anisotropic fractional images, and the trained SAM large model to obtain segmented images. The transformer architecture of the image encoder of the SAM large model includes a multimodal adaptor, which includes a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. This invention enables multimodal data fusion through a multimodal adaptor and achieves efficient and accurate image segmentation of T1-weighted images and anisotropic fractional images using a large SAM model. The client 110 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0026] Please see Figure 2 As shown, Figure 2 A flowchart illustrating a medical image segmentation method according to an embodiment of the present invention includes the following steps:

[0027] Step S101: Obtain the T1 weighted image and the anisotropic score image;

[0028] Step S102: Perform medical image segmentation based on the T1 weighted image, the anisotropic fractional image, and the trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder.

[0029] Among them, the SAM large model is a large model for medical image segmentation.

[0030] Specifically, such as Figure 3 As shown, Figure 3 Taking the transformer architecture of the image encoder in the SAM large model as an example, a multimodal adapter (Multimodal Adapter) was designed to be inserted into the SAM large model. The T1-weighted image is input into the SAM large model, and the FA modality data is input into the encoder of the multimodal adapter. Here, represents concatenation.

[0031] It should be understood that the multimodal adaptor in this embodiment can also be applied to other large models besides the SAM large model.

[0032] In one embodiment, the transformer architecture is used to perform layer-by-layer normalization on the T1-weighted image to obtain a first feature. The first feature is then processed using a multi-head attention mechanism to obtain a second feature. This second feature is input into the first encoder of the multimodal adaptor, and the anisotropic fractional image is input into the second encoder of the multimodal adaptor to obtain a third feature output by the multimodal adaptor. This third feature is then subjected to layer-by-layer normalization to obtain a fourth feature. The first and fourth features are then superimposed to obtain a fifth feature. This fifth feature is further normalized to obtain a sixth feature. The sixth feature is input into the multilayer perceptron in the transformer architecture to obtain a seventh feature output by the multilayer perceptron. Finally, the seventh feature is superimposed with the fifth feature to obtain an eighth feature. The above describes the processing of the image encoder in the SAM large model using the transformer architecture. The eighth feature is used by the SAM large model for subsequent medical image segmentation.

[0033] In one embodiment, before the steps of obtaining the T1-weighted image and the anisotropic fractional image, the method further includes:

[0034] Step S201: Obtain the multimodal dataset of the visual pathway and the initial SAM large model. The image encoder transformer architecture of the initial SAM large model includes a multimodal adaptor. The multimodal adaptor includes a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the attention mechanism layer. The output of the attention mechanism layer is connected to the input of the deformation transformation layer. The output of the deformation transformation layer is connected to one input of the feature concatenation layer. The output of the first encoder is connected to the other input of the feature concatenation layer. The output of the feature concatenation layer is connected to the input of the activation function layer. The output of the activation function layer is connected to the input of the third decoder.

[0035] Step S202: Freeze all parameters in the initial SAM large model except for the multimodal adaptor, fine-tune the initial SAM large model using the multimodal dataset of the visual pathway, and use the fine-tuned initial SAM large model as the trained SAM large model.

[0036] The medical image segmentation method proposed in this embodiment obtains a T1-weighted image and anisotropic fractional images, and then performs medical image segmentation based on the T1-weighted image, the anisotropic fractional images, and a trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. It can achieve multimodal data fusion through multimodal adaptors and realize efficient and accurate image segmentation of T1 weighted images and anisotropic fractional images by using the SAM large model.

[0037] Please see Figure 4 As shown, in one embodiment, a medical image segmentation apparatus is provided, the apparatus comprising: an acquisition module 10, used to acquire a T1-weighted image and an anisotropic fractional image;

[0038] The medical image segmentation module 20 is used to segment medical images based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain segmented images. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder.

[0039] In one embodiment, the medical image segmentation apparatus is further configured to: use a transformer architecture to perform layer-by-layer normalization processing on the T1-weighted image to obtain a first feature; perform multi-head attention processing on the first feature to obtain a second feature; input the second feature into the first encoder of the multimodal adaptor and input the anisotropic fractional image into the second encoder of the multimodal adaptor to obtain a third feature output by the multimodal adaptor; perform layer-by-layer normalization processing on the third feature to obtain a fourth feature; perform feature superposition on the first feature and the fourth feature to obtain a fifth feature; perform layer-by-layer normalization processing on the fifth feature to obtain a sixth feature; input the sixth feature into the multilayer perceptron in the transformer architecture to obtain a seventh feature output by the multilayer perceptron; and perform feature superposition on the seventh feature and the fifth feature to obtain an eighth feature.

[0040] In one embodiment, the medical image segmentation device is further configured to: acquire a multimodal dataset of the visual pathway and an initial SAM large model, wherein the image encoder of the initial SAM large model includes a multimodal adaptor in its transformer architecture, the multimodal adaptor including a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder; the output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder;

[0041] All parameters in the initial SAM large model, except for the multimodal adaptor, are frozen. The initial SAM large model is then fine-tuned using a multimodal dataset from the visual pathway. The fine-tuned initial SAM large model is then used as the trained SAM large model.

[0042] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a medical image segmentation method on the server side.

[0043] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a medical image segmentation method on the client side.

[0044] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the following steps:

[0045] Obtain the T1-weighted image and the anisotropic score image;

[0046] Medical image segmentation is performed based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder.

[0047] The medical image segmentation method proposed in this embodiment obtains a T1-weighted image and anisotropic fractional images, and then performs medical image segmentation based on the T1-weighted image, the anisotropic fractional images, and a trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. It can achieve multimodal data fusion through multimodal adaptors and realize efficient and accurate image segmentation of T1 weighted images and anisotropic fractional images by using the SAM large model.

[0048] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps:

[0049] Obtain the T1-weighted image and the anisotropic score image;

[0050] Medical image segmentation is performed based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder.

[0051] The medical image segmentation method proposed in this embodiment obtains a T1-weighted image and anisotropic fractional images, and then performs medical image segmentation based on the T1-weighted image, the anisotropic fractional images, and a trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. It can achieve multimodal data fusion through multimodal adaptors and realize efficient and accurate image segmentation of T1 weighted images and anisotropic fractional images by using the SAM large model.

[0052] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0053] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAM bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0054] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0055] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A medical image segmentation method, characterized in that, The medical image segmentation method includes: Obtain the T1-weighted image and the anisotropic score image; Medical image segmentation is performed based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain a segmented image. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. The transformer architecture is used to perform layer-by-layer normalization on the T1-weighted image to obtain a first feature. The first feature is then processed using a multi-head attention mechanism to obtain a second feature. The second feature is input into the first encoder of the multimodal adaptor, and the anisotropic fractional image is input into the second encoder of the multimodal adaptor to obtain a third feature output by the multimodal adaptor. The third feature is then subjected to layer-by-layer normalization to obtain a fourth feature. The first feature and the fourth feature are then superimposed to obtain a fifth feature. The fifth feature is then subjected to layer-by-layer normalization to obtain a sixth feature. The sixth feature is input into the multilayer perceptron in the transformer architecture to obtain a seventh feature output by the multilayer perceptron. The seventh feature is then superimposed with the fifth feature to obtain an eighth feature.

2. The medical image segmentation method according to claim 1, characterized in that, Before the steps of obtaining the T1-weighted image and the anisotropic fraction image, the method further includes: A multimodal dataset of the visual pathway and an initial SAM large model are obtained. The transformer architecture of the image encoder of the initial SAM large model includes a multimodal adaptor. The multimodal adaptor includes a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the attention mechanism layer. The output of the attention mechanism layer is connected to the input of the deformation transformation layer. The output of the deformation transformation layer is connected to one input of the feature concatenation layer. The output of the first encoder is connected to the other input of the feature concatenation layer. The output of the feature concatenation layer is connected to the input of the activation function layer. The output of the activation function layer is connected to the input of the third decoder. All parameters in the initial SAM large model, except for the multimodal adaptor, are frozen. The initial SAM large model is then fine-tuned using a multimodal dataset from the visual pathway. The fine-tuned initial SAM large model is then used as the trained SAM large model.

3. A medical image segmentation device, characterized in that, The medical image segmentation device includes: The acquisition module is used to acquire the T1 weighted image and the anisotropic score image; A medical image segmentation module is used to segment medical images based on the T1-weighted image, the anisotropic fractional image, and the trained SAM large model to obtain segmented images. The SAM large model's image encoder transformer architecture includes a multimodal adaptor, comprising a first encoder, a second encoder, a convolutional layer, an attention mechanism layer, a deformation transformation layer, a feature concatenation layer, an activation function layer, and a third decoder. The output of the second encoder is connected to the input of the convolutional layer, the output of the convolutional layer is connected to the input of the attention mechanism layer, the output of the attention mechanism layer is connected to the input of the deformation transformation layer, the output of the deformation transformation layer is connected to one input of the feature concatenation layer, the output of the first encoder is connected to the other input of the feature concatenation layer, the output of the feature concatenation layer is connected to the input of the activation function layer, and the output of the activation function layer is connected to the input of the third decoder. The transformer architecture is used to perform layer-by-layer normalization on the T1-weighted image to obtain a first feature. The first feature is then processed using a multi-head attention mechanism to obtain a second feature. The second feature is input into the first encoder of the multimodal adaptor, and the anisotropic fractional image is input into the second encoder of the multimodal adaptor to obtain a third feature output by the multimodal adaptor. The third feature is then subjected to layer-by-layer normalization to obtain a fourth feature. The first feature and the fourth feature are then superimposed to obtain a fifth feature. The fifth feature is then subjected to layer-by-layer normalization to obtain a sixth feature. The sixth feature is input into the multilayer perceptron in the transformer architecture to obtain a seventh feature output by the multilayer perceptron. The seventh feature is then superimposed with the fifth feature to obtain an eighth feature.

4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the medical image segmentation method as described in any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the medical image segmentation method as described in any one of claims 1 to 2.

Images