Automatic sam segmentation method, device and equipment for biomedical images

Abstract

The present application relates to the technical field of image segmentation, and particularly relates to an automatic SAM segmentation method, device and equipment for biomedical images. The method comprises the following steps: S1, performing image preprocessing on an original image to obtain a to-be-segmented image; S2, inputting the to-be-segmented image into a pre-trained automatic SAM segmentation model to output a segmentation result image corresponding to the to-be-segmented image. The automatic SAM segmentation model comprises an automatic segmentation module and a segmentation optimization module, wherein the automatic segmentation module comprises an image segmentation encoder, an image segmentation decoder and a ConvNeXt Prompt-Net module, and the segmentation optimization module comprises a display encoder and an optimizer. The present application can realize automatic segmentation of different medical segmentation targets with scattered points, and can effectively segment biomedical images with complex shapes and unclear edges.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to an automatic SAM segmentation method, apparatus, and device for biomedical images. Background Technology

[0002] With the continuous development of high-resolution microscopic optical imaging technology, terabytes (TB) and even petabytes (PB) of biomedical image data are being generated on a massive scale. These biomedical images contain diverse research objects, ranging from macroscopic anatomical structures such as organs and brain functional areas to microscopic fine structures such as neurons and blood vessels. Furthermore, the signal distribution characteristics presented in these images vary greatly due to the influence of imaging parameters and biological tissue sample preparation. How to accurately, efficiently, and automatically identify and segment targets of interest from these biomedical images has become one of the current technical bottlenecks in biomedical research. Traditional image segmentation techniques are often designed for specific types of targets, such as neuronal axons and vascular networks, exhibiting weak generalization ability. Moreover, they typically require developers to possess specialized knowledge of the research field to which the object being segmented belongs, resulting in long development cycles, high manpower consumption, and low return on investment. Therefore, developing a universal target detection and segmentation scheme for different types of biomedical image data has become a current research hotspot. The existing fine-tuned SAM image segmentation model, Poly-SAM, can only be used for specific tasks, such as polyp segmentation, and is not universal; while the improved SAM image segmentation model itself still requires manual calibration prompts in advance for its MedicalSAM Adapter. Summary of the Invention

[0003] To address the issues of existing SAM image segmentation models lacking universality and requiring prior manual calibration, this invention provides an automatic SAM segmentation method, apparatus, and device for biomedical images. This method enables automatic segmentation of medical segmentation targets with various scattered point shapes and can effectively segment biomedical images with complex shapes and unclear edges.

[0004] The automatic SAM segmentation method for biomedical images proposed in this invention specifically includes the following steps:

[0005] S1: Perform image preprocessing on the original image to obtain the image to be segmented;

[0006] S2: Input the image to be segmented into the pre-trained automatic SAM segmentation model and output the segmentation result image corresponding to the image to be segmented;

[0007] The automatic SAM segmentation model includes an automatic segmentation module and a segmentation optimization module. The automatic segmentation module includes an image segmentation encoder, an image segmentation decoder, and a ConvNeXt Prompt-Net module. The image segmentation encoder performs image embedding encoding on the image to be segmented and outputs image features. The ConvNeXt Prompt-Net module extracts implicit features from the image to be segmented and outputs cue embedding encoding. The image segmentation decoder combines the image features and cue embedding encoding to decode the image to be segmented and obtain a first binary image.

[0008] The segmentation optimization module includes a display encoder and an optimizer. The display encoder encodes the first binary image and outputs a feature vector. The optimizer decodes the image features and feature vectors to obtain a second binary image. The second binary image is used as the input of the display encoder to perform image segmentation optimization again until the preset number of optimizations is met, and the final segmentation result image is output.

[0009] Preferably, image preprocessing includes: using MATLAB image processing software to perform downsampling, cropping and stitching operations on the original image, and using Amira software to annotate the target to be segmented in the original image after the operation.

[0010] Preferably, the ConvNeXt Prompt-Net module includes a first 2D convolutional layer, a first normalization layer, a ConNextBlock submodule, a first downsampling module, a second downsampling module, and a second normalization layer, wherein the convolutional kernel of the first 2D convolutional layer is 4*4 and the stride is 4;

[0011] The image to be segmented, with a size of 1024*1024*3, is processed by a first 2D convolutional layer and corresponding normalization operations, resulting in a first feature map of size 256*256*64. This first feature map is then compressed through three cascaded ConNext Block sub-modules and a first downsampling module to extract its feature channels, resulting in a second feature map of size 256*256*64. The second feature map is further compressed through nine cascaded ConNext Block sub-modules and a second downsampling module to extract its feature channels, resulting in a third feature map of size 128*128*128. This third feature map is then compressed through three cascaded ConNext Block sub-modules to extract its feature channels, resulting in a fourth feature map of size 64*64*256. Finally, the fourth feature map is normalized through a second normalization layer, resulting in a cue embedding code of size 64*64*256.

[0012] Preferably, the ConNext Block submodule includes a depthwise separable 2D convolutional layer, a third normalized layer, a second 2D convolutional layer, a GELU activation function, a third 2D convolutional layer, a Layer-Scale layer, and a Drop-Path layer connected in sequence. The depthwise separable 2D convolutional layer has a 7*7 kernel with a stride of 1 and edge padding of 3. The second and third 2D convolutional layers have a 1*1 kernel with a stride of 1.

[0013] The input feature map of the ConNext Block submodule is processed through a depthwise separable 2D convolutional layer and corresponding normalization operations to obtain the fifth feature map. The fifth feature map is then processed through a second 2D convolution and corresponding GELU activation to obtain the sixth feature map. The sixth feature map is then processed through a third 2D convolutional layer and regularized by a Layer-Scale layer and a Drop-Path layer to obtain the seventh feature map. The seventh feature map is added to the input feature map of the ConNext Block submodule to obtain the output feature map of the ConNext Block submodule.

[0014] Preferably, both the first downsampling module and the second downsampling module include a fourth normalization layer and a fourth 2D convolution connected in sequence. The kernel of the fourth 2D convolution is 1*1 and the stride is 1. The input features of the first downsampling module or the second downsampling module are normalized and subjected to corresponding convolution operations to obtain the output features of the first downsampling module or the second downsampling module.

[0015] Preferably, the automatic SAM segmentation model uses BCE loss as the loss function.

[0016] The automatic SAM segmentation device for biomedical images proposed in this invention specifically includes the following steps:

[0017] The image acquisition unit is used to preprocess the original image to obtain the image to be segmented;

[0018] The Automatic SAM segmentation unit is used to input the image to be segmented into the pre-trained Automatic SAM segmentation model and output the segmentation result map corresponding to the image to be segmented.

[0019] The automatic SAM segmentation model includes an automatic segmentation module and a segmentation optimization module. The automatic segmentation module includes an image segmentation encoder, an image segmentation decoder, and a ConvNeXt Prompt-Net module. The image segmentation encoder performs image embedding encoding on the image to be segmented and outputs image features. The ConvNeXt Prompt-Net module extracts implicit features from the image to be segmented and outputs cue embedding encoding. The image segmentation decoder combines the image features and cue embedding encoding to decode the image to be segmented and obtain a first binary image.

[0020] The segmentation optimization module includes a display encoder and an optimizer. The display encoder encodes the first binary image and outputs a feature vector. The optimizer decodes the image features and feature vectors to obtain a second binary image. The second binary image is used as the input of the display encoder to perform image segmentation optimization again until the preset number of optimizations is met, and the final segmentation result image is output.

[0021] The computer device proposed in this invention includes:

[0022] At least one processor; and

[0023] A memory that is communicatively connected to at least one processor; wherein,

[0024] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform an automated SAM segmentation method for biomedical images.

[0025] The present invention proposes a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute an automatic SAM segmentation method for biomedical images.

[0026] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0027] (1) The automatic SAM segmentation method for biomedical images proposed in this invention further designs an auxiliary convolutional network ConvNeXtPrompt-Net based on the ConvNext Block sub-module, which realizes automatic acquisition of prompt embedding codes from the input image, eliminating manual prompts in SAM segmentation, and realizes automatic segmentation of different scattered point-like medical segmentation targets, and can effectively segment biomedical images with complex shapes and unclear edges.

[0028] (2) The automatic SAM segmentation method for biomedical images proposed in this invention, by designing a segmentation optimization module, performs multiple iterative optimizations on the second binarized image, effectively achieving fine segmentation of the segmentation result image. Compared with more advanced biomedical image segmentation methods, such as U-Net and Dense-Net segmentation networks, it has achieved highly competitive results on common metrics such as Dice, Jaccard, and Iou. Attached Figure Description

[0029] Figure 1 This is a structural block diagram of the automatic SAM segmentation model provided according to an embodiment of the present invention;

[0030] Figure 2This is a flowchart illustrating an automatic SAM segmentation method for biomedical images provided according to an embodiment of the present invention.

[0031] Figure 3 This is a network structure diagram of the ConvNeXt Prompt-Net module provided in an embodiment of the present invention;

[0032] Figure 4 This is a network structure diagram of the ConvNext Block submodule provided in an embodiment of the present invention;

[0033] Figure 5 This is a network structure diagram of the first / second downsampling module provided according to an embodiment of the present invention;

[0034] Figure 6 This is a structural block diagram of an automatic SAM segmentation device for biomedical images provided according to an embodiment of the present invention;

[0035] Figure 7 This is a structural block diagram of a computer device provided according to an embodiment of the present invention.

[0036] Reference numerals: 1. Image acquisition unit; 2. Automatic SAM segmentation unit; 3. Computer device; 4. External device; 5. Processing unit; 6. Bus; 7. Network adapter; 8. Input / output (I / O) interface; 9. Display; 10. System memory; 11. Random access memory (RAM); 12. Cache memory; 13. Storage system; 14. Program / utility; 15. Program module. Detailed Implementation

[0037] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0039] Figure 1 A structural block diagram of an automatic SAM segmentation model provided according to an embodiment of the present invention is shown.

[0040] like Figure 1As shown, the image to be segmented is input to the image segmentation encoder and the ConvNeXt Prompt-Net module for parallel processing. The results of the parallel processing are sent to the image segmentation decoder for decoding to obtain the first binary image. The first binary image is encoded by the display encoder and then sent to the optimizer along with the encoding result of the image segmentation encoder for optimization. The obtained second binary image is sent back to the display encoder for iterative processing until the preset number of optimizations is met, and the final segmentation result image is output.

[0041] Figure 2 The flowchart of an automatic SAM segmentation method for biomedical images provided according to an embodiment of the present invention is shown.

[0042] like Figure 2 As shown in the embodiments of the present invention, the automatic SAM segmentation method for biomedical images specifically includes the following steps:

[0043] S1: Perform image preprocessing on the original image to obtain the image to be segmented.

[0044] S2: Input the image to be segmented into the pre-trained automatic SAM segmentation model and output the segmentation result image corresponding to the image to be segmented.

[0045] The automatic SAM segmentation model includes an automatic segmentation module and a segmentation optimization module. The automatic segmentation module includes an image segmentation encoder, an image segmentation decoder, and a ConvNeXt Prompt-Net module. The image segmentation encoder is used to perform image embedding encoding on the image to be segmented and output image features. The ConvNeXt Prompt-Net module is used to extract implicit features of the image to be segmented and output cue embedding encoding. The image segmentation decoder is used to combine image features and cue embedding encoding to decode the image to be segmented and obtain a first binary image.

[0046] The segmentation optimization module includes a display encoder and an optimizer. The display encoder encodes the first binary image and outputs a feature vector. The optimizer decodes the image features and feature vectors to obtain a second binary image. The second binary image is used as the input of the display encoder to perform image segmentation optimization again until the preset number of optimizations is met, and the final segmentation result image is output.

[0047] Figure 3 The network structure of the ConvNeXt Prompt-Net module provided according to an embodiment of the present invention is shown.

[0048] like Figure 3As shown, the ConvNeXt Prompt-Net module includes a first 2D convolutional layer, a first normalization layer, a ConNext Block sub-module, a first downsampling module, a second downsampling module, and a second normalization layer. The first 2D convolutional layer has a 4*4 kernel and a stride of 4.

[0049] The image to be segmented, with a size of 1024*1024*3, is processed by a first 2D convolutional layer and corresponding normalization operations, resulting in a first feature map of size 256*256*64. This first feature map is then compressed through three cascaded ConNext Block sub-modules and a first downsampling module to extract its feature channels, resulting in a second feature map of size 256*256*64. The second feature map is further compressed through nine cascaded ConNext Block sub-modules and a second downsampling module to extract its feature channels, resulting in a third feature map of size 128*128*128. This third feature map is then compressed through three cascaded ConNext Block sub-modules to extract its feature channels, resulting in a fourth feature map of size 64*64*256. Finally, the fourth feature map is normalized through a second normalization layer, resulting in a cue embedding code of size 64*64*256.

[0050] The second feature map has a dimension of 64, the third feature map has a dimension of 128, and the fourth feature map has a dimension of 256.

[0051] Figure 4 The network structure of the ConvNext Block submodule provided according to an embodiment of the present invention is shown.

[0052] like Figure 4 As shown, the ConNext Block submodule includes a depthwise separable 2D convolutional layer, a third normalized layer, a second 2D convolutional layer, a GELU activation function, a third 2D convolutional layer, a Layer-Scale layer, and a Drop-Path layer connected in sequence. The depthwise separable 2D convolutional layer has a 7*7 kernel with a stride of 1 and edge padding of 3. The second and third 2D convolutional layers have a 1*1 kernel with a stride of 1.

[0053] The input feature map of the ConNext Block submodule is processed through a depthwise separable 2D convolutional layer and corresponding normalization operations to obtain the fifth feature map. The fifth feature map is then processed through a second 2D convolution and corresponding GELU activation to obtain the sixth feature map. The sixth feature map is then processed through a third 2D convolutional layer and regularized by a Layer-Scale layer and a Drop-Path layer to obtain the seventh feature map. The seventh feature map is added to the input feature map of the ConNext Block submodule to obtain the output feature map of the ConNext Block submodule.

[0054] The ConvNext Block submodule combines design concepts such as depthwise separable convolution (ResNeXt), inverse bottleneck layer (MobileNetv2), and large convolutional kernel (RepLKNet), demonstrating excellent performance in feature extraction.

[0055] Figure 5 The network structure of the first / second downsampling module provided according to an embodiment of the present invention is shown.

[0056] like Figure 5 As shown, both the first downsampling module and the second downsampling module include a fourth normalization layer and a fourth 2D convolution connected in sequence. The kernel of the fourth 2D convolution is 1*1 and the stride is 1. The input features of the first downsampling module or the second downsampling module are normalized and subjected to corresponding convolution operations to obtain the output features of the first downsampling module or the second downsampling module.

[0057] For ease of understanding, the following embodiments of the present invention use mouse brain data to provide a detailed description of the automatic SAM segmentation method for biomedical images provided by the present invention.

[0058] The mouse brain data underwent image preprocessing, which included downsampling, cropping, and stitching of the original images using MATLAB image processing software, and labeling the targets to be segmented in the original images using Amira software, resulting in Table 1.

[0059] Table 1. Division of the array mouse brain dataset

[0060]

[0061] The three sets of data shown in Table 1 were input into the automatic SAM segmentation model for training. The epoch was set to 50, and the training lasted for 50 epochs. The initial learning rate was set to 0.001, and after every 15 epochs of training, the learning rate was reduced to half of the current learning rate. The automatic SAM segmentation model used BCE loss as the loss function.

[0062] The formula for calculating BCE loss is:

[0063]

[0064] Where L is the loss function, and y represents the true value of the sample in the training set. This represents the predicted value of a sample in the training set.

[0065] According to equation (1), the backpropagation calculation process can be roughly expressed as follows (backpropagation calculates the gradient of the loss function with respect to each parameter using the chain rule of derivatives, and updates the parameters based on the gradient):

[0066]

[0067] Among them, W 1,1 σ1 represents the parameters of the automatic SAM segmentation model, σ1 is the slope obtained by differentiating the parameters of the automatic SAM segmentation model, and a1 is a constant.

[0068] The parameters W of the automatic SAM segmentation model are updated using the gradient descent method. 1,1 :

[0069] W 1,1 ′=W 1,1 -ασ1 (3);

[0070] Where α represents the learning rate, W 1,1 ′ represents the parameters of the updated automatic SAM segmentation model.

[0071] Print once per episode and save the model with the smallest loss function. Training can be completed in 50 epochs.

[0072] To compare the performance of existing SAM segmentation models with the automatic SAM segmentation model proposed in this embodiment of the invention in biomedical image segmentation, we used the three sets of data shown in Table 1 as input prompts to the original SAM model to test its segmentation performance. Simultaneously, this embodiment of the invention designed an ablation experiment, i.e., removing the segmentation optimization module and retaining only the automatic segmentation module, to test the image segmentation performance of the automatic SAM segmentation model proposed in this embodiment compared to existing SAM segmentation models.

[0073] During the experiment, similar training strategies and the same training data were used to train each SAM segmentation model. Finally, the results were evaluated by calculating Precision, Recall, Dice, and Jaccard.

[0074] Precision, also known as accuracy, represents the proportion of samples that are predicted to be positive but are actually positive. The formula is as follows:

[0075]

[0076] Recall, also known as the full detection rate, represents the proportion of actual positive samples out of the total number of positive samples predicted to be positive. The formula is as follows:

[0077]

[0078] In this context, TP, FP, and FN are concepts within the Confusion Matrix. TP (TruePositive): Correctly predicted positive examples. FP (False Positive): Correctly predicted positive examples. FN (FalseNegative): Correctly predicted negative examples.

[0079] The Dice coefficient is used to calculate the similarity between two sets, and the formula is as follows:

[0080]

[0081] Where |x| represents the number of elements in set x, |y| represents the number of elements in set y, and intersection(x,y) represents the intersection of the two sets.

[0082] The Jaccard coefficient only concerns whether the common features shared by individuals are consistent. It is generally used to detect the similarity and differences between finite sample sets. The formula is as follows:

[0083]

[0084] Where intersection(x,y) represents the intersection of x and y, and union(x,y) represents the union of x and y. The larger the Dice and Jaccard coefficients are, the higher the similarity between the two samples.

[0085] The detailed results of the comparative and ablation experiments are shown in Tables 2 and 3.

[0086] Table 2 Comparison of experimental results

[0087] U-Net MICCAI (2015) 0.8274 0.9056 0.9985 0.8285 Res-UNet IEEE (2018) 0.7637 0.8660 0.9798 0.7759 U-Net++ DLMIA (2018) 0.8563 0.9226 0.9909 0.8631 nnUNet CVPR (2020) 0.7863 0.8804 0.9895 0.7929 DeepLab v3+ ECCV (2018) 0.7321 0.8454 0.9656 0.7517 IBCP-SAM - 0.9550 0.9770 0.9965 0.9582

[0088] IBCP-SAM corresponds to the automatic SAM segmentation model proposed in this embodiment of the invention. As can be seen from the comparative experiments in Table 2, IBCP-SAM's performance is significantly superior to methods such as U-Net++ and DeepLab v3+.

[0089] Table 3 Ablation Experiment Results

[0090]

[0091]

[0092] In Table 3, the SAM (GT mask as prompt) column refers to the segmentation results obtained after inputting the three sets of data shown in Table 1 as prompts into the original SAM segmentation model. This can be used to demonstrate the segmentation performance of the original SAM segmentation model on biomedical images. The SAM-ConvNext structure is as follows: Figure 3 As shown, this is equivalent to removing the segmentation optimization module and retaining only the automatic segmentation module; IBCP-SAM(Optimizer×n) means using the segmentation optimization module for n iterations.

[0093] The results in Table 3 show:

[0094] 1) This invention has significant advantages over the original SAM segmentation model in the segmentation of biomedical images;

[0095] 2) The two-segment structure proposed in this embodiment of the invention can improve segmentation accuracy;

[0096] 3) The cyclic segmentation strategy proposed in this embodiment of the invention can further refine the segmentation target.

[0097] Figure 6 The structure of an automated SAM segmentation apparatus for biomedical images provided according to an embodiment of the present invention is shown.

[0098] like Figure 6 As shown, the automatic SAM segmentation device for biomedical images proposed in this invention specifically includes the following steps:

[0099] Image acquisition unit 1 is used to perform image preprocessing on the original image to obtain the image to be segmented;

[0100] Automatic SAM segmentation unit 2 is used to input the image to be segmented into the pre-trained automatic SAM segmentation model and output the segmentation result map corresponding to the image to be segmented;

[0101] The automatic SAM segmentation model includes an automatic segmentation module and a segmentation optimization module. The automatic segmentation module includes an image segmentation encoder, an image segmentation decoder, and a ConvNeXt Prompt-Net module. The image segmentation encoder performs image embedding encoding on the image to be segmented and outputs image features. The ConvNeXt Prompt-Net module extracts implicit features from the image to be segmented and outputs cue embedding encoding. The image segmentation decoder combines the image features and cue embedding encoding to decode the image to be segmented and obtain a first binary image.

[0102] The segmentation optimization module includes a display encoder and an optimizer. The display encoder encodes the first binary image and outputs a feature vector. The optimizer decodes the image features and feature vectors to obtain a second binary image. The second binary image is used as the input of the display encoder to perform image segmentation optimization again until the preset number of optimizations is met, and the final segmentation result image is output.

[0103] Accordingly, according to embodiments of the present invention, the present invention also provides a computer device, a readable storage medium, and a computer program product.

[0104] Figure 6 This is a schematic diagram of the structure of a computer device 3 provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 3 suitable for implementing embodiments of the present invention is shown. Figure 6 The computer device 3 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0105] like Figure 6 As shown, computer device 3 is represented in the form of a general-purpose computing device. Computer device 3 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0106] The components of the computer device 3 may include, but are not limited to: one or more processors or processing units 5, system memory 10, and bus 6 connecting different system components (including system memory 10 and processing unit 5).

[0107] Bus 6 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0108] Computer device 3 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 3, including volatile and non-volatile media, removable and non-removable media.

[0109] System memory 10 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 11 and / or cache memory 12. Computer device 3 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 13 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 6 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0110] A program / utility 14 having a set (at least one) of program modules 15 may be stored, for example, in memory 28. Such program modules 15 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 15 typically perform the functions and / or methods described in the embodiments of the present invention.

[0111] Computer device 3 can also communicate with one or more external devices 4 (e.g., keyboard, pointing device, display 9, etc.), and with one or more devices that enable a user to interact with computer device 3, and / or with any device that enables computer device 3 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 8. Furthermore, computer device 3 can also communicate with one or more networks (e.g., local area network (LA6), wide area network (WA6), and / or public networks, such as the Internet) via network adapter 7. As shown, network adapter 7 communicates with other modules of computer device 3 via bus 6. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computer device 3, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0112] The processing unit 5 executes various functional applications and data processing by running programs stored in the system memory 10, such as implementing the target tracking method based on a broad spectrum correlation fusion network provided in the embodiments of the present invention.

[0113] This invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored, wherein the program, when executed by a processor, is the target tracking method based on a broad-spectrum correlation fusion network provided in all embodiments of this application.

[0114] The computer storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0115] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0116] The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. The computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LA6) or a wide area network (WA6), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the target tracking method based on a broad-spectrum correlation fusion network as described above.

[0118] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0119] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0120] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automatic SAM segmentation method for biomedical images, characterized in that, Specifically, the steps include the following: S1: Perform image preprocessing on the original image to obtain the image to be segmented; S2: Input the image to be segmented into a pre-trained automatic SAM segmentation model and output the segmentation result image corresponding to the image to be segmented; The automatic SAM segmentation model includes an automatic segmentation module and a segmentation optimization module. The automatic segmentation module includes an image segmentation encoder, an image segmentation decoder, and a ConvNeXt Prompt-Net module. The image segmentation encoder performs image embedding encoding on the image to be segmented and outputs image features. The ConvNeXt Prompt-Net module extracts implicit features from the image to be segmented and outputs cue embedding encoding. The image segmentation decoder combines the image features and the cue embedding encoding to decode the image to be segmented, obtaining a first binary image. The ConvNeXt Prompt-Net module includes a first 2D convolutional layer, a first normalization layer, a ConNext Block submodule, a first downsampling module, a second downsampling module, and a second normalization layer. The first 2D convolutional layer has a 4-kernel designation. 4, step size is 4; Size is 1024 1024 The image to be segmented, as described in section 3, after passing through the first 2D convolutional layer and corresponding normalization operations, outputs a size of 256. 256 The first feature map of size 64, after being compressed through three cascaded ConNext Block sub-modules and the first downsampling module for feature extraction channels, has an output size of 256. 256 The second feature map of size 64, after being compressed through nine cascaded ConNext Block sub-modules and the second downsampling module for feature extraction channels, outputs a size of 128. 128 The third feature map of 128, after being compressed through three cascaded ConNext Block sub-modules for feature extraction channels, has an output size of 64. 64 The fourth feature map of size 256, after being normalized by the second normalization layer, has an output size of 64. 64 256 hints are embedded in the encoding; The segmentation optimization module includes a display encoder and an optimizer. The display encoder is used to encode the first binary image and output a feature vector. The optimizer is used to decode the image features and the feature vector to obtain a second binary image. The second binary image is used as the input of the display encoder to perform image segmentation optimization again until a preset number of optimizations is met, and the final segmentation result image is output.

2. The automatic SAM segmentation method for biomedical images according to claim 1, characterized in that, The image preprocessing includes: using MATLAB image processing software to perform downsampling, cropping and stitching operations on the original image, and using Amira software to annotate the target to be segmented in the original image after the operation.

3. The automatic SAM segmentation method for biomedical images according to claim 1, characterized in that, The ConNext Block submodule comprises, in sequence, a depthwise separable 2D convolutional layer, a third normalized layer, a second 2D convolution, a GELU activation function, a third 2D convolution, a Layer-Scale layer, and a Drop-Path layer. The convolutional kernel of the depthwise separable 2D convolutional layer is 7.

7. The stride is 1, and the edge padding is 3. The kernels of the second and third 2D convolutions are both 1. 1, with a step size of 1, where, The input feature map of the ConNext Block submodule is processed by the depthwise separable 2D convolutional layer and the corresponding normalization operation to obtain the fifth feature map. The fifth feature map is then processed by the second 2D convolution and the corresponding GELU activation to obtain the sixth feature map. The sixth feature map is then processed by the third 2D convolution operation and the regularization operation of the Layer-Scale layer and the Drop-Path layer to obtain the seventh feature map. The seventh feature map is added to the input feature map of the ConNext Block submodule to obtain the output feature map of the ConNext Block submodule.

4. The automatic SAM segmentation method for biomedical images according to claim 1, characterized in that, Both the first downsampling module and the second downsampling module include a fourth normalization layer and a fourth 2D convolution layer connected in sequence, wherein the kernel of the fourth 2D convolution layer is 1.

1. The step size is 1, wherein the input features of the first downsampling module or the second downsampling module are normalized and subjected to corresponding convolution operations to obtain the output features of the first downsampling module or the second downsampling module.

5. The automatic SAM segmentation method for biomedical images according to claim 1, characterized in that, The automatic SAM segmentation model uses BCE loss as the loss function.

6. An automated SAM segmentation device for biomedical images, characterized in that, Specifically, the steps include the following: The image acquisition unit is used to preprocess the original image to obtain the image to be segmented; The automatic SAM segmentation unit is used to input the image to be segmented into a pre-trained automatic SAM segmentation model and output the segmentation result image corresponding to the image to be segmented. The automatic SAM segmentation model includes an automatic segmentation module and a segmentation optimization module. The automatic segmentation module includes an image segmentation encoder, an image segmentation decoder, and a ConvNeXt Prompt-Net module. The image segmentation encoder performs image embedding encoding on the image to be segmented and outputs image features. The ConvNeXt Prompt-Net module extracts implicit features from the image to be segmented and outputs cue embedding encoding. The image segmentation decoder combines the image features and the cue embedding encoding to decode the image to be segmented, obtaining a first binary image. The ConvNeXt Prompt-Net module includes a first 2D convolutional layer, a first normalization layer, a ConNext Block submodule, a first downsampling module, a second downsampling module, and a second normalization layer. The first 2D convolutional layer has a 4-kernel designation. 4, step size is 4; Size is 1024 1024 The image to be segmented, as described in section 3, after passing through the first 2D convolutional layer and corresponding normalization operations, outputs a size of 256. 256 The first feature map of size 64, after being compressed through three cascaded ConNext Block sub-modules and the first downsampling module for feature extraction channels, has an output size of 256. 256 The second feature map of size 64, after being compressed through nine cascaded ConNext Block sub-modules and the second downsampling module for feature extraction channels, outputs a size of 128. 128 The third feature map of 128, after being compressed through three cascaded ConNext Block sub-modules for feature extraction channels, has an output size of 64. 64 The fourth feature map of size 256, after being normalized by the second normalization layer, has an output size of 64. 64 256 hints are embedded in the encoding; The segmentation optimization module includes a display encoder and an optimizer. The display encoder is used to encode the first binary image and output a feature vector. The optimizer is used to decode the image features and the feature vector to obtain a second binary image. The second binary image is used as the input of the display encoder to perform image segmentation optimization again until a preset number of optimizations is met, and the final segmentation result image is output.

7. A computer device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the automatic SAM segmentation method for biomedical images as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the automatic SAM segmentation method for biomedical images as described in any one of claims 1 to 5.

Images