Cancer image processing method, system, and medium based on mamba structure

Abstract

The application relates to the technical field of image processing, in particular to a cancer image processing method and system based on a Mamba structure and a medium, the method comprising the following steps: marking a lesion area and grading in an MRI image; performing pretreatment and image expansion processing on the MRI image to form a sample data set containing multiple image modalities; extracting multi-scale features of each image modality through an improved neural network model; feeding the multi-scale features through each Mamba block and a corresponding down-sampling layer to obtain modality output features; performing multi-task learning through a multi-task learning module to obtain a multi-task learning loss; updating parameters of the improved neural network model according to the multi-task learning loss to obtain a trained model; inputting newly collected MRI images into the trained model to obtain a lesion positioning result of a cancer image and a classification result of the cancer image; and the application can improve the detection and classification performance and efficiency of the cancer image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a cancer image processing method and system based on Mamba structure and a medium. BACKGROUND

[0002] In the related art, the diagnosis of cancer mainly relies on the judgment of doctors, but the diagnosis of radiologists is subjective and is limited by the experience and knowledge level of doctors. Therefore, how to realize the automatic and interactive diagnosis of cancer by combining medicine and artificial intelligence has become a major scientific problem currently faced. In order to meet the increasingly complex requirements of medical image processing, how to apply deep learning algorithm to intelligent analysis of cancer images to improve the accuracy and efficiency of auxiliary diagnosis of cancer has become a technical problem to be solved. SUMMARY

[0003] The present application aims to provide a cancer image processing method and system based on Mamba structure, which can improve the detection and classification performance and efficiency of cancer images.

[0004] In order to achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0005] In a first aspect, the present application provides a cancer image processing method based on Mamba structure, which comprises the following steps:

[0006] S100, acquiring an MRI image of prostate cancer, and marking the lesion area and grade in the MRI image;

[0007] S200, pre-processing the MRI image, performing image expansion processing on the pre-processed MRI image to form a sample data set containing multiple image modalities, and dividing the sample data set into a training set and a test set; the sample data set contains multiple cancer image samples, and each cancer image sample has a corresponding image modality;

[0008] S300, constructing an improved neural network model, inputting the cancer image samples in the training set into the improved neural network model, and extracting multi-scale features of each image modality; wherein the improved neural network model contains multiple multi-modal image feature extraction networks, the number of the multi-modal image feature extraction networks is consistent with the number of image modalities; the multi-modal image feature extraction network contains an improved V-Net network combined with SSM, and the improved V-Net network contains a CNN-SSM block combined with a convolutional neural network and a state space model;

[0009] S400, feeding the multi-scale features extracted by each Mamba block and the corresponding down-sampling layer to the multi-modal image feature extraction network to obtain modality output features of the encoder;

[0010] S500, constructing a multi-task learning module, inputting the multi-modal output features of the encoder into the multi-task learning module for multi-task learning to obtain a multi-task learning loss; wherein the multi-task learning module comprises a lesion detection module and a lesion grading module, the lesion detection module is used for positioning the lesion in the image, and the lesion grading module is used for classifying the lesion in the image;

[0011] S600, inputting the cancer image samples in the test set into the improved neural network model, calculating the multi-task learning loss of the improved neural network model, updating the parameters of the improved neural network model according to the multi-task learning loss until convergence, and obtaining a trained model;

[0012] S700, inputting the newly collected MRI image into the trained model to obtain the lesion positioning result of the cancer image and the classification result of the cancer image.

[0013] Preferably, in S300, the inputting of the cancer image samples in the training set into the improved neural network model to extract multi-scale features of each image modality comprises:

[0014] S310, inputting the cancer image samples of each image modality into the improved V-Net model in each multi-modal image feature extraction network for MxMxM convolution operation one by one to obtain a first intermediate feature map; wherein the improved V-Net model has a channel number N1 and a step K, and is followed by a ReLU function with a decay rate of 0.85;

[0015] S320, inputting the first intermediate feature map into the convolution block in the improved V-Net network for convolution operation to obtain a second intermediate feature map; wherein the convolution block comprises 2 MxMxM convolution operations, the channel number is N1, and each convolution operation is followed by a ReLU function operation with a decay rate of 0.85;

[0016] S330, inputting the second intermediate feature map into the max-pooling layer in the improved V-Net network for down-sampling operation to obtain a third intermediate feature map; wherein the step of the max-pooling layer is K, and the feature size of the third intermediate feature map is 1 / 2 of the second intermediate feature map;

[0017] S340, cyclically executing S320 and S330 until the size of the third intermediate feature map is less than a set value P, recording the cycle number L and the channel number N of the third intermediate feature map L+1 ;

[0018] S350, input the third intermediate feature map into a residual block in the improved V-Net network to obtain an output feature map; wherein the residual block includes two MxMxM convolution operations, and the number of channels is N L+1 , and each convolution operation is followed by a ReLU function operation with a decay rate of 0.85;

[0019] S360, form the output feature maps corresponding to each image modality into multi-scale features extracted by the improved neural network model.

[0020] Preferably, in S400, the multi-scale features extracted by feeding the multi-modal image feature extraction network through each Mamba block and the corresponding down-sampling layer are obtained. The modality output features of the encoder include:

[0021] S410, form the output feature maps corresponding to each image modality into multi-modal features corresponding to each image modality; Figure One one of the input multiple Mamba blocks, extract the 3D features of the output feature map;

[0022] S420, for the output feature maps corresponding to each image modality, adopt a flattening operation to reshape the 3D features of the output feature map into a 1D long sequence to obtain multi-modal features corresponding to each image modality;

[0023] S430, perform feature fusion processing on the multi-modal features corresponding to each image modality to obtain fused features;

[0024] S440, input the fused features into the Mamba module to obtain the multi-modal output features of the encoder:

[0025] Preferably, in S500, the multi-task learning module is constructed, the multi-modal output features of the encoder are input into the multi-task learning module for multi-task learning, and a multi-task learning loss is obtained, including:

[0026] S510, input the multi-modal output features of the encoder into a lesion detection module and a lesion grading module respectively for multi-task learning, and the multi-task learning includes a lesion positioning task and a lesion grading task;

[0027] S520, for each cancer lesion positioning, input the multi-modal output features into an attention module for lesion positioning processing to obtain output features of the attention module;

[0028] S530, determine a cross-entropy loss of cancer lesion grading based on the multi-modal output features and the output features of the attention module;

[0029] S540, determining a loss of each classification task based on the grading probability output by the lesion grading module in each image modality;

[0030] S550, determining a multi-task learning loss based on the cross-entropy loss of the cancer lesion grading and the loss of each classification task.

[0031] Preferably, the output feature of the attention module is:

[0032]

[0033] wherein ψ(·) represents a convolution operation with a step size of 1 and a convolution kernel of 1, σ(·) is used for activation, χ spatical represents spatial attention, and α dec represents channel attention, represents the output feature of the attention module.

[0034] Preferably, the cross-entropy loss of the cancer lesion grading is:

[0035]

[0036] wherein y img represents a multi-modal output feature, and L dec represents the cross-entropy loss of the cancer lesion grading.

[0037] Preferably, the loss of each classification task is represented as:

[0038]

[0039] wherein K represents the number of data sets, g k represents a true value label of grading, and p k represents a probability of grading output by the lesion grading module, and n represents the number of classification tasks, represents the loss of the nth classification task.

[0040] Preferably, the expression of the multi-task learning loss is:

[0041]

[0042] wherein α dec represents an uncertain weight, and α1...α n represent the weights of n classification tasks and are obtained through network learning; and L represents the multi-task learning loss.

[0043] In a second aspect, an embodiment of the present application provides a cancer image processing system based on a Mamba structure, the system comprising:

[0044] at least one processor;

[0045] at least one memory for storing the at least one program;

[0046] When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of the above.

[0047] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a processor executable program, and the processor executable program is used for executing the method of any one of the above when executed by a processor.

[0048] The present application has the following beneficial effects: The present application introduces a multi-modal deep multi-task self-attention network to solve the problems of multi-modal information fusion and multi-target classification in the cancer diagnosis process, interactively fuses multi-modal information, constructs multi-task correlation features, and improves the robustness and accuracy of the cancer lesion classification module. In addition, in order to solve the problem of low running efficiency of the multi-modal long sequence data model, a deep network model based on the Mamba structure is constructed, a state space model is introduced, the algorithm running time complexity is reduced, and the detection and classification performance and efficiency of the cancer image are improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 is a flowchart of the cancer image processing method based on the Mamba structure in the embodiments of the present application;

[0051] Figure 2 is a flowchart of the cancer image processing method based on the Mamba structure in the embodiments of the present application;

[0052] Figure 3 is a prostate cancer detection result schematic diagram provided by the embodiments of the present application;

[0053] Figure 4 is a structural schematic diagram of the cancer image processing system based on the Mamba structure in the embodiments of the present application. DETAILED DESCRIPTION

[0054] The concept, specific structure and generated technical effects of the present application will be clearly and completely described below in combination with embodiments and drawings, so as to fully understand the purposes, schemes and effects of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0055] Generally, mpMRI (Multi parametric MRI) includes both anatomical sequences (T1 Weighted Imaging (T1WI) and T2 Weighted Imaging (T2WI)) and functional sequences (Diffusion Weighted Imaging (DWI) and Dynamic Contrast Enhanced (DCE)). For example, the DWI image of prostate cancer is mainly used to detect tumors in the peripheral zone, the T2WI image is mainly used to detect tumors in the transition zone, and the ADC map combined with DWI images of different b values is one of the main modalities for diagnosing PCa. Therefore, it is far from enough to rely only on single modality image information for intelligent auxiliary diagnosis of PCa.

[0056] At present, the common cancer image classification is mainly through observation of single modality lesion information by doctors or observation of each image modality, which is very complex and tedious, and the accuracy and efficiency are very low.

[0057] Referring to Figure 1 The present application provides a cancer image processing method based on Mamba structure, which comprises the following steps:

[0058] S100, acquiring MRI images of prostate cancer, marking the lesion area and grading in the MRI images;

[0059] Specifically, the data of the present application is collected in Haikou People's Hospital and Xiangya Medical College Affiliated Hospital of Central South University, and contains 98 cases of prostate cancer patients from 2013 to 2016. Doctors performed magnetic resonance imaging on all patients and determined the suspected cancer. 98 cases of MRI images of patients with prostate cancer were collected. The three-dimensional DICOM format data was processed into two-dimensional slices and the image format was converted into BMP format, and the two-dimensional image size was 512*512. Then, under the guidance of experts, the lesion area of prostate in the MRI image was manually labeled by referring to the relevant literature and with the help of Photoshop drawing tools, and the.shp vector file composed of points, lines and surfaces was obtained as the true value map. The PI-RADS grading and tumor IISUP grading of the prostate in the MRI image were labeled, and the classification label file obtained was used as the gold standard.

[0060] In this example, all examinations were performed on a 3T scanner (Achieva 3T; Philips Healthcare, Eindhoven, The Netherlands) using a 32-channel phased-array coil. Prostate biopsies were performed during these examinations and were diagnosed as prostate cancer. Pathological diagnosis was performed by a hospital board-certified pathologist according to the Gleason grading system. The data of the present invention are corresponding to the image voxels of 512x512x22 for the first diagnosis of prostate cancer for 98 patients. The actual field of view (FOV) of the patients was 400mmx400mm with a thickness of 4mm. Note that the used dataset has been approved by the relevant hospital ethics committee and the consent of the informed patients has been obtained.

[0061] PI-RADS (Prostate Imaging Reporting and Data System) is a standardized grading system for prostate cancer imaging evaluation, mainly used for the interpretation of multi-parameter MRI (mpMRI) examination results. The PI-RADS grading system divides prostate lesions into 1 to 5 levels, and the higher the grade, the higher the possibility of prostate cancer.

[0062] The ISUP (International Society of Urological Pathology) grading system is a grading method for pathological evaluation, mainly used for the prognosis evaluation of prostate cancer and other urinary system tumors. The ISUP grading system is based on the Gleason scoring system and is simplified and optimized, dividing prostate cancer into 1 to 5 levels.

[0063] S200, pre-processing the MRI image, performing image expansion processing on the pre-processed MRI image to form a sample data set containing multiple image modalities, and dividing the sample data set into a training set and a test set; the sample data set contains multiple cancer image samples, each of which has a corresponding image modality;

[0064] In some embodiments, due to the large field of view of multi-modal MRI image acquisition, the prostate and tumor area occupies a small area of the image, so it is necessary to crop the original MRI image to extract the central region containing the prostate and tumor; since deep learning requires a large number of training samples, the cropped images are subjected to rigid transformation processing, including up-down, left-right flipping, and rotating by a certain angle to expand the sample size. The cropped MRI image and the expanded sample data are used as cancer image samples, and the cancer image samples are divided into a training set and a test set according to a certain proportion, and the cancer image samples include three image modalities of T2WI images, DWI images and ADC images.

[0065] S300, constructing an improved neural network model, inputting the cancer image samples in the training set into the improved neural network model to extract multi-scale features of each image modality; wherein the improved neural network model comprises a plurality of multi-modal image feature extraction networks, the number of the multi-modal image feature extraction networks is consistent with the number of image modalities; the multi-modal image feature extraction network comprises an improved V-Net network combined with SSM, and the improved V-Net network comprises a CNN-SSM block combined with a convolutional neural network and a state space model;

[0066] It should be noted that the T2WI images, DWI images and ADC images in the data set are respectively input into the multi-modal image feature extraction network, the cancer image samples of different image modalities show different information, and multi-modal fusion extraction can more accurately represent the prostate cancer area.

[0067] The CNN-SSM block improves multi-modal image feature extraction and classification by combining the advantages of convolutional neural network (CNN) in local feature extraction and the efficiency of state space model (SSM) in capturing long-range dependencies.

[0068] S400, feeding the multi-scale features extracted by the multi-modal image feature extraction network through each Mamba block and the corresponding down-sampling layer to obtain the modality output features of the encoder;

[0069] S500, constructing a multi-task learning module, inputting the multi-modal output features of the encoder into the multi-task learning module for multi-task learning to obtain a multi-task learning loss; wherein the multi-task learning module comprises a lesion detection module and a lesion grading module, the lesion detection module is used for positioning the lesion in the image, and the lesion grading module is used for classifying the lesion in the image;

[0070] S600, inputting the cancer image samples in the test set into the improved neural network model, calculating the multi-task learning loss of the improved neural network model, updating the parameters of the improved neural network model according to the multi-task learning loss until convergence, and obtaining a trained model;

[0071] S700, inputting the newly collected MRI image into the trained model to obtain the lesion positioning result of the cancer image and the classification result of the cancer image.

[0072] The test data is input into the trained model to obtain the cancer image detection result and the cancer image classification result; the result obtained in step S700 is compared with the label in step S100, and the performance of the proposed model is evaluated according to the evaluation index. The evaluation index of the cancer detection method includes correlation coefficient (CC), overlap rate (Overlap), Dice similarity coefficient (DSC) and accuracy (ACC).

[0073] The definitions are as follows:

[0074]

[0075] Wherein, A i and B i respectively represent the label map of the cancer area of the i-th scanning slice, and the model output.

[0076]

[0077] Wherein, A i and B i respectively represent the label map of the cancer area of the i-th scanning slice, and the model output.

[0078]

[0079] The cancer classification result evaluation method uses five performance evaluation indexes to quantitatively evaluate the prostate PI-RADS automatic grading result, including calculating ACC, precision (Precision, PRE), recall, specificity (Specificity, SPE) and F1 score (F1 score, F1) to evaluate the performance.

[0080]

[0081] Wherein, TP, TN, FP and FN represent true positive, true negative, false positive and false negative respectively.

[0082] The present application aims at the multi-modal information fusion and multi-target grading problem in the process of cancer diagnosis, introduces a multi-modal deep multi-task self-attention network, interacts and fuses multi-modal information and constructs multi-task correlation features, improves the robustness and accuracy of the cancer lesion grading module. In addition, in order to break through the problem of low running efficiency of multi-modal long sequence data model, a deep network model based on Mamba structure is constructed, a state space model is introduced, the algorithm running time complexity is reduced, and the detection and classification performance and efficiency of cancer image are improved.

[0083] Reference Figure 2In some improved embodiments, in S300, the cancer image samples in the training set are input into the improved neural network model to extract multi-scale features of each image modality, including:

[0084] In S310, the cancer image samples of each image modality are input into the improved V-Net model in each multi-modal image feature extraction network for MxMxM convolution operation to obtain a first intermediate feature map; wherein the improved V-Net model has a channel number of N1 and a step of K, followed by a ReLU function with a decay rate of 0.85;

[0085] The input of the improved neural network model is the MRI voxel of the cancer image sample, assuming that the MRI voxel is V={S0,…S i ,…S n}, wherein S i represents the i-th slice, d l and d w represent the size of the slice of the MRI image, d l represents the length of the slice, d w represents the width of the slice.

[0086] In S320, the first intermediate feature map is input into the convolution block in the improved V-Net network for convolution operation to obtain a second intermediate feature map; wherein the convolution block includes 2 MxMxM convolution operations, with a channel number of N1, and each convolution operation is followed by a ReLU function operation with a decay rate of 0.85;

[0087] In S330, the second intermediate feature map is input into the max-pooling layer in the improved V-Net network for down-sampling operation to obtain a third intermediate feature map; wherein the max-pooling layer has a step of K, and the feature size of the third intermediate feature map is 1 / 2 of the second intermediate feature map;

[0088] In S340, S320 and S330 are repeatedly executed until the size of the third intermediate feature map is less than a set value P, and the cycle number L and the channel number N L+1 of the third intermediate feature map are recorded;

[0089] In S350, the third intermediate feature map is input into the residual block in the improved V-Net network to obtain an output feature map; wherein the residual block includes 2 MxMxM convolution operations, with a channel number of N L+1 , and each convolution operation is followed by a ReLU function operation with a decay rate of 0.85;

[0090] In S360, the output feature maps corresponding to each image modality form the multi-scale features extracted by the improved neural network model.

[0091] Specifically, the output feature map corresponding to the T2WI image is denoted as a first feature map, the output feature map corresponding to the DWI image is denoted as a second feature map, and the output feature map corresponding to the ADC image is denoted as a third feature map; the first feature map, the second feature map, and the third feature map form the multi-scale features extracted by the improved neural network model.

[0092] In some improved embodiments, in S400, the multi-scale features extracted by feeding the multi-modal image feature extraction network through each Mamba block and the corresponding down-sampling layer are obtained, and the modal output features of the encoder are obtained, including:

[0093] In S410, the output features corresponding to each image modality are obtained. Figure One In each input Mamba block, 3D features of the output feature map are extracted;

[0094] In S420, for the output feature map corresponding to each image modality, a flattening operation is used to reshape the 3D features of the output feature map into a 1D long sequence, and the multi-modal features corresponding to each image modality are obtained.

[0095] Specifically, the first multi-scale 3D feature z0 is extracted by feeding the backbone layer through each Mamba block and the corresponding down-sampling layer, and for the multi-modal images corresponding to the same image modality, a flattening operation φ is used before the Mamba block to reshape the 3D features of each multi-modal image into a plurality of 1D long sequences, thereby realizing efficient sequence modeling with less sensory bias.

[0096] The first feature map, the second feature map, and the third feature map are respectively input into each Mamba block; for the 3D features in the cancer image sample, the 3D features are respectively reshaped into 1D long sequences by the formula z l-1 =φ(z l-1 ), and the multi-modal features

[0097] In S430, the multi-modal features corresponding to each image modality are subjected to feature fusion processing, and the fused features are obtained.

[0098] The feature fusion manner is to separately input the multi-modal features of each image modality, and to concatenate the multi-modal features of each image modality into the same sequence according to the corresponding sequence number in the node generation stage. The formula for feature fusion processing is:

[0099]

[0100] wherein, is the fused feature.

[0101] S440, input the fused features into the Mamba module to obtain the multi-modal output features of the encoder:

[0102]

[0103] wherein, is the multi-modal output feature of the encoder.

[0104] In some improved embodiments, in S500, the multi-task learning module is constructed, and the multi-modal output features of the encoder are input into the multi-task learning module for multi-task learning to obtain a multi-task learning loss, including:

[0105] S510, input the multi-modal output features of the encoder into the lesion detection module and the lesion grading module respectively for multi-task learning, and the multi-task learning includes a lesion positioning task and a lesion grading task;

[0106] Specifically, the multi-modal output features of the multi-encoder are input into the lesion detection module and the lesion grading module respectively for multi-task learning. The multi-task learning module learns through multiple tasks to respectively perform the lesion positioning task and the lesion grading task. The lesion grading task in this embodiment includes a PI-RADS grading task and a tumor ISUP grading task.

[0107] S520, for each cancer lesion positioning, input the multi-modal output features into an attention module for lesion positioning processing to obtain output features of the attention module;

[0108] Specifically, for each cancer lesion positioning, the multi-modal output features are input into the attention module for processing to obtain output features of the attention module; the output features of the attention module are:

[0109]

[0110] wherein, ψ(·) represents a convolution operation with a step size of 1 and a convolution kernel of 1, σ(·) is used for activation, χ spatical represents spatial attention, and a dec represents channel attention, represents the output features of the attention module.

[0111] S530, determine a cross-entropy loss of cancer lesion grading based on the multi-modal output features and the output features of the attention module;

[0112] For cancer lesion grading, the cross-entropy loss of the cancer lesion grading is:

[0113]

[0114] wherein, y imgdenote the multi-modal output features, L dec denote the cross-entropy loss of cancer lesion grading.

[0115] At the end of each branch of the joint grading task, a fully connected layer is used to classify each lesion. After generating the lesion-specific features, they are concatenated to obtain multi-modal features and train a classifier to make the final prediction.

[0116] S540, determine the loss of each classification task based on the grading probability output by the lesion grading module in each image modality;

[0117] The loss of each classification task is represented as:

[0118]

[0119] where K denotes the number of data sets, g k denote the true value label of grading, p k denote the probability of grading output by the lesion grading module, n denotes the number of classification tasks, denote the loss of the nth classification task.

[0120] Specifically, the loss function in the PI-RADS grading task is represented as:

[0121]

[0122] The loss function in the ISUP grading task is represented as:

[0123]

[0124] where K denotes the number of data sets, g k denote the true value label of PI-RADS grading, p k denote the probability of PI-RADS grading output by the lesion grading module. m k denote the true value label of ISUP grading, n k denote the probability of ISUP grading output by the lesion grading module, n denotes the n tasks of cancer classification.

[0125] S550, determine the multi-task learning loss based on the cross-entropy loss of cancer lesion grading and the loss of each classification task.

[0126] The expression of the multi-task learning loss is:

[0127]

[0128] where α dec denote the uncertain weight, α1... α ndenote weights of n classification tasks and are obtained by network learning; L denotes a multi-task learning loss.

[0129] Specifically, for the PI-RADS grading and ISUP grading in the lesion grading task, the final multi-task learning loss is:

[0130]

[0131] wherein, α dec , α pi-rads and α isup denote uncertain weights, weights of PI-RADS grading and weights of ISUP grading, and are obtained by network learning. The final model outputs the prostate lesion detection result, the prostate PI-RADS grading and the tumor ISUP grading.

[0132] In order to verify the accuracy of the method, the method in the cancer medical record image embodiment is used to analyze the prostate cancer medical record image, the prostate cancer detection result is as shown in Figure 3 , and the prostate cancer classification result is as shown in Table 1. Compared with the baseline (Baseline) network, only adding the SSM module (B+SSM), only adding the Mamba module (B+Mamba) and adding the SSM and Mamba modules but not multi-task learning processing (B+SSM+Mamba), the proposed method has greatly improved the classification result.

[0133] Table 1:

[0134]

[0135]

[0136] Corresponding to the method of Figure 1 , with reference to Figure 4 , the cancer image processing system based on the Mamba structure is provided, comprising:

[0137] at least one processor;

[0138] at least one memory for storing at least one program;

[0139] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0140] It can be seen that the contents in the above method embodiment are all applicable to the system embodiment, the system embodiment specifically implements the same functions as the above method embodiment, and achieves the same beneficial effects as the above method embodiment.

[0141] In addition, the embodiment of the present application further discloses a computer program product or a computer program, which is stored in a computer readable storage medium. A processor of a computer device can read the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the method described above. Similarly, the contents in the method embodiment described above are all applicable to the present storage medium embodiment, the present storage medium embodiment specifically implements the functions same as the method embodiment described above, and achieves the beneficial effects same as the method embodiment described above.

[0142] Those of ordinary skill in the art will understand that all or some of the methods disclosed above can be implemented as software, firmware, hardware, or appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically includes computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media.

[0143] The above is a specific description of the preferred embodiments of the present disclosure, but the present disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present disclosure, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present disclosure.

Claims

1. A method for cancer image processing based on Mamba structure, characterized in that, The method comprises the following steps: S100, acquiring an MRI image of prostate cancer, marking a lesion area and a grade in the MRI image; S200, preprocessing the MRI image, performing image expansion processing on the preprocessed MRI image to form a sample data set containing multiple image modalities, and dividing the sample data set into a training set and a test set; the sample data set contains multiple cancer image samples, and each cancer image sample has a corresponding image modality; S300, constructing an improved neural network model, inputting the cancer image samples in the training set into the improved neural network model, and extracting multi-scale features of each image modality; wherein the improved neural network model contains multiple multi-modal image feature extraction networks, the number of the multi-modal image feature extraction networks is consistent with the number of image modalities; the multi-modal image feature extraction network contains an improved V-Net network with joint SSM, and the improved V-Net network contains a CNN-SSM block mixed with a convolutional neural network and a state space model; the multi-scale features extracted by the improved neural network model include output feature maps corresponding to each image modality; S400, feeding the multi-scale features extracted by the multi-modal image feature extraction network through each Mamba block and the corresponding down-sampling layer to obtain the modal output features of the encoder; S500, constructing a multi-task learning module, inputting the multi-modal output features of the encoder into the multi-task learning module for multi-task learning to obtain a multi-task learning loss; wherein the multi-task learning module includes a lesion detection module and a lesion grading module, the lesion detection module is used for positioning the lesion in the image, and the lesion grading module is used for classifying the lesion in the image; S600, inputting the cancer image samples in the test set into the improved neural network model, calculating the multi-task learning loss of the improved neural network model, updating the parameters of the improved neural network model according to the multi-task learning loss until convergence, and obtaining a trained model; S700, inputting a newly acquired MRI image into the trained model to obtain a lesion positioning result of the cancer image and a classification result of the cancer image; S400 comprises: S410, inputting the output feature maps corresponding to each image modality into the multiple Mamba blocks one by one to extract 3D features of the output feature maps; S420, for the output feature maps corresponding to each image modality, adopting a flattening operation to reshape the 3D features of the output feature maps into a 1D long sequence to obtain multi-modal features corresponding to each image modality; S430, performing feature fusion processing on the multi-modal features corresponding to each image modality to obtain fused features; S440, inputting the fused features into the Mamba module to obtain the multi-modal output features of the encoder.

2. The method of claim 1, wherein, In S300, the inputting of the cancer image samples in the training set into the improved neural network model and the extraction of the multi-scale features of each image modality comprise: S310, inputting the cancer image samples of each image modality one by one into the improved V-Net model in each multi-modal image feature extraction network for MxMxM convolution operation to obtain a first intermediate feature map; wherein the number of channels of the improved V-Net model is , the step is K, and followed by a ReLU function with a decay rate of 0.85; S320, input the first intermediate feature map into a convolution block in the improved V-Net network for convolution operation to obtain a second intermediate feature map; wherein the convolution block includes 2 MxMxM convolution operations, and the number of channels is and each convolution operation is followed by a ReLU function operation with a decay rate of 0.

85. S330, input the second intermediate feature map into a max pooling layer in the improved V-Net network for downsampling operation to obtain a third intermediate feature map; wherein, the step length of the max pooling layer is K, and the feature size of the third intermediate feature map is 1 / 2 of the second intermediate feature map; S340, cyclically performing S320 and S330 until the size of the third intermediate feature map is less than a set value P, recording the cycle number L and the channel number of the third intermediate feature map ; S350, input the third intermediate feature map into a residual block in the improved V-Net network to obtain an output feature map; wherein the residual block comprises two MxMxM convolution operations, and the number of channels is and each convolution operation is followed by a ReLU function operation with a decay rate of 0.

85. S360, form the output feature maps corresponding to each image modality into the multi-scale features extracted by the improved neural network model.

3. The method of claim 2, wherein, In S500, the multi-task learning module is constructed, and the multi-modal output features of the encoder are input into the multi-task learning module for multi-task learning to obtain a multi-task learning loss, including: S510, input the multi-modal output features of the encoder into the lesion detection module and the lesion grading module respectively for multi-task learning, and the multi-task learning includes a lesion positioning task and a lesion grading task; S520, for each cancer lesion positioning, input the multi-modal output features into the attention module for lesion positioning processing to obtain output features of the attention module; S530, determine a cross-entropy loss of cancer lesion grading based on the multi-modal output features and the output features of the attention module; S540, determine a loss of each classification task based on the grading probability output by the lesion grading module in each image modality; S550, determine a multi-task learning loss based on the cross-entropy loss of cancer lesion grading and the loss of each classification task.

4. The method of claim 3, wherein, The output features of the attention module are: ； wherein, denotes a convolution operation with a step size of 1 and a convolution kernel of 1, for activation, denotes spatial attention, denotes channel attention, denotes the output feature of the attention module.

5. The method of claim 4, wherein, The cross-entropy loss of cancer lesion grading is: ； wherein, represents a multi-modal output feature, represents a cross-entropy loss for cancer lesion grading.

6. The method of claim 5, wherein, The loss of each classification task is expressed as: ； wherein, denotes the number of data sets, denotes the true label of the classification task, denotes the probability of the grade output by the lesion grading module, n denotes the number of the classification task, denotes the loss of the n-th classification task.

7. The method of claim 6, wherein, The expression of the multi-task learning loss is: ； wherein, denotes an uncertain weight, denotes the weight of n classification tasks, and is obtained by network learning; L denotes a multi-task learning loss.

8. A cancer image processing system based on Mamba architecture, characterized by, The system comprises: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1 to 7.

9. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor when executed by the processor is used to execute the method of any one of claims 1 to 7. The program executable by the processor when executed by the processor is used to execute the method of any one of claims 1 to 7.

Images