Lung sign classification method, system, device and medium based on mamba model

Abstract

The application discloses a lung sign classification method, system, device and medium based on a Mamba model, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of large calculation overhead and weak feature capturing capability in lung signs of different lung diseases in the prior art, and a lung sign classification model is constructed by using a Mamba model; the Mamba model applies a state space equation to describe state updating and output at each time step during training, the state updating equation is: and the state transition matrix is subjected to sparse processing. The method can effectively reduce the number of non-zero elements in the matrix, and uses a sparse matrix, so that the calculation overhead of the model can be significantly reduced, different lung disease samples can be dynamically adapted, the feature capturing capability of different samples is enhanced, the feature capturing of different lung disease samples is improved, and the classification accuracy of lung signs of different types of lung diseases is improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology and relates to the classification of lung signs, and more particularly to a method, system, device and medium for classifying lung signs based on the Mamba model. Background Technology

[0002] According to statistics, lung cancer is a serious threat to human health, and early detection and diagnosis are crucial for reducing mortality. In recent years, with the rapid development of artificial intelligence technology, AI has also seen rapid development and application in the medical field, especially in scenarios such as lung image detection, segmentation, and classification.

[0003] In the diagnosis and treatment of lung diseases, artificial intelligence (AI) technology is used to classify and detect lung auscultation sounds, but more commonly, it is used to detect and classify lung images. Based on this, existing technologies mostly employ CNN convolutional neural networks and U-Net neural networks to process lung images, while some also use Transformer model architectures with multi-layered stacked encoders and decoders.

[0004] Patent application number 202311808803.5 discloses a multimodal lung MRI image segmentation method that integrates deep learning with edge information. This method involves labeling lung proton MRI images, using the lung proton MRI images and corresponding lung hyperpolarized 129XeMRI images as samples to construct a training set; simultaneously constructing a multimodal lung MRI image segmentation network; setting a total loss function; training the multimodal lung MRI image segmentation network using the training set; and using the trained multimodal lung MRI image segmentation network to segment the lung hyperpolarized 129XeMRI images to be processed, obtaining segmented images. The multimodal lung MRI image segmentation network includes a segmentation module and an edge information module. The segmentation module includes an encoder and a decoder; the encoder includes a convolutional neural network, a feature fusion block, and a Transformer block; the decoder includes an upsampling layer and an output layer; and the edge information module includes convolutional blocks and edge detection blocks, specifically three convolutional blocks and three edge detection blocks.

[0005] As mentioned in the aforementioned patent application, the Transformer model architecture has achieved great success in the field of deep learning. However, as the model size increases and the processed sequences become longer, the limitation of its self-attention mechanism, whose computational cost increases quadratically with the context length, gradually becomes apparent. To overcome this shortcoming of the Transformer architecture, the new Mamba network emerged. The Mamba network merges the previous SSM architecture design with the Transformer's MLP block into a single block, forming a simple, homogeneous architecture design that includes a selective state space. It simplifies the deep sequence model architecture and possesses characteristics such as high quality, fast training and inference, and long context processing capabilities; it improves computational efficiency and reduces computational complexity while ensuring model performance. However, the Transformer model has high computational overhead when processing long sequence data, making it difficult to run efficiently on resource-constrained medical devices and difficult to dynamically adapt to the characteristics of lung nodules in different lung diseases.

[0006] To this end, patent application No. 202411491741.4 discloses a lung nodule detection method, system, and device based on Mamba and global multi-scale feature fusion. It improves the CenterNet algorithm to construct a lung nodule detection model, including replacing the original feature extractor in CenterNet with a Mamba-based feature extractor, adding a multi-scale feature aggregation module to fuse features from different scales; using the lung nodule detection model to detect lung CT images, including: feature encoding of the lung CT images; feature decoding of the encoded multi-scale feature maps; inputting multi-scale feature maps from different decoding stages into the multi-scale feature aggregation module to aggregate features with different functions; performing bounding box regression prediction, locating lung nodules through center point prediction, offset prediction, and size prediction, and combining these to obtain the final detection result.

[0007] The Mamba model is a novel deep learning model whose core is the Selective State Space Model (SSMs). In existing technologies, state updates in the Mamba model are primarily based on Selective State Space Models (SSMs), and the state update process is as follows:

[0008] Given an input sequence x1, x2, ..., x T The model's state update consists of the following steps: 1. Initialize the state: At time t=0, initialize the hidden state h0 as a zero vector or initialize it according to some random distribution; 2. Calculate the input gate: For each time step t, first calculate the input gate. It determines how much new input information is incorporated into the current state update; 3. Calculate candidate states: Next, calculate the candidate states. It is an intermediate representation based on the current input and the state at the previous time step; 4. Update state: based on the input gate and candidate states To update the hidden state at the current moment The updated formula is: In this way, the model can flexibly control how much new information is incorporated into the current state and how much of the previous state information is retained, depending on the value of the input gate.

[0009] However, the state transition matrix used in the existing Mambam model is usually dense, which leads to high computational cost. It is difficult for it to dynamically adapt to the lung nodule features of different lung diseases, and its ability to capture the lung nodule features of different lung diseases is low, ultimately affecting the detection of lung nodules. Summary of the Invention

[0010] The purpose of this invention is to address the technical problems of high computational overhead and weak ability to capture features in lung signs of different lung diseases in the prior art, and to provide a lung sign classification method, system, device and medium based on the Mamba model.

[0011] To achieve the above objectives, the present invention specifically adopts the following technical solution:

[0012] A lung sign classification method based on the Mamba model includes the following steps:

[0013] Step S1: Obtain sample data;

[0014] Acquire lung CT images and annotate lung features in the lung CT images to obtain image samples and label data;

[0015] Step S2: Construct a classification model for lung signs;

[0016] A lung sign classification model was constructed, and the Mamba model was adopted for the lung sign classification model;

[0017] Step S3: Train the lung sign classification model;

[0018] The lung CT image is divided into small blocks of size P·P, each block becoming a patch, and each patch is flattened into a one-dimensional vector. A linear transformation is then performed to obtain a high-dimensional representation. The high-dimensional representations of all patch blocks are combined into a sequence and used as input to the Mamba model;

[0019] The Mamba model uses a state-space equation to describe the state updates and outputs at each time step during training. The state update equation is:

[0020] ;

[0021] in, Indicates a hidden state. Represents the state transition matrix. This represents the input transformation matrix. Indicates input;

[0022] For the state transition matrix Perform sparsification processing;

[0023] Step S4, real-time classification;

[0024] The lung CT image to be detected is acquired and input into the lung sign classification model. The lung sign classification model outputs the lung sign classification result.

[0025] Furthermore, in step S2, the data space and / or time characteristics of the Mamba model are layered, and a different state transition matrix is ​​used for each layer. .

[0026] Furthermore, step S3 involves adjusting the state transition matrix. The specific steps for sparsification are as follows:

[0027] State transition matrix The sparsity mode is set to diagonal sparse or block sparse;

[0028] State transition matrix The storage format should be set to CSR or COO format;

[0029] Dynamically adjust the state transition matrix using a soft gating mechanism. The sparse pattern.

[0030] Furthermore, the state transition matrix in step S3 After low-rank decomposition, training is performed using singular value decomposition or nonnegative matrix decomposition; the low-rank decomposition is expressed as:

[0031] ;

[0032] ;

[0033] in, Represents the left factor matrix, Represents the right factor matrix, Indicates transpose. Represent real numbers, Represents the row rank of a matrix. Represents the column rank of the matrix.

[0034] Furthermore, the state transition matrix Using a dynamic state transition matrix, it can be represented as:

[0035] ;

[0036] in, Represents a learnable function. Indicates input, Indicates trainable parameters, This represents the input weight matrix. Represents the state weight matrix. Representation and matrix The bias term used in conjunction with it, Representation and matrix The bias term used in conjunction with it, This represents the activation function.

[0037] A lung sign classification system based on the Mamba model, comprising:

[0038] The sample data acquisition module is used to acquire lung CT images and annotate lung signs in the lung CT images to obtain image samples and label data.

[0039] The lung sign classification model construction module is used to construct a lung sign classification model, which adopts the Mamba model.

[0040] The lung sign classification model training module is used to divide lung CT images into small blocks of size P·P, each block becoming a patch, and flattening each patch into a one-dimensional vector. A linear transformation is then performed to obtain a high-dimensional representation. The high-dimensional representations of all patch blocks are combined into a sequence and used as input to the Mamba model;

[0041] The Mamba model uses a state-space equation to describe the state updates and outputs at each time step during training. The state update equation is:

[0042] ;

[0043] in, Indicates a hidden state. Represents the state transition matrix. This represents the input transformation matrix. Indicates input;

[0044] For the state transition matrix Perform sparsification processing;

[0045] The real-time classification module is used to acquire the lung CT image to be detected and input it into the lung sign classification model. The lung sign classification model outputs the lung sign classification result.

[0046] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method described above.

[0047] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the above-described method.

[0048] The beneficial effects of this invention are as follows:

[0049] 1. In this invention, the Mamba model is used as the lung sign classification model. The Mamba model training adopts an innovative state update algorithm to reduce the number of non-zero elements in the matrix and uses a sparse matrix, which can significantly reduce the computational cost of the model. It can dynamically adapt to the classification and detection of lung signs of different lung disease samples, enhance the ability to capture the features of different samples, improve the feature capture of different lung disease samples, and improve the classification accuracy of lung signs of different types of lung diseases.

[0050] 2. In this invention, regarding the state transition matrix... When performing sparsification, the state transition matrix is... The sparse mode is set to diagonal sparse or block sparse, which reduces the computational complexity to O(k) (k is the number of non-zero elements); and the storage is performed in CSR or COO format, which can further reduce the storage footprint. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0052] Therefore, all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0053] In this application, lung signs refer to inflammatory lesions in the lungs found through chest X-ray or CT examination, such as thickened or blurred lung markings, which can serve as an auxiliary basis for subsequent diagnosis of pneumonia.

[0054] In addition, the classification of lung signs in this application includes pulmonary nodules, lung masses, ground-glass opacities, and consolidation.

[0055] Example 1

[0056] This embodiment provides a lung sign classification method based on the Mamba model, including the following steps:

[0057] Step S1: Obtain sample data;

[0058] Acquire lung CT images and annotate lung features in the images to obtain image samples and label data.

[0059] Lung CT images and their corresponding sample data are sourced from clinical imaging databases and public medical imaging databases (such as LIDC-IDRI and TCIA) of medical institutions. The sample data undergoes preprocessing and annotation to meet research requirements. Preprocessing includes DICOM format conversion, image normalization, resampling, noise reduction, and contrast enhancement to improve data consistency and quality. The annotation process requires segmenting lung feature regions using professional medical imaging tools (such as ITK-SNAP and 3D Slicer) and labeling the corresponding lung feature regions. These lung feature region category labels include pulmonary nodules, lung masses, ground-glass opacities, and consolidation, resulting in labeled data. These labels are completed by professionals (such as radiologists), and the annotation results, together with the CT images, constitute the sample data and labeled data used for model training.

[0060] Step S2: Construct a classification model for lung signs;

[0061] A lung sign classification model was constructed, using the Mamba model to extract lung sign features from CT images.

[0062] This Mamba model can be implemented using existing Mamba models.

[0063] To reduce computational redundancy, the data space and / or temporal characteristics of the existing Mamba model are layered, with each layer using a different state transition matrix. The differences in the state transition matrix A across different layers mainly lie in the data characteristics and sparsity processing methods. The state transition matrix A for each layer is customized according to the spatial or temporal characteristics of the corresponding layer. For example, the matrix structure is optimized through low-rank decomposition, non-negative matrix decomposition, or specific sparsity patterns to adapt to different data distributions or modeling needs. This differentiated design not only reduces computational redundancy but also enhances the model's ability to express the characteristics of multiple layers.

[0064] Furthermore, this lung sign classification model can also employ knowledge distillation techniques, using a large Mamba model as the teacher network and a small Mamba model as the student network. A distillation loss function based on feature matching is designed, with the large Mamba model guiding the learning of the small Mamba model. Specifically, mean squared error is used to measure the similarity of intermediate features between the teacher and student networks. The loss function is as follows:

[0065] ;

[0066] in, Represents the total number of samples. This represents the intermediate features of the i-th sample in the teacher network. Let represent the intermediate features of the i-th sample in the student network.

[0067] Step S3: Train the lung sign classification model;

[0068] The lung CT image is divided into small blocks of size P·P, each block becoming a patch block. The k-th patch block is denoted as . ,available A patch block (of which, , These represent the length and width of the image, respectively. (Indicates the size of the block); flatten each patch block into a one-dimensional vector. ( , To map each patch block to a high-dimensional space, a one-dimensional vector is used. Perform linear transformation ( , Represents the projection matrix. (representing the bias), a low-dimensional representation of each patch block is obtained through a linear transformation. Mapping yields a high-dimensional representation Because image data contains spatial location information, location embedding is required. To save this information, ; A high-dimensional representation of all patch blocks (i.e., a high-dimensional representation after incorporating spatial location information) The sequence is composed of 1,000 elements and used as input to the Mamba model.

[0069] The Mamba model processes the input sequence through state-space equations: First, the Mamba model applies state-space equations to describe the state update at each time step, where the state update equation is:

[0070] ;

[0071] in, Indicates a hidden state. Represents the state transition matrix. This represents the input transformation matrix. This indicates input.

[0072] During training, the state transition matrix is... Low-rank decomposition is performed, and singular value decomposition or nonnegative matrix decomposition is used to ensure effective information transmission. The low-rank decomposition is expressed as:

[0073] ;

[0074] ;

[0075] in, Represents the left factor matrix, Represents the right factor matrix, Indicates transpose. Represents the real number field. Represents the row rank of a matrix. Represents the column rank of the matrix.

[0076] This optimization process reduces the computational complexity from... It reduces to O(dr) and is more suitable for lightweight embedded devices.

[0077] Furthermore, traditional SSM uses a fixed transition matrix, but the pulmonary signs of lung diseases are diverse. Therefore, this embodiment uses a different state transition matrix. A dynamic state transition matrix is ​​employed, enabling it to adjust dynamically based on input data. This state transition matrix is ​​generated using a learnable parameterized function. Specifically, it is represented as follows:

[0078] ;

[0079] in, Represents a learnable function. Indicates input, Let be the trainable parameters, such that It adaptively generates data based on input data, thereby enhancing the ability to model different lung diseases; This represents the input weight matrix. Represents the state weight matrix. Representation and matrix The bias term used in conjunction with it, Representation and matrix The bias term used in conjunction with it, This indicates the ReLU or Sigmoid activation function.

[0080] To reduce computational complexity, the state transition matrix was also modified. Perform sparsification processing; specifically:

[0081] 1. Transform the state transition matrix The sparsity mode is set to either diagonal sparsity or block sparsity. This is a static mode selection, meaning a fixed sparsity mode (either diagonal sparsity or block sparsity) is predefined in the model, reducing the computational complexity to [value missing]. k is the number of non-zero elements;

[0082] 2. State transition matrix The storage format is set to CSR (Compressed Sparse Row) or COO (Coordinate Format) to further reduce storage footprint;

[0083] 3. Use a soft gating mechanism to dynamically adjust the state transition matrix. The model employs a sparse pattern, selecting corner sparsity for some samples to capture local dependencies and block sparsity for others to capture block correlations. A soft gating mechanism dynamically adjusts the sparsity pattern, allowing the model to switch freely based on sample characteristics, thus improving flexibility and performance. The gating network comprises a feedforward neural network and a softmax function. After passing through the feedforward neural network, the softmax function outputs the activation probability of each sparsity pattern, selecting and switching the most suitable sparsity pattern (corner sparsity or block sparsity) based on this probability. The gating network dynamically selects the sparsity pattern by outputting the activation probability of each sparsity pattern through the softmax function, enabling the model to choose the most suitable sparsity pattern for prediction based on each sample. This structure significantly enhances the model's personalized modeling ability and generalization performance.

[0084] Step S4, real-time classification;

[0085] Acquire the lung CT image to be detected and input it into the lung sign classification model. The lung sign classification model outputs the lung sign classification result (that is, the classification result of the lung sign area in the lung CT image, which is one or more of the following: lung nodule, lung mass, ground-glass opacity, and consolidation).

[0086] Example 2

[0087] This embodiment provides a lung sign classification system based on the Mamba model, including:

[0088] The sample data acquisition module is used to acquire lung CT images and annotate lung signs in the lung CT images to obtain image samples and label data.

[0089] The lung sign classification model construction module is used to construct a lung sign classification model, which adopts the Mamba model.

[0090] The lung sign classification model training module is used to divide lung CT images into small blocks of size P·P, each block becoming a patch, and flattening each patch into a one-dimensional vector. A linear transformation is then performed to obtain a high-dimensional representation. The high-dimensional representations of all patch blocks are combined into a sequence and used as input to the Mamba model;

[0091] The Mamba model uses a state-space equation to describe the state updates and outputs at each time step during training. The state update equation is:

[0092] ;

[0093] in, Indicates a hidden state. Represents the state transition matrix. This represents the input transformation matrix. Indicates input;

[0094] For the state transition matrix Perform sparsification processing;

[0095] The real-time classification module is used to acquire the lung CT image to be detected and input it into the lung sign classification model. The lung sign classification model outputs the lung sign classification result.

[0096] Example 3

[0097] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform steps of a lung sign classification method based on the Mamba model.

[0098] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0099] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D-interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the memory may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the lung sign classification method based on the Mamba model. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0100] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to run program code stored in the memory or process data, for example, to run the program code for the lung sign classification method based on the Mamba model.

[0101] Example 4

[0102] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform steps of a lung sign classification method based on the Mamba model.

[0103] The computer-readable storage medium stores an interface display program that can be executed by at least one processor to cause the at least one processor to perform the steps of the lung sign classification method based on the Mamba model as described above.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the lung sign classification method based on the Mamba model described in the embodiments of this application.

Claims

1. A lung sign classification method based on Mamba model, characterized in that, The method comprises the following steps: Step S1, acquiring sample data; acquiring a lung CT image, and labeling lung signs in the lung CT image to obtain image samples and label data; Step S2, constructing a lung sign classification model; The lung sign classification model adopts a Mamba model; Step S3, training the lung sign classification model; The lung CT image is divided into small blocks of size P P, each of which becomes a patch block, and each patch block is flattened into a one-dimensional vector and a linear transformation is performed to obtain a high-dimensional representation The high-dimensional representation of all patch blocks is composed of a sequence as the input of the Mamba model; The Mamba model applies a state space equation to describe the state update and output at each time step during training, and the state update equation is: ； wherein, represents a hidden state, represents a state transition matrix, represents an input variation matrix, represents an input; on the state transition matrix performing sparsification processing; In the process of sparsing the state transition matrix The specific process is as follows: State transition matrix Sparse pattern of the matrix is set to be diagonal sparse or block sparse; State transition matrix The storage format of the state transition matrix is set to CSR or COO format. Dynamic adjustment of state transition matrix using soft gating mechanism Sparse mode; State transition matrix The dynamic state transition matrix is represented as: ； wherein, represents a learnable function, represents an input, represents a trainable parameter, represents an input weight matrix, represents a state weight matrix, represents a matrix a bias term used in conjunction with, represents a matrix a bias term used in conjunction with, represents an activation function; Step S4, real-time classification; acquiring a lung CT image to be detected, and inputting the lung CT image into the lung sign classification model, and the lung sign classification model outputs a lung sign classification result.

2. The method of classifying lung signs based on Mamba model as claimed in claim 1, wherein, In step S2, the data space or / and the temporal characteristics of the Mamba model are stratified and each layer uses a different state transition matrix .

3. The method of classifying lung signs based on Mamba model as claimed in claim 1, wherein, The state transition matrix in step S3 The low-rank decomposition processing is trained by using singular value decomposition or non-negative matrix factorization; wherein the low-rank decomposition is represented as: ； ； wherein, denotes a left factor matrix, denotes a right factor matrix, denotes a transpose, denotes a real number, denotes a row rank of a matrix, denotes a column rank of a matrix.

4. A lung sign classification system based on Mamba model, characterized in that, The method comprises the following steps: The sample data acquisition module is configured to acquire a lung CT image, label lung signs in the lung CT image, and obtain image samples and label data; The lung sign classification model construction module is configured to construct a lung sign classification model, and the lung sign classification model adopts a Mamba model; The lung sign classification model training module is configured to divide the lung CT image into small blocks with a size of P*P, each small block becomes a patch block, and each patch block is flattened into a one-dimensional vector and linear transformation to obtain a high-dimensional representation The high-dimensional representation of all patch blocks is combined into a sequence as the input of the Mamba model. The Mamba model applies a state space equation to describe the state update and output at each time step during training, and the state update equation is: ； wherein, represents a hidden state, represents a state transition matrix, represents an input variation matrix, represents an input; on the state transition matrix performing sparsification processing; In the process of sparsing the state transition matrix Specifically, the process is as follows: State transition matrix Sparse pattern of the matrix is set to be diagonal sparse or block sparse; State transition matrix The storage format of the state transition matrix is set to CSR or COO format. Dynamic adjustment of state transition matrix using soft gating mechanism Sparse mode; State transition matrix The dynamic state transition matrix is represented as: ； wherein, represents a learnable function, represents an input, represents a trainable parameter, represents an input weight matrix, represents a state weight matrix, represents a matrix a bias term used in conjunction with, represents a matrix a bias term used in conjunction with, represents an activation function; The real-time classification module is configured to acquire a lung CT image to be detected, and input the lung CT image into the lung sign classification model, and the lung sign classification model outputs a lung sign classification result.

5. A computer device, characterized by: The computer program is stored in the memory and executed by the processor, so that the processor executes the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer program is stored in the memory and executed by the processor, so that the processor executes the steps of the method according to any one of claims 1 to 3.

Images