Liver and tumor recognition method and device fusing attention mechanism and multi-scale convolution and readable storage medium thereof

Abstract

The invention provides a liver and tumor recognition method and device fusing an attention mechanism and multi-scale convolution and a readable storage medium, and aims to solve the problems of insufficient global information capture, poor multi-scale feature adaptation, weak spatial perception and the like in the existing medical image segmentation technology. A convolution attention module and a coordinate attention module are embedded in jump connection to enhance key feature focusing, and a multi-scale down-sampling block is designed to adapt to tumor multi-scale features. Through data preprocessing and model training and optimization, accurate segmentation of the liver and the tumor is realized. Experiments show that the method significantly improves the segmentation precision in public and clinical data sets, especially improves the detection rate of small tumors and the boundary integrity of large tumors, and provides reliable support for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a liver and tumor recognition method, device, and readable storage medium thereof involving the fusion of an attention mechanism and multi-scale convolution. Background Art

[0002] In the field of medical image analysis, image segmentation is a key technology for achieving accurate identification and quantitative analysis of liver and tumors. In early research, traditional image segmentation methods have long dominated clinical applications, including: threshold segmentation (such as the Otsu algorithm), which is suitable for X-ray bone images; region growing algorithms, which achieve tumor segmentation in CT images by expanding seed points, but rely on the selection of initial points; edge detection (Canny / Sobel operators), which locate organ boundaries based on gradient changes and are commonly used in ultrasound images; level set methods, which maintain topological structures through partial differential equations and are suitable for MR brain tissue segmentation; and GraphCut, which uses graph theory energy functions to characterize tumor regions. However, these methods generally rely on manually designed features and prior knowledge, and have the disadvantages of high parameter sensitivity and poor robustness. They are difficult to adapt to the increasing complexity of medical images and cannot meet the high clinical requirements for accuracy and stability.

[0003] To address these issues, deep learning algorithms are becoming mainstream. Their advantage lies in the fact that they can automatically learn hierarchical feature representations through large-scale data training without requiring manual feature design, and are more robust to noise, artifacts, and individual differences. However, existing deep learning-based segmentation methods still have significant limitations: 1. Inherent flaws of traditional convolution: The local receptive field and weight sharing characteristics of the convolution operation limit the ability to capture global contextual information, making it difficult to model long-range dependencies in medical images (such as the spatial correlation of dispersed lesions); 2. Single feature extraction method: Fixed-scale convolution kernels cannot adapt to anatomical structures with vastly different scales within the same image (e.g., large tumors coexisting with tiny lesions), resulting in inadequate capture of multi-scale features. 3. Insufficient spatial coordinate perception: The lack of an effective spatial position encoding mechanism makes it difficult to accurately identify complex structures such as vascular bifurcations and tumor boundaries, affecting segmentation and positioning accuracy.

[0004] The current mainstream U-Net architecture and its variants (such as Unet, nnUnet, and MedNeXt) show obvious shortcomings in liver tumor segmentation: on the one hand, the backbone network optimized for natural images has low sensitivity to the weak contrast features of tumors in medical images, resulting in a low detection rate for tiny lesions <5mm; on the other hand, the single-scale feature fusion mechanism cannot adapt to the multi-scale characteristics of tumors - the segmentation of large tumors has blurred edges due to the limited receptive field, and small tumors are incorrectly filtered due to the loss of downsampling information.

[0005] Therefore, there is an urgent need for a segmentation method that can take into account both global and local features, adapt to multi-scale tumor characteristics and enhance spatial perception capabilities. Summary of the Invention

[0006] The embodiments of the present invention provide a liver and tumor recognition method, device and readable storage medium thereof that integrate an attention mechanism with multi-scale convolution. The conventional segmentation method in the current technology relies on artificial features and has poor robustness. The deep learning-based method has the problems of insufficient global context information capture, single feature extraction method and weak spatial coordinate perception ability. In particular, the U-Net architecture is difficult to adapt to the multi-scale characteristics and weak contrast characteristics of tumors, resulting in low detection rate of small tumors and blurred edge segmentation of large tumors.

[0007] The core technology of this invention is mainly based on the MedNeXt network framework, adopts the U-Net encoder-decoder structure, integrates CBAM and coordinate attention (CA) mechanisms in skip connections, and introduces a multi-scale convolution module. By enhancing the fusion of global and detailed features and adapting to the multi-scale characteristics of tumors, accurate segmentation of the liver and tumors can be achieved.

[0008] In a first aspect, the present invention provides a liver and tumor recognition method that integrates an attention mechanism and multi-scale convolution, the method comprising the following steps: Step 1: Preprocess the input 3D medical image data to obtain standardized training data; Step 2: Build a segmentation model based on the U-Net encoder-decoder architecture. The segmentation model includes: The encoder consists of multiple MedNeXt blocks and an improved multi-scale downsampling block. The multi-scale downsampling block uses at least two convolution kernels of different sizes to extract features in parallel. The decoder consists of multiple MedNeXt upsampling blocks; A skip connection is used to transfer features extracted by the encoder to the decoder, and at least one attention module is embedded in the skip connection, and the attention module is selected from a convolutional attention module and a coordinate attention module; Step 3: Train the segmentation model based on the preprocessed training data and optimize the model parameters through the loss function; Step 4: Input the 3D medical image to be identified into the trained segmentation model and output the segmentation results of the liver and tumor.

[0009] Furthermore, the preprocessing in step 1 includes the following sub-steps: 1.1. Convert medical imaging data from DICOM or NIFTY format to array format; 1.2. Crop the converted array and keep the non-zero area; 1.3. Resample the cropped array to unify the physical space size of the voxels; 1.4. Standardize the resampled array to make the grayscale value distribution consistent.

[0010] Furthermore, the multi-scale downsampling block in step 2 uses three different sizes of convolution kernels, namely 3×3×3, 5×5×5 and 7×7×7, and the step size of each convolution kernel is 2, which is used to simultaneously extract local detail features and global morphological features.

[0011] Furthermore, the convolutional attention module in step 2 includes a channel attention submodule and a spatial attention submodule. The specific processing process is as follows: The channel attention submodule performs global average pooling and global maximum pooling on the input feature map to obtain two spatial context descriptors. It generates channel attention weights through a shared multi-layer perceptron and weights the feature map by the channel dimension. The spatial attention submodule performs channel-based pooling and splicing on the channel-weighted feature maps, generates spatial attention weights through the convolutional layer, and weights the feature maps by spatial dimensions.

[0012] Furthermore, the coordinate attention module in step 2 includes a three-dimensional coordinate encoding submodule, and the specific processing process is as follows: Perform global average pooling on the input feature map along the X, Y, and Z directions to obtain one-dimensional features in three directions; After concatenating the one-dimensional features, the convolutional layer and batch normalization are performed to generate a fused feature map. The fused feature map is split into features in the X, Y, and Z directions, and the attention weights in the three directions are generated through the convolution layer and the Sigmoid activation function respectively, and the original feature map is weighted by the three-dimensional coordinates.

[0013] Furthermore, the structure of the MedNeXt block includes: Depthwise separable convolutional layer with a kernel size of 3×3×3; Expanded convolution layer: the number of output channels of the expanded convolution layer is R times the number of input channels, where R is the expansion rate; GELU activation function; Compressed convolutional layer, which performs channel-by-channel compression on feature maps; Residual connection, connecting the input features with the output features of the compressed convolutional layer.

[0014] Furthermore, the training process in step 3 includes: Initialize model hyperparameters, including training rounds, initial learning rate, batch size, and optimizer. The optimizer is the SGD optimizer. Performing data enhancement on the preprocessed training data. The data enhancement includes spatial transformation and intensity adjustment. The spatial transformation is selected from rotation and scaling. The intensity adjustment is selected from contrast adjustment and Gaussian noise addition. The difference between the predicted value and the label is calculated using a joint loss function that combines Dice loss and cross entropy loss; The learning rate is adjusted using a polynomial learning rate decay strategy.

[0015] In a second aspect, the present invention provides a device for implementing a liver and tumor recognition method that integrates an attention mechanism and multi-scale convolution, comprising: Data preprocessing module, used for format conversion, cropping, resampling and standardization of 3D medical imaging data; Model building module, used to build segmentation models; The training module is used to train the segmentation model based on the preprocessed data and optimize the parameters; The recognition module is used to input the image to be recognized and output the segmentation result.

[0016] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned liver and tumor recognition method that integrates the attention mechanism and multi-scale convolution.

[0017] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a liver and tumor recognition method based on the above-mentioned fusion attention mechanism and multi-scale convolution.

[0018] The main contributions and innovations of the present invention are as follows: 1. Significantly enhanced global feature extraction capabilities: Through the CBAM module's channel attention (highlighting lesion-related feature channels) and spatial attention (focusing on key anatomical regions), it breaks through the limitations of the local receptive field of traditional convolution and effectively captures the spatial correlation of distant lesions.

[0019] 2. Multi-scale feature fusion is more adaptable to tumor characteristics: Three-branch multi-scale convolution (3×3×3, 5×5×5, and 7×7×7 convolution kernels in parallel) dynamically fuses features of different scales. The small kernel accurately captures the texture of tiny tumors, while the large kernel captures the global morphology of large tumors, solving the problem of insufficient multi-scale lesion segmentation accuracy in the same image.

[0020] 3. Improved spatial coordinate perception accuracy: The CA module decomposes three-dimensional global pooling into one-dimensional encoding in the X, Y, and Z directions, embedding precise spatial position information. This allows accurate identification of complex structures such as irregular tumor boundaries and vascular bifurcations, improving the sensitivity of small lesion detection.

[0021] 4. Preserving hierarchical feature integrity: The U-Net architecture based on the MedNeXt block effectively avoids spatial information attenuation in deep networks through the encoder-decoder symmetric structure and skip connections, combined with the residual connection mechanism, and fuses low-level positioning information with high-level semantic information to improve segmentation continuity.

[0022] 5. Significant Clinical Practicality: Experimental data show that the introduction of the attention mechanism and multi-scale convolution improves the Dice index of liver tumor segmentation by 1.3%-3.3% on FLARE23, LiTS2017, and clinical enhanced CT datasets. This improves the detection rate of small lesions <5 mm and the accuracy of large tumor edge segmentation, providing reliable support for clinical diagnosis.

[0023] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a liver and tumor recognition method integrating an attention mechanism and multi-scale convolution according to an embodiment of the present invention; Figure 2 2. This is a diagram of the liver tumor segmentation model architecture that integrates the attention mechanism and multi-scale convolution according to an embodiment of the present invention; Figure 3 is an architecture diagram of a convolutional attention module and a coordinate attention module according to an embodiment of the present invention; Figure 4 is an architecture diagram of MedNeXt and an upsampling block according to an embodiment of the present invention; Figure 5 is a diagram of a multi-scale MedNeXt downsampling block architecture according to an embodiment of the present invention; Figure 6 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0026] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0027] In the existing technology, traditional segmentation methods rely on artificial features and have poor robustness. Deep learning-based methods have problems such as insufficient capture of global context information, single feature extraction method, and weak spatial coordinate perception ability. In particular, the U-Net architecture has difficulty adapting to the multi-scale characteristics and weak contrast characteristics of tumors, resulting in low detection rate of small tumors and blurred edge segmentation of large tumors.

[0028] Based on this, the present invention solves the problems existing in the prior art by adopting a segmentation model based on the U-Net encoder-decoder architecture.

[0029] Example 1 The present invention aims to propose a liver and tumor recognition method that integrates attention mechanism and multi-scale convolution. Figure 1 , the method comprises the following steps: Step 1: Preprocess the input 3D medical image data to obtain standardized training data; In this embodiment, the general preprocessing of 3D medical image segmentation can be divided into four steps, namely data format conversion, cropping, resampling, and normalization.

[0030] Step 1.1: Data Format Conversion: Common medical image formats include DICOM (.dcm extension) and NIFTY (.nii or .nii.gz extension). These formats are not very convenient to manipulate directly. Typically, the data is read using the corresponding Python library and converted into a NumPy array for subsequent processing. Each case's data is stored as a NumPy array (npz) and a corresponding pickle file (pkl).

[0031] Step 1.2: Image Cropping: Cropping the 3D medical image to its non-zero region. This involves finding a minimal 3D bounding box within the image, with values ​​outside the bounding box set to zero. This bounding box is then used to crop the image. While the cropped image has no effect on the final segmentation result, it does reduce image size, avoids unnecessary computation, and improves computational efficiency.

[0032] Step 1.3, Resampling: The purpose of data resampling is to address the problem of inconsistent spacing between the actual spatial dimensions represented by individual voxels across different images in some 3D medical image datasets. Because convolutional neural networks operate only in voxel space, they ignore information about the size of actual physical space. To address this discrepancy, the image data must be resized in voxel space to ensure that the actual physical space represented by each voxel is consistent across the different image data.

[0033] Step 1.4, Standardization: Finally, standardize the grayscale values ​​of each image in the training set to have a consistent distribution. For example, by performing per-image Z-score normalization, the grayscale distribution of each medical image in the training set is aligned to a near-standard normal distribution, laying the foundation for the subsequent MedNeXt model to learn consistent and stable pathological features.

[0034] Step 2: Build a segmentation model based on the U-Net encoder-decoder architecture. The segmentation model includes: The encoder consists of multiple MedNeXt blocks and an improved multi-scale downsampling block. The multi-scale downsampling block uses at least two convolution kernels of different sizes to extract features in parallel. The decoder consists of multiple MedNeXt upsampling blocks; A skip connection is used to transfer features extracted by the encoder to the decoder, and at least one attention module is embedded in the skip connection, and the attention module is selected from a convolutional attention module and a coordinate attention module.

[0035] Specifically, if Figure 2 In the figure, the segmentation model includes the MedNeXt block (purple), the MedNeXt upsampling block (cyan), the improved multi-scale downsampling block (cyan), the CBAM module (red), the CA module (red), etc. It adopts the U-Net encoder-decoder structure to extract features through downsampling and further transfer features to the decoder through skip connections, and finally outputs the model segmentation result.

[0036] Among them, the segmentation model uses the classic U-Net as the basic framework, focusing on exploring the impact of different attention mechanisms and feature extraction methods on the model's segmentation performance. When building the network, the MedNext block was innovatively used to construct the U-Net, and the network structure was improved by using the efficient feature extraction method in the MedNext block. At the same time, the model improved the depthwise separable convolution, using convolution kernels of different sizes and applying each convolution kernel at different scales to better obtain the multi-scale features of the image and more clearly express the subtle local information and macroscopic semantics of the image. It also attempted to add CBAM and CA attention mechanisms to the jump connections in the U-Net, respectively using CBAM's attention mechanism in the channel and spatial dimensions and CA's focus on coordinates to obtain global features, highlight important parts, suppress unimportant parts, and comprehensively improve the model's ability to process complex medical images.

[0037] like Figure 3 As shown in the left figure (a) of the figure, the CBAM module is composed of channel attention and spatial attention in series. The channel attention module first compresses the spatial dimensions of the input features through global average pooling and global maximum pooling. After multi-layer perceptron transformation, it is normalized using the sigmoid function to generate a channel attention map. The spatial attention module then performs channel-based pooling and concatenation on the input features. This is then passed through a convolutional layer to generate a spatial attention map, which is also normalized using the sigmoid function to focus on important spatial position features. The CBAM attention mechanism can simultaneously extract both channel and spatial information of the input features, helping the model achieve more accurate fine-grained segmentation.

[0038] Unlike conventional methods that directly incorporate CBAM into modules, this method introduces CBAM into skip connections, eliminating the need for the model to recalculate gradients multiple times during forward propagation. This significantly reduces computational complexity. Furthermore, incorporating CBAM into skip connections enables the model to perform attention weighting during the feature fusion phase, preserving the detailed information of lower-level features while optimizing the semantic information of the features. More importantly, this structural design is particularly well-suited to the technical challenges of liver tumor segmentation. The model can achieve the following during the feature fusion phase: 1) Enhance the characteristic responses of significantly enhanced regions during the arterial phase through channel attention adaptation; 2) Using spatial attention to accurately locate the blurred area of ​​the tumor edge; 3) Selectively retain the characteristics of small lesions in the low-level network.

[0039] like Figure 3As shown in the right figure (b) in the figure, compared to CBAM, the CA module does not lose spatial information when learning channel attention, and can grasp the correct spatial position during the channel attention process. Traditional coordinate attention first uses a global pooling operation after the feature map input, dividing the pooled result into the horizontal (X) and vertical (Y) directions to obtain long-range dependencies in both directions. The features obtained by horizontal and vertical pooling are then concatenated and subjected to a convolution operation to achieve a certain degree of interactive fusion of the features in the two directions. The fused features are then batch normalized and an activation function is added. The feature map is then split into horizontal and vertical directions for convolution operations. This allows for simultaneous attention in both horizontal and vertical directions. Finally, the features obtained after the convolution are subjected to a sigmoid operation to produce two attention maps. These two maps reflect the horizontal and vertical presence of the object, helping the model to locate the target object and further enhance model accuracy.

[0040] To more efficiently process 3D medical imaging data and accurately locate liver tumors, this solution innovatively proposes a 3D-CA module, extending the Coordinate Attention (CA) mechanism. This improvement addresses the complex distribution of liver tumors in three dimensions and significantly enhances the model's spatial perception through stereoscopic coordinate positioning. This 3D coordinate positioning mechanism offers the following advantages: 1) More comprehensive spatial perception: It can simultaneously capture the spatial characteristics of liver tumors in three dimensions and accurately identify the boundaries of irregularly growing tumors; 2) More accurate detection of small lesions: Through three-dimensional attention weighting, the characteristic responses of small tumor lesions at all levels are enhanced; 3) Clearer anatomical relationships: Combined with the three-dimensional course characteristics of the liver, the relationship between tumors can be better distinguished.

[0041] like Figure 4 As shown in the left figure (a) in the figure, the basic MedNeXt block first passes through a depth-wise separable convolution with a kernel size of C and C output channels, and then passes through a convolutional layer with channels, where R is the expansion rate, which improves the feature extraction ability of the model through the expansion rate, and then is activated by GELU. Finally, it passes through a convolutional layer with a convolution kernel and the number of output channels to perform channel-by-channel compression on the feature map, and adds residual connections to make the gradient transfer smoother. Figure 4 The right image (b) in the figure is the MedNeXt upsampling module, which uses transposed convolution and stride control for upsampling.

[0042] like Figure 5As shown in the figure, multi-scale convolution kernels utilize convolution kernels of different sizes in parallel and series, effectively adapting to the variable characteristics of liver tumors. Small convolution kernels can better learn local details, while large convolution kernels can achieve a larger receptive field to learn the correlation between tumor and normal tissue. Combining the advantages of multiple scales, it can enhance the detection of small tumors while avoiding segmentation interruptions during the segmentation of large tumors, helping to improve the accuracy of the overall segmentation results.

[0043] By introducing a multi-scale convolutional structure, this scheme deeply optimizes the variable characteristics of liver tumors and effectively solves the problem of discontinuous segmentation of necrotic areas inside large tumors.

[0044] In this embodiment, the training phase of the segmentation model includes the following steps: Step 2.1. Initialize model parameters: The model hyperparameters required for model training, including training rounds (epochs), initial learning rate (learning rate), batch size (batch size) and optimizer type (SGD optimizer), need to be manually initialized.

[0045] Step 2.2, Loading the Data Generator: During the training phase, the basic data generator generated during the data preprocessing phase needs to be loaded into the model. This generator is responsible for providing the raw images and annotated data. To further improve the model's generalization capabilities, the data is processed by the data augmentor before entering the network. This process sequentially performs spatial transformations (such as rotation and scaling) and intensity adjustments (such as contrast and Gaussian noise), ultimately outputting enhanced training and validation sets.

[0046] Step 2.3: During model training, each training round first obtains batches of image data and corresponding labels from the data generator, converts them into tensors, and then loads them onto the GPU for accelerated computing.

[0047] Step 2.3.1: At the beginning of training, the optimizer will first clear the gradient cache to prevent gradient accumulation from affecting parameter updates.

[0048] Step 2.3.2: First, forward propagation is performed. During this process, feature information is propagated according to the specific structure of the model and the prediction results are calculated. There are three model options: ① Add CBAM (Convolutional Block Attention Block) to the skip connection Given an intermediate feature map As input, CBAM sequentially infers a 1D channel attention map and a 2D spatial attention map , the complete attention formula is: ,

[0049] in Represents bitwise multiplication.

[0050] Among them, the channel attention branch generates two different spatial context descriptors by global average pooling (GAP) and global maximum pooling (GMP) of the input features. Then, these two descriptors generate attention weights through a shared MLP (including ReLU). Therefore, the complete formula of the channel attention branch is:

[0051] in, is the sigmoid function.

[0052] The spatial attention branch uses two pooling operations to aggregate the information of a feature map, and then uses a standard convolutional layer for connection and convolution operations to obtain a two-dimensional spatial attention map. The specific formula is:

[0053] in for The convolution kernel.

[0054] ②Adding CA (Coordinate Attention) to the skip connection Given an input F, the input feature map is first divided into three directions: X, Y, and Z, and global average pooling is performed to obtain feature maps in the three directions of X, Y, and Z, namely , , .

[0055] Then the feature maps of the three directions of the global receptive field are spliced ​​together and then fed into the shared convolution kernel as The convolution module then feeds the batch normalized feature map into the Sigmoid activation function to obtain the feature map , the specific formula is: (BN( (Concat[ , , ]))) Then the feature map The convolution kernel is performed in the original three directions: The convolution of and respectively obtains the feature maps with the same number of channels as the original ones, and after the Sigmoid activation function, the attention weights of the feature maps in three directions are obtained. Therefore, , , .

[0056] Finally, by multiplying the original feature map with weighted calculations, we can obtain the feature map with attention weights in three directions.

[0057] ③Use multi-scale downsampling module For the input feature map F, we first use 3×3×3, 5×5×5, and 7×7×7 convolution kernels (stride=2) for spatial downsampling and feature extraction to obtain a feature map containing multi-scale information. The specific formula is: (F), (F), (F)] Then, multiple 1×1×1 convolutions are used to adjust the number of channels, and the final multi-scale information is bitwise multiplied with the original feature map to obtain the final feature map.

[0058] Step 2.3.3: After the model is trained, the difference between the predicted result and the actual result is calculated through the loss function, backpropagation is performed, and the optimizer updates the model parameters according to the gradient returned by the loss function.

[0059] The model adopts a combination of Dice loss (Dice Coefficient Loss, ) and Cross-Entropy Loss (Cross-Entropy Loss, ), and further iteratively update the model parameters by combining the two through appropriate weights. The Loss calculation formula used is:

[0060] Here, α and β are weight coefficients, and α + β = 1. For example, in liver tumor segmentation, α is often set to 0.6-0.8 and β is set to 0.2-0.4, giving priority to ensuring the accuracy of small target segmentation.

[0061] Step 2.3.4: After each training round, the learning rate is dynamically adjusted using a polynomial learning rate decay strategy. The formula is: ) Among them, lr initial is the initial learning rate, epoch is the current training round, max_epochs is the total training round, and exponent is the decay exponent.

[0062] Step 3: Train the segmentation model based on the preprocessed training data and optimize the model parameters through the loss function; In this example, "model performance evaluation" is a key step in verifying the effectiveness of the liver and tumor recognition method proposed in this invention. It aims to quantify the model segmentation performance through objective indicators and experimental data. The specific process and core content are as follows: First, based on the task requirements (liver and tumor segmentation), the trained and optimized MedNeXt segmentation model (an improved model that integrates attention mechanism and multi-scale convolution) is selected, and the unlabeled original 3D medical images (such as CT images) are input into the model. The model outputs the segmentation results (i.e., the division and labeling of the liver and tumor areas) through the encoder-decoder structure and attention and multi-scale feature fusion mechanism. This result can directly assist doctors in clinical diagnosis and treatment decision-making.

[0063] To fully verify the performance, the evaluation was conducted on multiple representative datasets, with the Dice coefficient (an indicator that measures the overlap between the predicted area and the true annotated area; the higher the value, the better the segmentation accuracy) as the core quantitative standard: In the public dataset FLARE23, after the introduction of coordinate attention (CA), the Dice coefficient of liver tumor segmentation increased from 67.89 to 68.32; after the introduction of convolutional attention (CBAM), it increased to 69.12; In the public dataset LiTS2017, after introducing multi-scale convolution, the Dice coefficient increased from 61.23 to 61.33; In the enhanced CT dataset (clinical data) of Zhejiang Cancer Hospital, the Dice coefficient increased from 70.91 to 73.21 after the introduction of CBAM, and increased to 72.32 after the introduction of multi-scale convolution.

[0064] Through the above experiments, the improvement effect of the attention mechanism (CBAM, CA) and multi-scale convolution module in the present invention on segmentation accuracy is intuitively demonstrated, and the effectiveness and generalization ability of the model in different data scenarios are verified, providing a reliable performance basis for clinical applications.

[0065] Step 4: Input the 3D medical image to be identified into the trained segmentation model and output the segmentation results of the liver and tumor.

[0066] For example, a pre-trained MedNeXt segmentation model is selected based on the task type. The original unlabeled 3D medical image is input and the labeled results after segmentation are output. Doctors can use the labeled results to assist in treatment.

[0067] Example 2 Based on the same concept, the present invention also proposes a device for implementing the liver and tumor recognition method integrating the attention mechanism and multi-scale convolution in Example 1, comprising: Data preprocessing module, used for format conversion, cropping, resampling and standardization of 3D medical imaging data; Model building module, used to build segmentation models; The training module is used to train the segmentation model based on the preprocessed data and optimize the parameters; The recognition module is used to input the image to be recognized and output the segmentation result.

[0068] Example 3 This embodiment also provides an electronic device, referring to Figure 6 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0069] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0070] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0071] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0072] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the liver and tumor recognition methods integrating the attention mechanism and multi-scale convolution in the above embodiments.

[0073] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0074] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0075] The input / output device 408 is used to input or output information.

[0076] Example 4 This embodiment also provides a readable storage medium, which stores a computer program. The computer program includes program code for controlling a process to execute a process. The process includes the liver and tumor recognition method that integrates the attention mechanism and multi-scale convolution according to Example 1.

[0077] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0078] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0079] Embodiments of the present invention can be implemented by computer software, which is executable by the data processor of the mobile device, such as in the processor entity, or is implemented by hardware, or is implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer executable components configured to perform the embodiment when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logical circuits, boxes and functions, or a combination of program steps and logical circuits, boxes and functions. The software can be stored in physical media such as memory chips or storage blocks implemented in the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. Physical media is non-transient media.

[0080] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A liver and tumor recognition method that integrates attention mechanism and multi-scale convolution, characterized by: The following steps are involved: Step 1: Preprocess the input 3D medical image data to obtain standardized training data; Step 2: Build a segmentation model based on the U-Net encoder-decoder architecture, which includes: An encoder, consisting of multiple MedNeXt blocks and an improved multi-scale downsampling block, wherein the multi-scale downsampling block uses at least two convolution kernels of different sizes to extract features in parallel; The decoder consists of multiple MedNeXt upsampling blocks; A skip connection, configured to transmit features extracted by the encoder to the decoder, wherein at least one attention module is embedded in the skip connection, and the attention module is selected from a convolutional attention module and a coordinate attention module; Step 3: training the segmentation model based on the preprocessed training data and optimizing the model parameters through the loss function; Step 4: Input the 3D medical image to be identified into the trained segmentation model and output the segmentation results of the liver and tumor.

2. The liver and tumor recognition method integrating attention mechanism and multi-scale convolution as claimed in claim 1, characterized in that: The pre-processing described in step 1 includes the following sub-steps: 1.

1. Convert medical imaging data from DICOM or NIFTY format to array format; 1.

2. Crop the converted array and keep the non-zero area; 1.

3. Resample the cropped array to unify the physical space size of the voxels; 1.

4. Standardize the resampled array to make the grayscale value distribution consistent.

3. The liver and tumor recognition method integrating attention mechanism and multi-scale convolution as claimed in claim 1, characterized in that: The multi-scale downsampling block in step 2 uses three convolution kernels of different sizes, namely 3×3×3, 5×5×5 and 7×7×7, with a step size of 2, to simultaneously extract local detail features and global morphological features.

4. The liver and tumor recognition method integrating attention mechanism and multi-scale convolution as claimed in claim 1, characterized in that: The convolutional attention module in step 2 includes a channel attention submodule and a spatial attention submodule. The specific processing process is as follows: The channel attention submodule performs global average pooling and global maximum pooling on the input feature map to obtain two spatial context descriptors. It generates channel attention weights through a shared multi-layer perceptron and weights the feature map by the channel dimension. The spatial attention submodule performs channel-based pooling and splicing on the channel-weighted feature maps, generates spatial attention weights through the convolutional layer, and weights the feature maps by spatial dimensions.

5. The liver and tumor recognition method integrating attention mechanism and multi-scale convolution as claimed in claim 1, characterized in that: The coordinate attention module in step 2 includes a three-dimensional coordinate encoding submodule, and the specific processing process is as follows: Perform global average pooling on the input feature map along the X, Y, and Z directions to obtain one-dimensional features in three directions; The one-dimensional features are concatenated and processed through a convolution layer and batch normalization to generate a fused feature map; The fused feature map is split into features in three directions: X, Y, and Z. The attention weights in the three directions are generated through the convolution layer and the Sigmoid activation function respectively, and the original feature map is weighted in three dimensions.

6. The liver and tumor recognition method integrating attention mechanism and multi-scale convolution as claimed in claim 1, characterized in that: The structure of the MedNeXt block includes: Depthwise separable convolutional layer with a kernel size of 3×3×3; An extended convolutional layer, wherein the number of output channels of the extended convolutional layer is R times the number of input channels, where R is the expansion rate; GELU activation function; Compressed convolutional layer, which performs channel-by-channel compression on feature maps; Residual connection, connecting the input features with the output features of the compressed convolutional layer.

7. The liver and tumor recognition method integrating attention mechanism and multi-scale convolution according to any one of claims 1 to 6, characterized in that: The training process described in step 3 includes: Initialize model hyperparameters, including training rounds, initial learning rate, batch size, and optimizer, where the optimizer is the SGD optimizer; Performing data enhancement on the preprocessed training data, wherein the data enhancement includes spatial transformation and intensity adjustment, wherein the spatial transformation is selected from rotation and scaling, and the intensity adjustment is selected from contrast adjustment and Gaussian noise addition; The difference between the predicted value and the label is calculated using a joint loss function that combines Dice loss and cross entropy loss; The learning rate is adjusted using a polynomial learning rate decay strategy.

8. A device for implementing the liver and tumor recognition method integrating attention mechanism and multi-scale convolution as described in any one of claims 1 to 7, characterized in that: include: Data preprocessing module, used for format conversion, cropping, resampling and standardization of 3D medical imaging data; A model building module, used for building the segmentation model; The training module is used to train the segmentation model based on the preprocessed data and optimize the parameters; The recognition module is used to input the image to be recognized and output the segmentation result.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the liver and tumor recognition method that integrates the attention mechanism and multi-scale convolution as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes the liver and tumor recognition method that integrates the attention mechanism and multi-scale convolution according to any one of claims 1 to 7.

Images