KTFE-YOLO model-based kidney focus image analysis method and system

Abstract

The invention discloses a kidney lesion image analysis method and system based on a KTFE-YOLO model. The method comprises the steps of obtaining training data and processing the training data; carrying out lesion labeling on a kidney image and storing the lesion labeling as a YOLO format label; a KTFE-YOLO model is constructed: on the basis of a YOLOv11n-seg architecture, C3k2 of a sixth layer and a eighth layer in backbone is replaced by a fuzzy edge enhancement module, and C2PSA of a tenth layer is replaced by a multi-scale convolution fusion attention mechanism module; constructing a bounding box loss function to regress bounding box parameters; based on a bounding box loss function, performing parameter optimization on the KTFE-YOLO model by using the training data; and carrying out classification and segmentation on the kidney images by using the optimized KTFE-YOLO model. According to the method, the segmentation performance of fuzzy boundaries, tumors with complex shape features and small-size targets is remarkably improved while the real-time detection speed is kept.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for analyzing kidney lesion images based on a Kidney-Tumor Feature Enhanced YOLO (KTFE-YOLO) model. Background Art

[0002] In the field of medical image processing, accurate classification and segmentation of renal lesions are crucial for the diagnosis, treatment planning, and prognosis of kidney disease. As a vital excretory and endocrine organ in the human body, the kidneys are susceptible to a wide variety of lesions, including benign and malignant tumors, cysts, and stones. Different types of lesions vary in morphology, size, location, and imaging appearance. Accurately distinguishing these lesions can help doctors better understand the condition and provide personalized treatment strategies for patients. Therefore, effective detection of renal lesions is a key task in medical image analysis.

[0003] In recent years, the rise of machine learning and deep learning technologies has brought new opportunities for medical image analysis. Deep learning methods such as CNN and U-Net have been applied to kidney disease segmentation in medical images, but existing technologies face many challenges:

[0004] (1) The boundaries of renal lesions are unclear, the shapes are complex and diverse, and the contrast with the surrounding tissues is low, which makes it difficult to accurately outline the edges of the lesions.

[0005] (2) Some renal cysts and stones are too small and are easily submerged in the background noise in the downsampled feature map, resulting in poor regression positioning accuracy.

[0006] Furthermore, two-stage segmentation algorithms such as Mask-RCNN and U-Net are time-consuming and cannot meet clinical needs. Traditional one-stage algorithms such as YOLOv5 and SOLO can achieve high-speed segmentation. For example, patent specification CN116912212A discloses a lung nodule CT image analysis method based on the YOLO-CSC model, but these algorithms lack accuracy.

[0007] YOLOv11 is the latest model in the YOLO series. It has made a series of updates to the YOLOv8 architecture, mainly replacing the Cross Stage Partial Bottleneck with 2 convolutions (C2f) in the backbone with a Cross Stage Partial Bottleneck with 3 convolutions and 2 smaller convolutional layers (C3k2). It also adds Cross Stage Partial with Pyramid Squeeze Attention (C2PSA) to the 10th layer of the backbone. This further improves segmentation accuracy while maintaining faster detection speed. However, the performance of this new network architecture on specific medical imaging datasets remains to be verified.

[0008] In order to improve the classification and segmentation accuracy of renal lesions in medical images, an image analysis method and system that can finely segment lesions with fuzzy boundaries, complex shapes, and dense micro-features is urgently needed. Summary of the Invention

[0009] To address the aforementioned technical issues and shortcomings in the field, the present invention provides a renal lesion image analysis method and system based on the KTFE-YOLO model to achieve high-precision classification and segmentation of renal lesions in medical images. KTFE-YOLO, based on the YOLOv11n-seg architecture, integrates and improves the feature extraction module and attention mechanism, and introduces an innovative bounding box loss function. This significantly improves segmentation performance for fuzzy boundaries, complex tumors, and small objects while maintaining real-time detection speed.

[0010] The specific technical solutions are as follows:

[0011] In a first aspect, the present invention provides a method for analyzing renal lesion images based on the KTFE-YOLO model, comprising:

[0012] Obtain training data and process it: annotate kidney images with lesions and store them in YOLO format.

[0013] Constructing the KTFE-YOLO model: Based on the YOLOv11n-seg architecture, the C3k2 in the 6th and 8th layers of the backbone network are replaced with the Blurred Edge Enhanced Model (BEEM), and the C2PSA in the 10th layer is replaced with the Multi-scale Convolution Fusion Attention (MCFA) module. The Blurred Edge Enhancement module is based on the Cross Stage Partial Network (CSP) design. After the input passes through a point-by-point convolution expansion channel, it is split into two branches: one branch is forward propagated, and the other branch passes through a bottleneck block. The bottleneck block consists of a convolutional layer, a Channel Attention (CA) mechanism, a Global Spatial Attention (GSA) mechanism, and a Transformer Self Attention (TSA) mechanism. Finally, the two branches are concatenated channel by channel, and the number of channels is restored by point-by-point convolution, so that the output tensor maintains the same spatial resolution as the input.

[0014] Construct a bounding box loss function to regress bounding box parameters;

[0015] Based on the constructed bounding box loss function, the parameters of the KTFE-YOLO model are optimized using the training data;

[0016] The optimized KTFE-YOLO model is used to classify and segment kidney images.

[0017] The kidney imaging pictures of the present invention may include one or more of kidney CT images and kidney magnetic resonance images (MRI) images.

[0018] The training data is obtained and processed, and the kidney images can be annotated manually. For example, the existing annotation tool labelme is used to annotate the kidney lesions in the area to be segmented, and the labels are stored as YOLO format labels, and are divided into a training set and a validation set according to a certain ratio.

[0019] To address the problems of fuzzy edges, diverse morphologies, and easily confused backgrounds in renal lesions, especially renal tumors, the BEEM module of the present invention can take into account both shallow fine-grained edge details and deep global semantic information, enhancing the ability to extract fuzzy, irregular, and morphologically diverse edges of renal lesions, and significantly improving the boundary accuracy of instance segmentation of benign and malignant renal tumors and cysts.

[0020] Furthermore, in the present invention, the fuzzy edge enhancement module:

[0021] Let F be the input feature map tensor. F is first mapped into two parts f1 and f2 with the same number of channels through point-by-point convolution. f1 then simplifies the channels through point-by-point convolution and captures the local context through 3×3 large kernel convolution. The formula is as follows:

[0022] f1,f2=split(Conv 1×1 (F))

[0023] f1'=Conv 3×3 (Conv 1×1 (f1))

[0024] Among them, split means the segmentation of the channel, Conv 1×1 Represents point-by-point convolution, Conv 3×3 Indicates convolution with a convolution kernel of 3;

[0025] After segmentation, f1' is extracted through a bottleneck block that integrates the channel attention mechanism, the global spatial attention mechanism, and the Transformer self-attention mechanism to obtain deep features f1", and is channel-joined with the shallow features of f2. Finally, the number of channels is restored by point-by-point convolution. The calculation formula is as follows:

[0026]

[0027] F out =Conv 1×1 (concat(f1”,f2))

[0028] Among them, GSA(*) represents the output of the global spatial attention mechanism, CA(*) represents the output of the channel attention mechanism, TSA(*) represents the output of the Transformer self-attention mechanism, and g represents the result after obtaining the weight on the channel through channel attention. Indicates multiplication, concat indicates channel concatenation, F out Represents the output of the fuzzy edge enhancement module.

[0029] In this invention, the channel attention mechanism performs average pooling and maximum pooling on the input, extracts the dependencies between different channels through the dimension reduction convolution layer, then uses the ReLU activation function to enhance the expression, and generates the channel importance score through the restoration convolution. The channel attention mechanism formula is as follows:

[0030] CA(X)=δ(Conv 1×1up (β(Conv 1×1down (AP(X))))+Conv 1×1up (β(Conv1×1down (MP(X)))))

[0031] Among them, X represents input, AP and MP represent average pooling and maximum pooling respectively, Conv 1×1down and Conv 1×1up They represent dimensionality reduction and dimensionality increase using convolution with a convolution kernel of 1, β represents the ReLU activation function, δ represents the Sigmoid activation function, and CA(X) represents the output of the channel attention mechanism.

[0032] In this invention, the global spatial attention mechanism first linearly projects the input into the query (Q), key (K), and value (V) required for the attention mechanism to calculate through three one-dimensional convolutions, then calculates the spatial attention, and finally concatenates the output with the input residual. The global spatial attention mechanism formula is as follows:

[0033] Q = Conv Q (X), K = Conv K (X), V = Conv V (X)

[0034]

[0035] GSA(X)=γO+X

[0036] Among them, X represents the input, Conv Q 、Conv K and Conv V Represent the convolution kernels used to generate Q (query), K (key) and V (value), respectively. The Softmax function is used to normalize a set of real numbers. k is the scaling factor of the attention head, O represents the calculation result of the spatial attention mechanism, γ is a learnable scalar whose initial value is 0, and GSA(X) represents the output of the global spatial attention mechanism.

[0037] In this paper, the Transformer self-attention mechanism flattens the input to generate feature representations of query (Q), key (K), and value (V), then calculates the attention weights through dot product operations, aggregates global feature information and outputs the results. The formula of the Transformer self-attention mechanism is as follows:

[0038] Q=Reshape(X), K=Reshape(X), V=Reshape(X)

[0039]

[0040] Here, X represents the input, the Reshape function is used to flatten the input tensor, and TSA(X) represents the output of the Transformer self-attention mechanism.

[0041] Furthermore, in the present invention, the multi-scale convolution fusion attention mechanism module applies self-attention to the vertical axis and horizontal axis respectively based on axial attention. After the input is divided into two parallel self-attention branches, they are respectively subjected to multi-scale convolution to extract features of receptive fields of different sizes. After that, the point-by-point convolution outputs are in the format (Q, K, V) required for calculating attention. After cross-fusion of the vertical and horizontal axes, the obtained outputs are residually connected with the original input and the final result is output. Specifically including:

[0042] Input feature f with batch size B, number of channels C, image height H, and image width W input , which is divided into two parallel branches: horizontal axis X and vertical axis Y; after the two branches pass through the normalization layer (Norm), multi-scale convolution kernels are applied to this network to effectively capture contextual information at multiple scales. Subsequently, the outputs of these convolutions are aggregated and passed through point-by-point convolution. The formula is as follows:

[0043]

[0044] in, represents one-dimensional multi-scale convolution, i represents the multi-scale convolution label, F1 and F2 represent two paths that capture multi-scale information along different spatial axes;

[0045] For F1 and F2, they are reshaped into the format required for attention calculation, and the results are Q1, K1, V1 (horizontal axis X) and Q2, K2, K2 (vertical axis Y). In the attention calculation stage, Q2 from the vertical axis is correlated with K1 and V1 from the horizontal axis, and Q1 from the horizontal axis is correlated with K2 and V2 from the vertical axis. Finally, the output is obtained by summing with the input, as shown below:

[0046] Q1, K1, V1 = Reshape (F1)

[0047] Q2, K2, V2 = Reshape (F2)

[0048]

[0049] f output =Conv 1×1 (out1)+Conv 1×1 (out2)+f input

[0050] Among them, out1 and out2 are the results after attention mechanism calculation and multi-dimensional fusion, f output is the output.

[0051] In the above-described construction of the KTFE-YOLO model, it is preferred to add an additional segmentation detection head on the shallow feature map to utilize more local information.

[0052] Furthermore, in the present invention, the bounding box loss function constructed includes an innovative loss function NWD-CIoU that combines the normalized Wasserstein distance and the Complete-IoU bounding box loss function. For example, the constructed NWD-CIoU bounding box loss function L is expressed as follows:

[0053] L=λL CIoU +(1-λ)L NWD

[0054] Among them, L represents NWD-CIoU loss, L CIoU represents the CIoU loss, L NWD represents the normalized Wasserstein distance, and λ represents the scaling factor.

[0055] Normalized Wasserstein distance is to model the predicted bounding box and the true bounding box as 2D Gaussian distributions and calculate the similarity between them. NWD The expression is as follows:

[0056]

[0057] Where N is the Gaussian distribution modeled according to the predicted bounding box, N gt is a Gaussian distribution modeled after the true bounding box, Computes the distance between two Gaussian distributions, where C is a constant.

[0058] Complete-IoU bounding box loss function L CIoU After obtaining the prior box corresponding to each box, take out the predicted bounding box corresponding to the prior box, and use the real bounding box and the predicted bounding box to calculate the loss:

[0059]

[0060] Among them, IoU (Intersection over Union) calculates the ratio of the intersection and union of the predicted bounding box and the true bounding box, b and b gt denote the center coordinates of the predicted bounding box and the true bounding box, ρ is the Euclidean distance, c denotes the diagonal length of the minimum bounding box covering the two bounding boxes, α is the weight parameter, and v measures the consistency of the aspect ratio.

[0061] Based on the constructed bounding box loss function, the parameters of the KTFE-YOLO model are optimized using training data. Through continuous iterative learning, when the value of the constructed bounding box loss function converges to a minimum, the optimized KTFE-YOLO model is saved.

[0062] During the KTFE-YOLO model training process, you can set training parameters such as training rounds and batch size to iterate the model multiple times.

[0063] In the kidney lesion image analysis method based on the KTFE-YOLO model, the user can select the optimized KTFE-YOLO model, select the kidney image to be detected and import it, and use the optimized KTFE-YOLO model to classify and segment the kidney image.

[0064] In the classification and segmentation of kidney images using the optimized KTFE-YOLO model, users can adjust the intersection over union (IoU) threshold, confidence interval, image size, whether to retain the classification label, whether to display the confidence level, and whether to display the detection box as needed. After the task is completed, you can choose to save the image with the labeled results to achieve the best detection effect.

[0065] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory. When the computer program is run, the processor executes the kidney lesion image analysis method based on the KTFE-YOLO model described in the first aspect.

[0066] In a third aspect, the present invention provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a computer device, the computer device executes the kidney lesion image analysis method based on the KTFE-YOLO model described in the first aspect.

[0067] In a fourth aspect, the present invention provides a renal lesion image analysis system based on the KTFE-YOLO model, comprising:

[0068] The data acquisition unit is used to acquire and process training data, annotate kidney images with lesions, and store them as YOLO format labels;

[0069] The model construction unit is used to build the KTFE-YOLO model. Based on the YOLOv11n-seg architecture, the C3k2 in the 6th and 8th layers of the backbone are replaced with a fuzzy edge enhancement module, and the C2PSA in the 10th layer is replaced with a multi-scale convolutional fusion attention mechanism module. The fuzzy edge enhancement module is based on a cross-stage partial network design. After the input passes through a point-by-point convolution expansion channel, it is divided into two branches. One branch is forward propagated, and the other branch passes through a bottleneck block. The bottleneck block is composed of a convolutional layer, a channel attention mechanism, a global spatial attention mechanism, and a Transformer self-attention mechanism. Finally, the two branches are spliced through the channels, and the number of channels is restored by point-by-point convolution to ensure that the output tensor maintains the same spatial resolution as the input.

[0070] A bounding box loss function construction unit, used to construct a bounding box loss function to regress bounding box parameters;

[0071] The training unit is used to optimize the parameters of the KTFE-YOLO model using the training data based on the constructed bounding box loss function;

[0072] Application unit, used to classify and segment kidney images using the optimized KTFE-YOLO model.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] 1. This paper proposes a kidney tumor feature enhancement YOLO (KTFE-YOLO, Figure 2 By introducing the fuzzy edge enhancement module (BEEM), the multi-scale convolutional fusion attention mechanism (MCFA) and the new bounding box loss function in the backbone, the model improves the segmentation performance of images with fuzzy edges, complex shape features and tiny lesions while maintaining fast segmentation performance.

[0075] 2. YOLOv11 feature extraction module C3k2 ( Figure 3 ) mainly relies on local convolution to encode image information, but the receptive field of the convolution layer is limited and the perception of global context information is insufficient. This results in the segmentation edge being fragmented and the defect of missed segmentation when segmenting the fuzzy edge of the kidney lesion. In response to the above problems, the present invention provides a fuzzy edge enhancement module (BEEM, such as Figure 4). The module interactively couples three attention mechanisms: channel attention (CA), global spatial attention (GSA), and Transformer self-attention (TSA) under a specific order and fusion strategy to expand the dimension of the intermediate feature layer. Then, through the residual connection method, the model's ability to express image edges and details is enhanced. The module integrates deep global rich information and shallow local detail features, focusing on strengthening the ability to extract blurred, irregular, and multi-morphological edges of kidney lesions, thereby significantly improving the boundary accuracy of instance segmentation.

[0076] In addition, compared with the schemes of using channel attention, global spatial attention or Transformer self-attention (TSA) to extract deep features respectively, or cascading them sequentially, the fuzzy edge enhancement module of the present invention obtains weights in the channel dimension and spatial dimension respectively through different attention mechanisms, and uses these weights to perform weighted processing on the input features, thereby enhancing the ability to represent key information. At the same time, it integrates information at different levels through residual connections, significantly improving the segmentation performance of irregular renal lesion structures and fuzzy edge parts in the kidney segmentation task.

[0077] 3. Because C2PSA in YOLOv11 still has a large computational load and its performance on medical kidney images needs improvement, this paper replaces C2PSA with a multi-scale convolutional fusion attention mechanism (MCFA). MCFA uses a multi-scale convolutional fusion axial attention mechanism to capture information from different spatial locations and global context at various scales, while reducing the computational load of C2PSA in medical imaging tasks.

[0078] 4. A new bounding box loss function is used to replace the original CIoU loss calculation method. This calculation method improves the segmentation performance of small and dense stones and cysts and reduces the risk of model overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 This is the existing YOLOv11 framework diagram;

[0080] Figure 2 This is the KTFE-YOLO framework diagram of the present invention;

[0081] Figure 3 This is the existing C3k2 module framework diagram;

[0082] Figure 4 This is the BEEM module framework diagram of the present invention;

[0083] Figure 5 This is the existing C2PSA framework diagram;

[0084] Figure 6 This is the MCFA framework diagram of the present invention;

[0085] Figure 7 These are images of kidney lesions, including: (a) a kidney cyst; (b) a kidney stone; (c) a kidney tumor.

[0086] Figure 8 For the present invention Figure 7 Result graph;

[0087] Figure 9 The figure shows a heatmap comparison of the existing YOLOv11 model and the present invention for detecting malignant kidney tumors, where: (a) is a heatmap of the existing YOLOv11 model for detecting malignant kidney tumors; (b) is a heatmap of the present invention for detecting malignant kidney tumors. DETAILED DESCRIPTION

[0088] The present invention will be further described below with reference to the accompanying drawings and specific examples. It should be understood that these examples are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0089] A kidney lesion image analysis method based on the KTFE-YOLO model is proposed by using the existing YOLOv11 architecture ( Figure 1 ) based on the improved KTFE-YOLO model ( Figure 2 ), specifically, by introducing the Fuzzy Edge Enhancement Module (BEEM) and the Multi-Scale Convolutional Fusion Attention Mechanism (MCFA) into the backbone for the first time, adding a new shallow segmentation head to the head, and introducing a new bounding box loss function when calculating the loss. This significantly improves the segmentation accuracy of the model for renal lesions in medical images. Specifically, the following steps are involved:

[0090] Step (1), training data processing:

[0091] Manual annotation is used to annotate medical images such as kidney CT images and MRI images. The existing annotation tool labelme is used to annotate the kidney lesions in CT images and MRI images to be segmented, and the coordinates of the annotated points will be stored in a TXT format file.

[0092] The original images and corresponding labels are randomly assigned according to the ratio of 8:2 between the training set and the validation set.

[0093] Step (2), building the KTFE-YOLO model:

[0094] In the backbone of the YOLOv11n-seg architecture, C3k2( Figure 3) is replaced by the Blurred Edge Enhanced Model (BEEM, Figure 4 ).

[0095] In the backbone of the YOLOv11n-seg architecture, C2PSA ( Figure 5 ) is replaced by the Multi-scale Convolution Fusion Attention mechanism (MCFA, Figure 6 ).

[0096] An innovative bounding box loss function is used to regress the bounding box parameters.

[0097] Add an additional segmentation detection head on the shallow feature map.

[0098] Follow the above steps to build the KTFE-YOLO model. When the loss function converges to a minimum through iterative learning, save the model.

[0099] Furthermore, in step (2), the construction of the KTFE-YOLO model specifically includes:

[0100] (2.1) Replace C3k2 in the 6th and 8th layers of the YOLOv11 architecture with BEEM modules:

[0101] The C3k2 module has difficulty accurately segmenting edges when faced with targets with fuzzy edges, complex shape features, and easily confused with the background. The BEEM module is designed based on the Cross Stage Partial Network (CSP). After the input passes through a point-by-point convolution expansion channel, it is divided into two branches. One branch propagates forward, and the other branch passes through an innovative bottleneck block. The bottleneck block consists of a convolutional layer, a channel attention mechanism (CA), a global spatial attention mechanism (GSA), and a Transformer Self-Attention (TSA) jointly optimized. Finally, the two branches are spliced through the channels, and then the number of channels is restored by point-by-point convolution, so that the output tensor maintains the same spatial resolution as the input. By expanding the dimensions of the intermediate layers, the module extracts and mixes shallow local and deep global information, enhances feature representation, and improves the segmentation of renal lesions.

[0102] (2.2) Replace the C2PSA in the 10th layer of the YOLOv11 architecture with the MCFA module:

[0103] The MCFA module applies self-attention to both the vertical and horizontal axes based on axial attention. The input is split into two parallel self-attention branches, each undergoing multi-scale convolution to extract features of varying receptive fields. The point-by-point convolution then outputs the required attention calculation format (Q, K, V). After cross-fusion of the vertical and horizontal axes, the resulting outputs are residually connected with the original input to produce the final result. MCFA can capture global contextual information for segmenting objects with complex and variable edge shapes without excessive computational overhead.

[0104] (2.3) An innovative bounding box loss function is used to regress the bounding box parameters:

[0105] By adaptively combining the Normalized Wasserstein Distance (NWD) and the Complete-IoU loss function (CIoU), we construct a new loss function calculation method. This method dynamically adjusts the proportion of each component in the loss function during training, allowing small objects to be better regressed to the ground truth bounding box, improving small object detection and segmentation performance.

[0106] (2.4) Add shallow small target detection head:

[0107] YOLOv11's three traditional segmentation detection heads are insufficiently scaled to effectively detect very small objects. Small objects typically occupy a limited pixel area in an image and are often overwhelmed by larger primary features in higher-level feature maps. Therefore, this method adds an additional segmentation detection head to shallow feature maps to leverage more local information, thereby improving the detection accuracy of small objects.

[0108] (2.5) Training model:

[0109] By setting training parameters such as the number of training rounds and batch size, and iterating the model multiple times, we determined the optimal model, the kidney tumor feature-enhanced YOLO model, after multiple experiments.

[0110] Step (2.1) specifically includes:

[0111] Let F be the input feature map tensor. F is first mapped into two parts f1 and f2 with the same number of channels through point-by-point convolution. f1 then simplifies the channels through point-by-point convolution and captures the local context through 3×3 large kernel convolution. The formula is as follows:

[0112] f1,f2=split(Conv 1×1 (F)),

[0113] f1'=Conv3×3 (Conv 1×1 (f1)),

[0114] Among them, split means the segmentation of the channel, Conv 1×1 Represents point-by-point convolution, Conv 3×3 Represents a convolution with a convolution kernel of 3.

[0115] After the split, f1' immediately passes through a bottleneck block that integrates the channel attention mechanism, the global spatial attention mechanism, and the Transformer self-attention mechanism to obtain f1", and then channels are spliced with the shallow features of f2. Finally, the number of channels is restored by point-by-point convolution. The calculation formula is as follows:

[0116]

[0117] F out =Conv 1×1 (concat(f1',f2)),

[0118] Among them, GSA(*) represents the output of the global spatial attention mechanism, CA(*) represents the output of the channel attention mechanism, TSA(*) represents the output of the Transformer self-attention mechanism, and g represents the result after obtaining the weight on the channel through channel attention. Indicates multiplication, concat indicates channel concatenation, F out Represents the output of the fuzzy edge enhancement module.

[0119] The calculation model of each attention is:

[0120] (2.1.1) The channel attention mechanism performs average pooling and maximum pooling on the input X, extracts the dependencies between different channels through the dimension reduction convolution layer, then uses the ReLU activation function to enhance the expression, and generates the channel importance score through the restoration convolution. Finally, the two are combined and the output is activated. The formula is as follows:

[0121] CA(X)=δ(Conv 1×1up (β(Conv 1×1down (AP(X))))+Conv 1×1up (β(Conv 1×1down (MP(X))))),

[0122] Among them, X represents input, AP and MP represent average pooling and maximum pooling respectively, Conv 1×1down and Conv 1×1upThey represent dimensionality reduction and dimensionality increase using convolution with a convolution kernel of 1, β represents the ReLU activation function, δ represents the Sigmoid activation function, and CA(X) represents the output of the channel attention mechanism.

[0123] (2.1.2) The global spatial attention mechanism first linearly projects the input X into the query (Q), key (K), and value (V) required by the attention mechanism through three one-dimensional convolutions, then calculates the spatial attention, and finally connects the output with the input X residual. The formula is:

[0124] Q = Conv Q (X), K = Conv K (X), V = Conv V (X),

[0125]

[0126] GSA(X)=γO+X,

[0127] Among them, X represents the input, Conv Q 、Conv K and Conv V Represent the convolution kernels used to generate Q (query), K (key) and V (value), respectively. The Softmax function is used to normalize a set of real numbers. k is the scaling factor of the attention head, O represents the calculation result of the spatial attention mechanism, γ is a learnable scalar whose initial value is 0, and GSA(X) represents the output of the global spatial attention mechanism.

[0128] (2.1.3) The Transformer self-attention mechanism flattens the input X to generate feature representations of the query (Q), key (K), and value (V). It then calculates the attention weights through a dot product operation, aggregates the global feature information, and outputs the result. The formula is as follows:

[0129] Q=Reshape(X), K=Reshape(X), V=Reshape(X),

[0130]

[0131] Here, X represents the input, the Reshape function is used to flatten the input tensor, and TSA(X) represents the output of the Transformer self-attention mechanism.

[0132] Through the above steps, the BEEM module can take into account both shallow fine-grained edge details and deep global semantic information, enhance the ability to extract fuzzy, irregular, and morphologically diverse edges of kidney lesions, and significantly improve the boundary accuracy of instance segmentation of benign and malignant kidney tumors and cysts.

[0133] Step (2.2) specifically includes:

[0134] Input feature f with batch size B, number of channels C, image height H, and image width W input , dividing it into two parallel branches: the horizontal axis X and the vertical axis Y.

[0135] Taking the horizontal axis X as an example, after passing through the normalization layer (Norm) to stabilize the feature distribution, multi-scale convolution kernels are applied to this network to effectively capture contextual information at multiple scales. The same is true for the Y axis. The outputs of these convolutions are then aggregated and passed through a point-by-point convolution, as shown below:

[0136]

[0137] Among them, f input Indicates input, represents one-dimensional multi-scale convolution, i represents the multi-scale convolution label, and F1 and F2 represent two paths that capture multi-scale information along different spatial axes.

[0138] For F1 and F2, they are reshaped into the format required for attention calculation, and the results are Q1, K1, V1 (horizontal axis X) and Q2, K2, K2 (vertical axis Y). In the attention calculation stage, Q2 from the vertical axis is correlated with K1 and V1 from the horizontal axis, and Q1 from the horizontal axis is correlated with K2 and V2 from the vertical axis. Finally, the output is obtained by summing with the input, as shown below:

[0139] Q1, K1, V1 = Reshape(F1),

[0140] Q2, K2, V2 = Reshape(F2),

[0141]

[0142] f output =Conv 1×1 (out1)+Conv 1×1 (out2)+f input ,

[0143] Among them, Reshape is a common function in deep learning, which is used to flatten the input tensor. The Softmax function is used to normalize a set of real numbers. Out1 and out2 are the results of attention mechanism calculation and multi-dimensional fusion. input is the input of the module, f output is the output.

[0144] Step (2.3) specifically includes:

[0145] The innovative bounding box loss function NWD-CIoU consists of two parts: normalized Wasserstein distance and Complete-IoU bounding box loss function.

[0146] Normalized Wasserstein distance first models the bounding boxes as 2D Gaussian distributions, and then proposes a new metric called NWD to calculate the similarity between them through their corresponding Gaussian distributions:

[0147]

[0148] Where N is the Gaussian distribution modeled according to the predicted bounding box, N gt is a Gaussian distribution modeled after the true bounding box, Calculate the distance between two Gaussian distributions, C is a constant, L NWD Represents the NWD metric.

[0149] Complete-IoU bounding box loss function L CIoU After obtaining the prior box corresponding to each box, take out the predicted bounding box corresponding to the prior box, and use the real bounding box and the predicted bounding box to calculate the loss:

[0150]

[0151] Among them, IoU (Intersection over Union) calculates the ratio of the intersection and union of the predicted bounding box and the true bounding box, b and b gt denotes the center coordinates of the predicted bounding box and the true bounding box, ρ is the Euclidean distance, c represents the diagonal length of the minimum bounding box covering the two bounding boxes, α is the weight parameter, v measures the consistency of the aspect ratio, and L CIoU represents the CIoU loss.

[0152] Based on the concepts of CIoU and NWD, we have combined a new bounding box loss function (NWD-CIoU) to calculate it. This method dynamically adjusts the proportion of each component in the loss function during the training phase, allowing small objects to be better regressed to the true annotation box, improving the detection and segmentation performance of small objects. The specific formula is as follows:

[0153] L=λL CIoU +(1-λ)L NWD ,

[0154] Among them, λ represents the scaling factor and L represents the NWD-CIoU loss.

[0155] By calculating the loss function value of the network model in each round of iteration, the obtained loss function value converges to the minimum, and then the model is saved.

[0156] Step (3), kidney lesion classification and segmentation:

[0157] Select the trained model weights and adjust the Intersection over Union (IoU) threshold and confidence interval as needed to perform classification and segmentation tasks on kidney images to achieve the best detection results.

[0158] Figure 7 are three medical image pictures of kidney lesions randomly selected from the validation set. Figure 7 (a) is a picture of renal cyst. Figure 7 (b) is a diagram of kidney stones. Figure 7 (c) is a picture of a kidney tumor.

[0159] The method of the present invention can well handle the segmentation tasks of small targets and those with fuzzy edges that are easily confused with the background. The BEEM module enhances the feature extraction of edges that are easily confused with the background and fuzzy edges by fusing the triple attention mechanism with the residual connection. The MCFA well captures the global multi-scale and multi-dimensional information. The introduction of the new bounding box loss function calculation method enhances the model's detection effect on tiny kidney stones or renal cysts. Figure 8 It can be seen that the KTFE-YOLO proposed in this invention has high segmentation and detection accuracy.

[0160] Figure 9 In order to compare the performance of the existing YOLOv11 framework and the KTFE-YOLO model of the present invention through the heat map technology, the location with more red areas indicates that it is more likely to be the location of the lesion.

[0161] from Figure 9 (a) It can be seen that the existing YOLOv11 model is not accurate enough in detecting tumors, and its ability to extract edge features is poor.

[0162] from Figure 9 (b) It can be seen that the present invention enhances the feature extraction of tumor edges and improves segmentation accuracy by fusing shallow local information with deep enhanced global information and capturing global context semantic information.

[0163] A computer device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory. When the computer program is run, the processor executes the kidney lesion image analysis method based on the KTFE-YOLO model.

[0164] A computer-readable storage medium stores a program or instruction, which, when executed by a computer device, enables the computer device to execute the renal lesion image analysis method based on the KTFE-YOLO model.

[0165] A kidney lesion image analysis system based on the KTFE-YOLO model, including:

[0166] The data acquisition unit is used to acquire and process training data, annotate kidney images with lesions, and store them as YOLO format labels;

[0167] The model construction unit is used to build the KTFE-YOLO model. Based on the YOLOv11n-seg architecture, the C3k2 in the 6th and 8th layers of the backbone are replaced with a fuzzy edge enhancement module, and the C2PSA in the 10th layer is replaced with a multi-scale convolutional fusion attention mechanism module. The fuzzy edge enhancement module is based on a cross-stage partial network design. After the input passes through a point-by-point convolution expansion channel, it is divided into two branches. One branch is forward propagated, and the other branch passes through a bottleneck block. The bottleneck block is composed of a convolutional layer, a channel attention mechanism, a global spatial attention mechanism, and a Transformer self-attention mechanism. Finally, the two branches are spliced through the channels, and the number of channels is restored by point-by-point convolution to ensure that the output tensor maintains the same spatial resolution as the input.

[0168] A bounding box loss function construction unit, used to construct a bounding box loss function to regress bounding box parameters;

[0169] The training unit is used to optimize the parameters of the KTFE-YOLO model using the training data based on the constructed bounding box loss function;

[0170] Application unit, used to classify and segment kidney images using the optimized KTFE-YOLO model.

[0171] The segmentation and classification results of the KTFE-YOLO model proposed in this invention can alleviate the challenges faced by radiologists, such as heavy diagnostic tasks and the difficulty in balancing efficiency and accuracy. At the same time, this invention can assist doctors in clinical diagnosis, improve diagnostic efficiency and accuracy, and the lesion segmentation results can also provide auxiliary work for oncologists in target area delineation.

[0172] The proposed KTFE-YOLO model enables automatic classification and segmentation of renal lesions within milliseconds, significantly reducing the burden on radiologists and addressing challenges such as high workload and the difficulty in balancing efficiency and accuracy. The high-precision segmentation results generated by the model not only provide an intuitive reference for clinical diagnosis, improving diagnostic speed and accuracy, but can also be used for target volume delineation in radiotherapy departments, providing reliable boundary information and volumetric assessments for radiotherapy treatment planning, further improving the quality and efficiency of multidisciplinary collaborative treatment.

[0173] In addition, it should be understood that after reading the above description of the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the claims attached to this application.

Claims

1. A method for analyzing renal lesion images based on the KTFE-YOLO model, characterized in that: include: Obtain training data and process it: annotate kidney images with lesions and store them in YOLO format. Build the KTFE-YOLO model: Based on the YOLOv11n-seg architecture, replace the C3k2 in the 6th and 8th layers of the backbone with a fuzzy edge enhancement module, and replace the C2PSA in the 10th layer with a multi-scale convolutional fusion attention mechanism module; The fuzzy edge enhancement module is based on a cross-stage partial network design. After the input passes through a point-by-point convolution expansion channel, it is divided into two branches. One branch propagates forward, and the other branch passes through a bottleneck block. This bottleneck block consists of a convolutional layer, a channel attention mechanism, a global spatial attention mechanism, and a Transformer self-attention mechanism. Finally, the two branches are spliced together through channels, and the number of channels is restored by point-by-point convolution, so that the output tensor maintains the same spatial resolution as the input. Construct a bounding box loss function to regress bounding box parameters; Based on the constructed bounding box loss function, the parameters of the KTFE-YOLO model are optimized using the training data; The optimized KTFE-YOLO model is used to classify and segment kidney images.

2. The kidney lesion image analysis method based on the KTFE-YOLO model according to claim 1, characterized in that: The kidney image includes one or more of a kidney CT image and a kidney MRI image.

3. The kidney lesion image analysis method based on the KTFE-YOLO model according to claim 1, characterized in that: The fuzzy edge enhancement module: Let F be the input feature map tensor. F is first mapped into two parts f1 and f2 with the same number of channels through point-by-point convolution. f1 then simplifies the channels through point-by-point convolution and captures the local context through 3×3 large kernel convolution. The formula is as follows: f1,f2=split(Conv 1×1 (F)) f1'=Conv 3×3 (Conv 1×1 (f1)) Among them, split means the segmentation of the channel, Conv 1×1 Represents point-by-point convolution, Conv 3×3 Indicates convolution with a convolution kernel of 3; After segmentation, f1' is extracted through a bottleneck block that integrates the channel attention mechanism, the global spatial attention mechanism, and the Transformer self-attention mechanism to obtain deep features f1", and is channel-joined with the shallow features of f2. Finally, the number of channels is restored by point-by-point convolution. The calculation formula is as follows: F out =Conv 1×1 (concat(f1”,f2)) Among them, GSA(*) represents the output of the global spatial attention mechanism, CA(*) represents the output of the channel attention mechanism, TSA(*) represents the output of the Transformer self-attention mechanism, and g represents the result after obtaining the weight on the channel through channel attention. Indicates multiplication, concat indicates channel concatenation, F out Represents the output of the fuzzy edge enhancement module.

4. The kidney lesion image analysis method based on the KTFE-YOLO model according to claim 1, characterized in that: The multi-scale convolution fusion attention mechanism module applies self-attention to the vertical and horizontal axes based on axial attention. After the input is divided into two parallel self-attention branches, they undergo multi-scale convolution to extract features of receptive fields of different sizes. The point-by-point convolution output is then converted into the format required for calculating attention. After cross-fusion of the vertical and horizontal axes, the outputs are residually connected with the original input to output the final result.

5. The kidney lesion image analysis method based on the KTFE-YOLO model according to claim 1, characterized in that: In the construction of the KTFE-YOLO model, an additional segmentation detection head is added to the shallow feature map.

6. The method for analyzing renal lesion images based on the KTFE-YOLO model according to claim 1, wherein: The constructed bounding box loss function includes an innovative loss function NWD-CIoU that combines the normalized Wasserstein distance and the Complete-IoU bounding box loss function.

7. The method for analyzing renal lesion images based on the KTFE-YOLO model according to claim 1, wherein: In the classification and segmentation of kidney images using the optimized KTFE-YOLO model, users can adjust the intersection-over-union ratio threshold, confidence interval, image size, whether to retain the classification label, whether to display the confidence level, and whether to display the detection box as needed. After the task is completed, the user can choose to save the image with the labeled results.

8. A computer device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, wherein: When the computer program is run, the processor is enabled to execute the kidney lesion image analysis method based on the KTFE-YOLO model according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a computer device, the computer device executes the kidney lesion image analysis method based on the KTFE-YOLO model according to any one of claims 1 to 7.

10. A kidney lesion image analysis system based on the KTFE-YOLO model, characterized in that: include: The data acquisition unit is used to acquire and process training data, annotate kidney images with lesions, and store them as YOLO format labels; The model building unit is used to build the KTFE-YOLO model. Based on the YOLOv11n-seg architecture, the C3k2 in the 6th and 8th layers of the backbone are replaced with a fuzzy edge enhancement module, and the C2PSA in the 10th layer is replaced with a multi-scale convolutional fusion attention mechanism module; The fuzzy edge enhancement module is based on a cross-stage partial network design. After the input passes through a point-by-point convolution expansion channel, it is divided into two branches. One branch propagates forward, and the other branch passes through a bottleneck block. This bottleneck block consists of a convolutional layer, a channel attention mechanism, a global spatial attention mechanism, and a Transformer self-attention mechanism. Finally, the two branches are spliced together through channels, and the number of channels is restored by point-by-point convolution, so that the output tensor maintains the same spatial resolution as the input. A bounding box loss function construction unit, used to construct a bounding box loss function to regress bounding box parameters; The training unit is used to optimize the parameters of the KTFE-YOLO model using the training data based on the constructed bounding box loss function; Application unit, used to classify and segment kidney images using the optimized KTFE-YOLO model.

Images