Medical image depth segmentation method based on B-spline function and Transform

Abstract

The invention provides a medical image segmentation method based on a B-spline function and a Transform, and solves the technical problem that in the Transform, the parameter quantity of the MLP is huge, and the MLP generally does not have interpretability, so that the generalization ability and the characterization ability of a model are weak. According to the technical scheme, the method comprises the following steps of 1, preprocessing an input medical image; step 2, replacing a self-attention mechanism in Transfomer and a weight parameter of an MLP in a feedforward neural network by using a B-spline-based spline function; step 3, inputting the preprocessed image data set into an encoder and decoder network based on a B-spline function for training to obtain an optimal model; and step 4, after training is completed, inputting the medical image verification set into the obtained optimal model. According to the method, the focus can be segmented more accurately, and the accuracy and robustness are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and particularly relates to a medical image segmentation method based on the Kolmogorov - Arnold representation theorem Background Art

[0002] Medical image segmentation technology is one of the core tasks in the field of medical image analysis. Its goal is to separate the regions of interest (such as organs, lesions, tumors, etc.) in medical images from the background, providing important support for subsequent diagnosis, treatment plan formulation, and disease monitoring. In recent years, with the improvement of the resolution and imaging quality of medical imaging devices (such as MRI, CT, ultrasound, etc.), the amount of medical image data has increased exponentially. Traditional manual segmentation methods, due to their long time consumption, dependence on expert experience, and susceptibility to subjective factors, can no longer meet the actual clinical needs. Therefore, automated and intelligent medical image segmentation technology has become a research hotspot

[0003] Traditional medical image segmentation methods include threshold - based, region - growing, edge - detection, and watershed - transformation methods, etc. These methods usually rely on low - level features of images (such as gray - scale values, gradient information, etc.), are sensitive to changes in image quality, noise, and target shape, and are difficult to handle complex medical image scenarios. With the introduction of machine learning, segmentation methods based on statistical modeling and classifiers (such as support vector machines and random forests) have provided new ideas for improving segmentation performance. However, these methods usually require the design of complex feature - extraction processes and have limited adaptability to specific tasks

[0004] In recent years, the rise of deep learning, especially convolutional neural networks (CNNs), has brought revolutionary breakthroughs to medical image segmentation. Deep - learning methods can automatically extract multi - level image features from a large amount of labeled data through an end - to - end learning mode, thus achieving precise segmentation of complex structures. Among them, the U - Net architecture, as a milestone model in medical image segmentation, effectively captures multi - scale features and preserves image detail information through a symmetric encoder - decoder structure and skip - connection design. Since then, various improved U - Net variants (such as Attention U - Net, 3D U - Net, Dense U - Net, etc.) have emerged continuously, further improving the accuracy and robustness of segmentation

[0005] Although deep learning techniques have made significant progress in medical image segmentation, there are still many challenges in practical applications. On the one hand, the diversity of different imaging devices and imaging protocols leads to significant differences in data distribution, restricting the generalization ability of the model. On the other hand, medical images usually contain more noise, artifacts, and blurred boundaries, which pose difficulties for the accurate segmentation of the target area. In addition, in some application scenarios (such as tumor detection), extremely high reliability requirements are placed on the segmentation results. Therefore, uncertainty estimation and model interpretability need to be introduced to assist clinical decision-making.

[0006] How to solve the above technical problems is the subject of this invention. Summary of the Invention

[0007] The present invention provides a deep medical image segmentation method based on B-spline functions and Transformer. The technical problem to be solved is how to simplify the calculation of Transformer without changing the number of parameters, thereby further enhancing the generalization ability of the original U-net model.

[0008] To achieve the above object of the invention, the technical solution adopted by the present invention is: a medical image segmentation method based on an encoder-decoder of B-spline functions, comprising the following steps:

[0009] Step 1: Preprocess the input image dataset, perform data augmentation, and divide the dataset into a training set, a test set, and a validation set in a ratio of 8:1:1;

[0010] Step 2: Construct a feed-forward neural network based on B-spline functions, the encoder-decoder network based on B-spline functions and Transfomer. Use B-spline functions to replace the self-attention mechanism in Transfomer and the weight parameters of MLP in the feed-forward neural network. Directly connect two neurons through spline functions to further enhance the generalization ability and robustness of the network, and solve the significant differences in data distribution caused by the diversity of different imaging devices and imaging protocols.

[0011] Step 3: Input the preprocessed image dataset into the encoder and decoder networks based on B-spline functions for training to obtain an optimal model

[0012] Step 4: After training is completed, input the image dataset into the obtained optimal model to detect the segmentation results of the medical image dataset

[0013] The specific process in the above Step 1 is as follows:

[0014] Step 1.1: Divide the medical image dataset into a training set and a validation set according to a ratio of 8:1:1.

[0015] Step 1.2: Use cv2 to load the images and masks in the training set and test set of the medical image dataset, and load the sizes of the images and masks to 256×256.

[0016] Before the mask is converted to a PIL image, it is grayscaled, and then the pixel values are normalized to [0,1] by dividing by 255. The grayscaling and normalization formulas are as follows:

[0017]

[0018] Among them, R, G, and B are the values of the red, green, and blue channels of the image respectively, and Out is the image after grayscaling and normalization.

[0019] Step 1.4: Convert the image from BGR format to RGB format.

[0020] Step 1.5: Perform operations such as random scaling, random horizontal flipping, random vertical flipping, random cropping, and normalization processing on the training set data in sequence to perform data augmentation and preprocessing work. This data augmentation process not only improves the robustness and generalization ability of the model by introducing diversity and uncertainty, but also increases the attention to target details, making the model more adaptable to slight changes in the data distribution. Mean=(0.402, 0.402, 0.402), Std=(0.298, 0.298, 0.298), and the normalization processing formula is as follows:

[0021]

[0022] Among them, Input is the input image, Mean is the mean value, Std is the standard deviation, and Normalized is the image after normalization processing.

[0023] In the specific step 2, it is as follows:

[0024] Step 2.1: In the encoder of U-net, due to the advantage of Transfomer in capturing long-range dependencies, we combine U-net with Transfomer to enhance the global information modeling ability of U-net. We replace the traditional convolution operation with a Transfomer encoder block. Each encoder consists of a self-attention mechanism and a feed-forward neural network. The self-attention mechanism helps the model capture the global information in the image and enhances the model's understanding of long-range dependencies and global context.

[0025] Step 2.2: Change the weight representation in the original multi-layer perceptron (MLP) to a B-spline function instead of a real number. The function directly connects two neurons, replacing the linear weight in the MLP.

[0026] Any continuous function f(x1,…,x n ) can be represented as a nested combination of a finite number of univariate functions (as shown in the following figure, where and Φ q are both univariate functions)

[0027]

[0028] For a smooth f:[0,1] n →R, as follows

[0029]

[0030] where, x p represents the p-th component of the vector x and contributes a term to the sum of the q-th outer function. The index q is used to traverse each component of the outer function Φ. Therefore, there is a univariate function processing the p-th component of the input vector x and contributing a term to the sum of the q-th outer function. The theorem states that any multivariate f can be represented by 2n + 1 such outer functions - each outer function Φ q is a univariate function that acts on the sum composed of the outputs of the inner univariate functions .

[0031] Each function can be represented by univariate functions and sums. Therefore, high-dimensional functions can thus be reduced to learning a polynomial number of one-dimensional functions. However, the one-dimensional functions are smooth or even fractal, so they may not be learnable in practice.

[0032] For this, we use the B-spline function of a smooth curve, which is composed of multiple piecewise polynomials spliced together, and each piecewise polynomial is defined by a set of control points. For B-spline, the function has the same continuity at the knots within its domain, and its polynomial representation can be expressed by the Cox-de Boor recurrence formula

[0033] Step 2.3: Facing the supervised learning task composed of the input-output pairs {x i ,y i}, find f such that for all data points, y i ≈f(x i ), so that we only need to find the appropriate univariate functions and Φ qThat's all. Each one-dimensional function is thus parameterized by a B-spline curve, where the learnable parameter coefficients of the local B-spline basis functions

[0034] In the MLP, a layer (consisting of a linear transformation and a non-linear transformation) is defined, and stacking more layers makes the network deeper. The KAN layer with the output dimension can be defined as a one-dimensional function matrix

[0035] Φ = {φ q,p},p = 1, 2, …, n in , q = 1, 2…, n out ,

[0036] where the function φ q,p has trainable parameters, and the internal function φ q,p After forming a KAN layer, where the input dimension n in = n, and the output dimension n out = 2n + 1. Each input undergoes such a set of dimensionality transformations to fully capture the input feature information and transform it into an intermediate representation.

[0037] The external function Φ q forms a KAN layer, where the input dimension n in = 2n + 1, and the output dimension n out = 1. This layer integrates all the outputs of the internal function layer to form the final output.

[0038] The grid step size is calculated based on the initialized grid start and end ranges and the number of segments of the grid. Then, the grid points are generated using the order of the B-spline basis functions and extended to all channels. Next, the input feature x is locally fitted using the B-spline basis functions originally constructed with the grid points. First, the B-spline basis functions of order 0 are constructed, and then the higher-order B-spline basis functions are recursively defined. Finally, the B-spline basis function matrix is output.

[0039] The process is formulated as follows:

[0040]

[0041] grid[i] = grid range [0] + i·h, i = -s, -s + 1, …, grid size + s

[0042] grid = grid.expand(in features , -1)

[0043]

[0044] (batch size, in features, grid size + spline order)

[0045] where h is the grid step size, grid_range is the range of the spline function grid, grid_size is the number of segments of the grid, s is the order of the B-spline basis function, i is the index of the grid point, and grid is the complete array of grid points, representing the knots of the discrete spline function. Represents the zero-order (constant order) spline basis function. The k-th order B-spline basis function batch size represents the number of input samples, and in features represents the number of input features per sample.

[0046] grid size + spline order represents the length of the support interval of each spline basis function

[0047] Step 2.4:

[0048] Optimize the residual activation function, and the activation function φ(x) is the sum of the basis function b(x) and the spline function

[0049] φ(x) = w(b(x) + spline(x))

[0050] Set b(x) to

[0051] b(x) = silu(x) = x / (1 + e -x )

[0052] spline(x) is parameterized as a linear combination of B-splines, where c i is trainable

[0053]

[0054] Finally, w is well combined with b(x) and the spline function, and at the same time, the overall amplitude of the activation function can still be better controlled through the w factor

[0055] The initial value of each activation function spline(x) ≈ 0, while w is initialized according to Xavier initialization. For each grid in real-time according to its input activation, solve the problem that splines are defined on a bounded region, but the activation function may exceed the fixed region during training.

[0056] First, perform piecewise polynomial fitting of the input x through the B-spline basis function and the learned weight W spline, and then the above operations are implemented through matrix multiplication to form a tensor representation. Then the input x is processed by an activation function, and the activated input is processed by a linear transformation. Finally, the basic linear features and piecewise fitting features are fused to generate the final output.

[0057] The flow formula is as follows:

[0058]

[0059] y spline = B(x)·W spline [i]

[0060] x act = SiLU(x)

[0061] y base = W base ·x act

[0062] y = y base + y spl i ne

[0063] where y spline is the spline fitting output, b i (x) represents the B-spline basis function, which is the eigenvalue of the input x on the grid i and represents the local feature fitting result of x. W spline [i] is the learnable weight corresponding to the i-th grid, representing the weight contribution of the feature under the action of the i-th spline basis function. grid_size is the number of segments of the spline grid, B(x) is the matrix of B-spline basis function values of the input x, SiLU(x) represents the activation function, and x act is the input feature after non-linear transformation by the activation function, and y base represents the global feature extracted by the global linear transformation. y represents the final feature output, which is composed of the weighted sum of the global feature and the local feature.

[0064] Step 3.1: The input image I is first mapped into the discrete spline grid space through the gridification process to generate the feature representation input to the spline encoder

[0065] B(x) = B-spline(x,grid)

[0066] where B(x) represents the B-spline basis function matrix, which represents the spline expansion of the input x on the grid grid. x is the input feature, and grid is the spline grid, which defines the range and number of segments of the discrete interval

[0067] Then, the weight matrix W of the spline encoder is used spline, Calculate the linear combination of spline basis functions and extract local features

[0068] F spline = B(x)·W spline

[0069] where F spline is the local feature representation, and W spline represents the weight matrix corresponding to the B-spline

[0070] Meanwhile, the input is used to obtain the global feature through a basic linear transformation

[0071] F base = SiLU(x)·W base

[0072] where F base represents the global feature, SiLU(x) is the non-linear activation of the input feature x, and W base is the weight matrix of the global feature.

[0073] Add and fuse the local feature F spline and the global feature F base and re-enter them into the spline encoder to generate the final input feature

[0074] F enc = F spline + F base

[0075] The encoder extracts multi-level features in stages

[0076]

[0077] where F enc represents the feature map of the input feature output by the i-th layer of the encoder, EncoderStage (i) represents the i-th layer in the encoder, which is used to extract multi-scale features, represents the feature map output by the i-th layer of the encoder

[0078] Step 3.2: The decoder at each stage takes the output feature of the encoder and the upsampled feature of the corresponding stage as inputs and reconstructs the input features using the spline structure

[0079]

[0080] represents the feature map output by the i-th layer of the encoder, represents the result of upsampling the output feature of the previous-level decoder, DecoderStage (i)Denote the $i$-th layer in the decoder, which is used to fuse the features of the encoder. The feature map output by the $i$-th layer of the decoder.

[0081] Align the spline-fitted features and the upsampled features in the channel dimension and then add them element-wise.

[0082]

[0083] Where Denote the features output by the $i$-th layer of the decoder. The features obtained by upsampling the output of the previous layer of the decoder. Denote the fused features.

[0084] For the original features Perform attention enhancement, extract the spatial attention weights.

[0085]

[0086] Where Perform global average pooling on the feature map to generate global information. Perform global max pooling on the feature map to generate global information, Conv 1×1 Use a 1×1 convolution to reduce the feature dimension, $\sigma$ represents the activation function, which is used to normalize the weights, $M$ s Represents the spatial attention weights.

[0087] Perform global max pooling on the feature map to generate global information and extract the channel attention weights.

[0088]

[0089] Where Perform global average pooling on the feature map to generate global information. Perform global max pooling on the feature map to generate global information, KAN represents the KAN neural network module based on the B-spline function, $M$ c Represents the channel attention weights.

[0090] Weight the features using the attention weights.

[0091]

[0092] Where Denote the spatially attention-weighted features. Denote the channel attention-weighted features.

[0093] Denote the final fused features.

[0094] Step 3.3: During the training process, compare the error between the decoder output and the target image, which is the reconstruction loss

[0095]

[0096] where is the reconstruction loss, y pred is the prediction result output by the decoder, that is, the feature map output by the network, y true is the target true label, that is, the actual annotation image of image segmentation, and DiceLoss is a loss form of the Dice coefficient, which is used to measure the similarity between two images

[0097] Perform regularization loss, including L1 regularization and entropy regularization

[0098]

[0099] where is the regularization loss, ‖W spline ‖1 is to perform L1 regularization on the spline weights to constrain the sum of the absolute values of the weights represents the entropy loss, which is used to measure the distribution complexity of the feature map, and λ entropy is the weight hyperparameter that controls the entropy regularization term and is used to balance the contributions of the entropy loss and L1 regularization to the total loss

[0100] Calculate the total loss

[0101]

[0102] where represents the total loss function, which combines the reconstruction loss and the regularization loss. λ is the weight hyperparameter used to balance the contributions of the reconstruction loss and the regularization loss

[0103] Calculate the gradient through backpropagation, update the network weights, and at the same time adjust the positions of the spline grid points according to the input data distribution to ensure that the spline function can adapt to the distribution

[0104]

[0105] where W are all the weight parameters in the network is the total loss is the gradient of the weight parameter W, and η is the learning rate that controls the step size of weight update

[0106] The trained encoder and decoder networks based on B-spline functions can accurately capture global and local features, achieve high-quality image feature extraction and reconstruction, and finally obtain the optimal model

[0107] Step 3.4 Use Precision, Accuracy, Balanced Accuracy, Recall, and F1-score as the evaluation metrics of the model to evaluate the training and test results. The functional expressions of each evaluation metric are as follows:

[0108]

[0109] Among them, TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative respectively. Precision represents the precision rate, which measures the proportion of samples that are actually positive among the samples predicted as positive by the model. Accuracy represents the accuracy rate, which measures the proportion of samples correctly predicted by the model in the total samples. Recall represents the recall rate, which measures the proportion of all actually positive samples correctly identified by the model. F1-score is the harmonic mean of precision and recall, used to balance between these two metrics, and is particularly suitable for class imbalance problems.

[0110] Step 4: After the model training is completed, use the optimal model during training to verify on the medical image validation set to detect the segmentation effect in polyp medical images.

[0111] Compared with the existing technologies, the beneficial effects of the technical solution of the present invention are:

[0112] 1. The present invention combines the characteristics and advantages of different medical image segmentation methods and designs a relatively lightweight fully automatic medical image segmentation network, which improves the lesion segmentation performance of the model without increasing many parameters and computational complexity.

[0113] 2. By introducing the B-spline function and the fusion module based on the attention mechanism, this technical solution can capture global and local features more efficiently. Compared with the existing technologies, it can not only highlight the global semantic information but also retain the detailed features, thus improving the feature expression ability and the overall performance of the model. The multi-layer perceptron (MLP) is a basic theoretical module of deep learning and is currently the default model that can be used to approximate non-linear functions, and its representational ability has been proved by the universal approximation theorem. However, MLP also has obvious disadvantages. For example, in Transformer, the number of parameters of MLP is huge and it usually does not have interpretability.

[0114] 3. The present invention proposes a dynamic adaptive B-spline module based on channel attention and spatial attention. By introducing the dynamic adjustment of piecewise polynomials and a multi-layer attention mechanism, this module achieves precise modeling and global optimization of feature extraction. First, the module uses a dynamic grid adjustment strategy to adaptively adjust the positions of spline grid points according to the distribution of input data, ensuring that the B-spline function can flexibly adapt to the diversity of different features. During the feature extraction process, the channel attention mechanism extracts significant feature information in the channel dimension through global average pooling and max pooling, and generates channel attention weights through a multi-layer perceptron for weighting, thus achieving the enhancement of important feature channels. At the same time, the spatial attention mechanism fuses global and local features in the spatial dimension through convolution operations to generate spatial attention weights, improving the focusing ability on key regions. Finally, through the element-wise fusion of channel and spatial weighted features, the dynamic adaptive adjustment model dynamically adapts the focus of attention on the input features, achieving the efficient combination of global information and local details. This method not only enhances the adaptability of the model to diverse data based on the dynamic adjustment of spline grid points, but also reduces the computational complexity through lightweight design, making it suitable for resource-constrained application scenarios. Compared with the prior art, the present invention demonstrates higher segmentation accuracy, better edge detail processing ability, and stronger robustness in scenarios such as complex background segmentation and low-contrast medical image analysis, providing important technical support for tasks such as object detection and segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, but do not constitute a limitation to the present invention.

[0116] Figure 1 It is a schematic diagram of the overall process of the embodiment of the present invention.

[0117] Figure 2 It is a schematic diagram of the B-spline feedforward neural network in the present invention.

[0118] Figure 3 It is a schematic diagram of an nth-order spline function.

[0119] Figure 4 It is a complete network architecture diagram of the attention mechanism in the present invention.

[0120] Figure 5 It is an Encoder architecture diagram of the attention mechanism in the present invention.

[0121] Figure 6 It is an Encoder architecture diagram of the combination of the spline function and the transformer in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0122] In order to make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0123] Embodiment 1

[0124] Figure 1 It is a schematic diagram of the overall process of the method for this embodiment. First, the medical image dataset is divided into a training set, a test set, and a validation set according to a ratio of 8:1:1; subsequently, the images in the training set are preprocessed and data-augmented. Next, the training parameters based on the B-spline function and Transformer are set. Then, the model is trained on the preprocessed and data-augmented training set, and the optimal model parameters are saved. After the training is completed, the optimal model parameters are loaded, and the segmentation performance of the model is tested on the validation set.

[0125] The specific implementation steps of the above method are as follows

[0126] Step 1.1: Divide the medical image dataset into a training set and a validation set according to a ratio of 8:1:1.

[0127] Step 1.2: Use cv2 to load the images and masks in the training set and the test set of the medical image dataset, and load the sizes of the images and masks to 256×256.

[0128] Step 1.3: The mask is grayscaled before being converted into a PIL image, and then the pixel values are normalized to [0,1] by dividing by 255. The grayscaling and normalization formulas are as follows:

[0129]

[0130] Among them, R, G, and B are the values of the red, green, and blue channels of the image respectively, and Out is the image after grayscaling and normalization.

[0131] Step 1.4: Convert the image from BGR format to RGB format.

[0132] Step 1.5: Perform operations such as random scaling, random horizontal flipping, random vertical flipping, random cropping, and normalization processing on the training set data in sequence to perform data augmentation and preprocessing work. This data augmentation process not only improves the robustness and generalization ability of the model by introducing diversity and uncertainty, but also increases the attention to the target details, making the model more adaptable to slight changes in the data distribution. Mean=(0.402, 0.402, 0.402), Std=(0.298, 0.298, 0.298), and the normalization processing formula is as follows:

[0133]

[0134] Among them, Input is the input image, Mean is the mean value, Std is the standard deviation, and Normalized is the image after normalization processing.

[0135] Specifically, in the above-mentioned step 2:

[0136] Step 2.1: In the encoder of U-net, due to the advantage of Transfomer in capturing long-range dependencies, we combine U-net with Transfomer to enhance the global information modeling ability of U-net. We replace the traditional convolution operation with the Transfomer encoder block. Each encoder consists of a self-attention mechanism and a feed-forward neural network. The self-attention mechanism helps the model capture the global information in the image and enhances the model's understanding of long-range dependencies and global context.

[0137] Step 2.2: Change the weight representation in the original multi-layer perceptron (MLP) to a B-spline function instead of a real number. The function directly connects two neurons to replace the linear weight in the MLP.

[0138] Any continuous function f(x1,…,x n ) can be expressed as a nested combination of a finite number of univariate functions (as shown in the following figure, where and Φ q are both univariate functions)

[0139]

[0140] For a smooth f:[0,1] n →R, as follows

[0141]

[0142] where, x p represents the p-th component of the vector x and contributes a term to the summation of the q-th external function. The index q is used to traverse each component of the external function Φ. Therefore, there is a univariate function processing the p-th component of the input vector x and contributing a term to the summation of the q-th external function. The theorem states that any multivariate f can be represented by 2n + 1 such external functions - each external function Φ q is a univariate function that acts on the summation composed of the outputs of the internal univariate functions to represent any multivariate f.

[0143] Each function can be represented by unary functions and summation. Therefore, high-dimensional functions can thus be reduced to learning a polynomial number of one-dimensional functions. However, one-dimensional functions can be smooth or even fractal, and thus may not be learnable in practice.

[0144] For this, we use the B-spline function of a smooth curve, which is composed of multiple piecewise polynomials spliced together, and each piecewise polynomial is defined by a set of control points. For B-spline, the function has the same continuity at the knots within its domain, and its polynomial representation can be expressed by the Cox-de Boor recurrence formula.

[0145] Step 2.3: Facing the supervised learning task composed of the input-output task pair {x i , y i}, find f such that for all data points, y i ≈ f(x i ). Thus, it only needs to find appropriate unary functions and Φ q . Thus, each one-dimensional function parameterizes the B-spline curve, where the learnable parameter coefficients

[0146] In the MLP, define a layer (composed of a linear transformation and a non-linear transformation), stack more layers to make the network deeper. The KAN layer with an output dimension can be defined as a one-dimensional function matrix

[0147] Φ = {φ q,p}, p = 1, 2, …, n in , q = 1, 2, …, n out ,

[0148] where the function φ q,p has trainable parameters. After the internal function φ q,p forms a KAN layer, where the input dimension n in = n, and the output dimension n out = 2n + 1. Each input passes through such a set of dimensionality conversions to fully capture the input feature information and transform it into an intermediate representation.

[0149] The external function Φ q forms a KAN layer, where the input dimension n in = 2n + 1, and the output dimension n out = 1. This layer integrates all the outputs of the internal function layer to form the final output.

[0150] Calculate the grid step size based on the initialized grid start and end ranges and the number of segments of the grid. Then generate grid points using the order of the B-spline basis functions and extend them to all channels. Next, use the B-spline basis functions originally constructed with the grid points to locally fit the input feature x. First, construct the B-spline basis functions of order 0, then recursively define the higher-order B-spline basis functions, and finally output the B-spline basis function matrix.

[0151] The process is formulated as follows:

[0152]

[0153] grid[i] = grid range [0] + i·h, i = -s, -s + 1, …, grid size + s

[0154] grid = grid.expand(in features , -1)

[0155]

[0156] (batch size, in features, grid size + spline order)

[0157] where h is the grid step size, grid_range is the range of the spline function grid, grid_size is the number of segments of the grid, s is the order of the B-spline basis functions, i is the index of the grid points, and grid is the complete array of grid points, representing the nodes of the discrete spline function. Represents the B-spline basis functions of order 0 (constant order). The B-spline basis functions of order k, batch size represents the number of input samples, and in features represents the number of input features of each sample.

[0158] grid size + spline order represents the length of the support interval of each B-spline basis function

[0159] Step 2.4:

[0160] Optimize the residual activation function. The activation function φ(x) is the sum of the basis function b(x) and the spline function

[0161] φ(x) = w(b(x) + spline(x))

[0162] Set b(x) to

[0163] b(x) = silu(x) = x / (1 + e -x )

[0164] The spline(x) is parameterized as a linear combination of B-splines, where c i is trainable

[0165]

[0166] Finally, w is well combined with b(x) and the spline function, and the overall amplitude of the activation function can still be better controlled by the w factor

[0167] The initial value of each activation function is spline(x) ≈ 0, while w is initialized according to Xavier initialization. Each grid is activated in real time according to its input, solving the problem that splines are defined on a bounded region, but the activation function may exceed the fixed region during training

[0168] First, a piecewise polynomial fitting of the input x is performed through B-spline basis functions and the learned weights W spline , and then the above operations are implemented through matrix multiplication to form a tensor representation. Then the input x is processed by the activation function, the activated input is processed by a linear transformation, and finally the basic linear features and piecewise fitting features are fused to generate the final output

[0169] The flow formula is as follows

[0170] y spline = B(x) · W spline [i]

[0171] x act = SiLU(x)

[0172] y base = W base · x act

[0173] y = y base + y spline

[0174] where y spline is the spline fitting output, b i (x) represents the B-spline basis function, which is the eigenvalue of the input x on the grid i and represents the local feature fitting result of x. W spline [i] is the learnable weight corresponding to the i-th grid, representing the weight contribution of the feature under the action of the i-th spline basis function. grid_size is the number of segments of the spline grid, B(x) is the matrix of B-spline basis function values of the input x, SiLU(x) represents the activation function, x actThe input features y after non-linear transformation by the activation function base represents the global features extracted by global linear transformation, and y represents the final feature output, which is composed of the weighted sum of global features and local features.

[0175] Step 3.1: The input image I is first mapped into a discrete spline grid space through a grid process to generate a feature representation as the input to the spline encoder

[0176] B(x) = B-spline(x, grid)

[0177] where B(x) represents the B-spline basis function matrix, representing the spline expansion of the input x on the grid grid, x is the input feature, and grid is the spline grid, which defines the range of the discrete interval and the number of segments

[0178] Then, using the weight matrix W of the spline encoder spline , calculate the linear combination of the spline basis functions to extract local features

[0179] F spline = B(x) · W spline

[0180] where F spline is the local feature representation, and W spline represents the weight matrix corresponding to the B-spline

[0181] At the same time, the input obtains global features through a basic linear transformation

[0182] F base = SiLU(x) · W base

[0183] where F base represents the global feature, SiLU(x) is the non-linear activation of the input feature x, and W base is the weight matrix of the global feature.

[0184] Add and fuse the local feature F spline and the global feature F base , and re-enter the spline encoder to generate the final input feature

[0185] F enc = F spline + F base

[0186] The encoder extracts multi-level features in stages

[0187]

[0188] where F encRepresents the input feature map of the i-th layer output of the encoder, EncoderStage (i) Represents the i-th layer in the encoder, used to extract multi-scale features Represents the feature map output by the i-th layer of the encoder

[0189] Step 3.2: The decoder of each stage takes the output features of the encoder And the upsampled features of the corresponding stage As inputs, and uses a spline structure to reconstruct the input features

[0190]

[0191] Represents the feature map output by the i-th layer of the encoder Represents the result after upsampling the output features of the previous decoder stage, DecoderStage (i) Represents the i-th layer in the decoder, used to fuse the features of the encoder The feature map output by the i-th layer of the decoder

[0192] Align the spline fitting features and the upsampled features in the channel dimension and add them element by element

[0193]

[0194] Where Represents the output features of the i-th layer of the decoder The features obtained by upsampling the output of the previous decoder layer Represents the fused features

[0195] Enhance the attention of the original features Extract spatial attention weights

[0196]

[0197] Where Perform global average pooling on the feature map to generate global information Perform global max pooling on the feature map to generate global information, Conv 1×1 Use a 1×1 convolution to reduce the feature dimension, σ represents the activation function, used to normalize the weights, M s Represents the spatial attention weight

[0198] Perform global max pooling on the feature map to generate global information and extract channel attention weights

[0199]

[0200] Where Perform global average pooling on the feature map to generate global information. Perform global max pooling on the feature map to generate global information. KAN represents the KAN neural network module based on the B-spline function, and M c represents the channel attention weight

[0201] Use the attention weight to weight the features

[0202]

[0203] where represents the spatially attention-weighted feature, represents the channel attention-weighted feature,

[0204] represents the final fused feature.

[0205] Step 3.3: During training, compare the error between the decoder output and the target image, the reconstruction loss

[0206]

[0207] where is the reconstruction loss, y pred is the predicted result of the decoder output, that is, the feature map output by the network, y true is the target true label, that is, the actual annotation image of image segmentation. DiceLoss is a loss form of the Dice coefficient, which is used to measure the similarity between two images

[0208] Perform regularization loss, including L1 regularization and entropy regularization

[0209]

[0210] where is the regularization loss, ‖W spline ‖1 is to perform L1 regularization on the spline weight, which constrains the sum of the absolute values of the weights, represents the entropy loss, which is used to measure the distribution complexity of the feature map, λ entropy is the weight hyperparameter that controls the entropy regularization term and is used to balance the contributions of the entropy loss and L1 regularization to the total loss

[0211] Calculate the total loss

[0212]

[0213] where represents the total loss function, which combines the reconstruction loss and the regularization loss. λ is the weight hyperparameter, which is used to balance the contributions of the reconstruction loss and the regularization loss.

[0214] Calculate the gradient through backpropagation, update the network weights, and at the same time adjust the positions of the spline grid points according to the input data distribution to ensure that the spline function can adapt to the distribution

[0215]

[0216] where \(W\) are all the weight parameters in the network, is the total loss For the gradient of the weight parameter \(W\), \(\eta\) is the learning rate, which controls the step size of weight update.

[0217] The trained encoder and decoder networks based on B-spline functions can accurately capture global and local features, achieve high-quality image feature extraction and reconstruction, and finally obtain the optimal model.

[0218] Step 3.4 uses Precision, Accuracy, Balanced Accuracy, Recall, and F1-score as evaluation metrics of the model to evaluate the training and test results. The functional expressions of each evaluation metric are as follows:

[0219]

[0220] where \(TP\), \(TN\), \(FP\), and \(FN\) represent true positive, true negative, false positive, and false negative respectively. Precision represents the precision rate, which measures the proportion of samples that are actually positive among the samples predicted as positive by the model. Accuracy represents the accuracy rate, which measures the proportion of samples correctly predicted by the model in the total samples. Recall represents the recall rate, which measures the proportion of all actually positive samples correctly identified by the model. F1-score is the harmonic mean of precision and recall, which is used to balance between these two metrics and is especially suitable for class imbalance problems.

[0221] Compare this embodiment with the current mainstream dermatological image segmentation models on the ISIC 2018 dataset, and the results are shown in Table 1.

[0222] Table 1 Performance comparison of segmentation models on the ISIC 2018 dataset

[0223] Segmentation model Precision Accuracy Recall VGG16 74.68 74.50 73.82 ResNet-50 75.62 75.48 74.37 ResNet-101 75.43 75.21 76.84 Vit 77.65 77.42 74.23 The method of this embodiment 82.13 83.23 82.91

[0224] Table 1 compares the performance of different segmentation models on the ISIC-2018 dataset. The evaluation metrics include Precision, Accuracy, and Recall. From the results, the method of this embodiment achieves the best performance in all three metrics, with Precision being 82.13%, Accuracy being 83.23%, and Recall being 82.91%, indicating that its segmentation performance is better than other comparison models.

[0225] Specifically, the Precision, Accuracy, and Recall of VGG16 are 74.68%, 74.50%, and 73.82% respectively. ResNet-50 shows a slightly improved performance in these three metrics, with Precision being 75.62%, Accuracy being 75.48%, and Recall being 74.37%. The performance of ResNet-101 is close to that of ResNet-50, with Precision being 75.43%, Accuracy being 75.21%, and Recall being 76.84%. ViT performs the best among these four models, with Precision being 77.65%, Accuracy being 77.42%, and Recall being 74.23%. The Precision, Accuracy, and Recall of the method of this embodiment are 82.13%, 83.23%, and 82.91% respectively, showing relatively prominent performance.

[0226] Generally speaking, through effective improvement strategies, the method of this embodiment has achieved significant performance improvement in the skin disease image segmentation task, especially showing excellent performance in the Precision, Accuracy, and Recall metrics, demonstrating stronger prediction performance and overall effects.

[0227] Example 2

[0228] In this embodiment, by comparing with the VGG16 and ViT models, the effectiveness of the method of this embodiment in skin disease image segmentation is further verified. The experiment uses the ALL-IDB dataset, and the evaluation metrics include Precision (Precision rate), Recall (Recall rate), F1-score (harmonic mean of Precision and Recall rate), and Accuracy (Accuracy rate).

[0229] Table 2 Performance comparison of different segmentation models on the ALL-IDB dataset

[0230]

[0231] In the comparison with VGG16 and ViT, the method of this embodiment shows significant advantages:

[0232] 1. Precision: The precision rate of the method in this embodiment is 99.90%, which is 3.75% higher than that of VGG16 and 1.75% higher than that of ViT. This indicates that the method in this embodiment can more accurately identify positive class samples and reduce the occurrence of false positives.

[0233] 2. Recall: The recall rate of the method in this embodiment is 99.85%, which is higher than 96.15% of VGG16 and 98.08% of ViT, showing better performance of this method in capturing positive class samples.

[0234] 3. F1 - score: The F1 - score of the method in this embodiment is 99.85%, which is 3.70% higher than that of VGG16 and 1.77% higher than that of ViT, indicating that the method in this embodiment performs excellently in balancing precision and recall.

[0235] 4. Accuracy: The accuracy rate of the method in this embodiment is 99.86%, which is 3.71% higher than that of VGG16 and 1.78% higher than that of ViT, reflecting higher overall accuracy.

[0236] From the results of this group of experiments, it can be seen that the method in this embodiment is superior to VGG16 and ViT in terms of Precision, Recall, F1 - score and Accuracy indicators, indicating that the technical solution in this embodiment has higher accuracy and robustness in dealing with skin disease image segmentation tasks.

[0237] Example 3

[0238] This embodiment further compares the method in this embodiment with other popular segmentation network models, such as WBCsNet and EfficientV2, aiming to verify the superiority of the method in this embodiment in skin disease image segmentation tasks.

[0239] Table 2 Performance comparison of different segmentation models on the ALL - IDB dataset (Example 3)

[0240]

[0241]

[0242] In the comparison with WBCsNet and EfficientV2, the method in this embodiment shows obvious advantages:

[0243] 1. Precision: The precision rate of the method in this embodiment is 99.47%, which is 1.45% higher than that of VGG16 and 2.57% higher than that of ViT. This indicates that the method in this embodiment has a significant improvement in the ability to correctly predict positive class samples.

[0244] 2. Recall: The recall rate of the method in this embodiment is 99.46%, higher than 98.01% of VGG16 and 96.90% of ViT, indicating that this method has better performance in capturing positive samples.

[0245] 3. F1-score: The F1-score of the method in this embodiment is 99.46%, 1.45% higher than that of VGG16 and 2.57% higher than that of ViT, indicating that the method in this embodiment performs excellently in balancing precision and recall.

[0246] 4. Accuracy: The accuracy rate of the method in this embodiment is 99.46%, 1.45% higher than that of VGG16 and 2.56% higher than that of ViT, reflecting higher overall accuracy.

[0247] From the results of this group of experiments, it can be seen that the method in this embodiment is superior to VGG16 and ViT in terms of Precision, Recall, F1-score and Accuracy indicators, proving that the technical solution in this embodiment has higher accuracy and robustness in the image segmentation task.

[0248] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A deep medical image segmentation method based on B-spline functions and Transformer, characterized in that It includes the following steps: Step 1: Preprocess the input image dataset, perform data augmentation, and divide the dataset into a training set, a test set 1, and a validation set according to the ratio of 8:1:1; Step 2: Construct a feed-forward neural network based on the B-spline function, the encoder-decoder network based on the B-spline function and Transfomer. Use the B-spline function to replace the self-attention mechanism in Transfomer and the weight parameters of the MLP in the feed-forward neural network, and directly connect two neurons through the B-spline function; Step 3: Input the preprocessed image dataset into the encoder and decoder network based on the B-spline function for training to obtain an optimal model; Step 4: After training, input the image dataset into the obtained optimal model to detect the segmentation result of the medical image dataset.

2. The medical image depth segmentation method based on B-spline function and Transformer according to claim 1, wherein, In the said Step 1, it includes the following steps: Step 1.1: Divide the medical image dataset into a training set and a validation set according to the ratio of 8:1:1; Step 1.2: Use cv2 to load the images and masks in the training set and test set of the medical image dataset, and load the sizes of the images and masks to 256×256; Step 1.3: The mask is grayscaled before being converted to a PIL image, and then the pixel values are normalized to [0,1] by dividing by 255. The grayscaling and normalization formulas are as follows: where R, G, and B are the values of the red, green, and blue channels of the image respectively, and Out is the grayscaled and normalized image; Step 1.4: Convert the image from BGR format to RGB format; Step 1.5: Perform operations such as random scaling, random horizontal flipping, random vertical flipping, random cropping, and standardization processing on the training set data in sequence to perform data augmentation and preprocessing work. This data augmentation process introduces diversity and uncertainty. The standardization processing formula is as follows: where Input is the input image, Mean is the mean, Std is the standard deviation, and Normalized is the image after standardization processing.

3. The medical image depth segmentation method based on B-spline function and Transformer according to claim 1, characterized in that, The said Step 2 includes the following steps: Step 2.1: In the encoder of U-net, combine U-net with Transfomer. Each encoder consists of a self-attention mechanism and a feed-forward neural network. The self-attention mechanism helps the model capture the global information in the image and enhances the model's understanding of long-range dependencies and global context; Step 2.2: Change the weight representation in the original multi-layer perceptron MLP to a B-spline function. The B-spline function connects two neurons to replace the linear weight in the MLP; Any continuous function f(x1,…,x n ) is expressed as a nested combination of a finite number of single-variable functions, where and Φ q are both single-variable functions For a smooth function $f:[0,1]$ n $\to\mathbb{R}$, where x p represents the p-th component of the vector x, and contributes a term to the sum of the q-th outer function, where the index q is used to iterate over each component of the outer function Φ. Thus, there is a unary function that processes the p-th component of the input vector x and contributes a term to the sum of the q-th outer function. The theorem states that with 2n + 1 such outer functions - each outer function Φ q is a unary function that acts on the sum consisting of the outputs of the inner unary functions to represent any multivariate f; Step 2.3: For the supervised learning task composed of the input-output task pair {x i , y i}, find f such that for all data points y i ≈ f(x i ). It is sufficient to find appropriate unary functions and Φ q . Thus, each one-dimensional function is parameterized by a B-spline curve, where the learnable parameter coefficients of the local B-spline basis functions; In the MLP, define a layer consisting of a linear transformation and a non-linear transformation. The KAN layer with an output dimension is defined as a one-dimensional function matrix Φ = {φ q,p}, p = 1, 2, …, n in , q = 1, 2…, n out , Among them, the function φ q,p has trainable parameters, and the internal function φ q,p forms a KAN layer. After that, where the input dimension n in = n, and the output dimension n out = 2n + 1. Each input passes through such a set of dimensionality conversions to capture the input feature information and transform it into an intermediate representation; External function Φ q Form a KAN layer, where the input dimension n in = 2n + 1, and the output dimension n out = 1. This layer integrates all the outputs of the internal function layer to form the final output; Calculate the grid step size based on the initialized grid start and end ranges and the number of segments of the grid. Then, generate grid points using the order of the B-spline basis functions and extend them to all channels. Next, use the B-spline basis functions originally constructed with the grid points to perform local fitting on the input feature x. First, construct the 0th-order B-spline basis function, and then recursively define higher-order B-spline basis functions. Finally, output the B-spline basis function matrix; The process is formulated as follows: grid[i] = grid range [0] + i·h, i = -s, -s + 1, …, grid size + s grid = grid.expand(in features , -1) (batch size, in features, grid size + spline order) where h is the grid step size, grid_range is the range of the spline function grid, grid_size is the number of segments of the grid, s is the order of the B-spline basis function, i is the index of the grid point, and grid is the complete array of grid points, representing the knots of the discrete spline function. represents the zero-order spline basis function. The order of the B-spline basis function batch size represents the number of input samples, and in features represents the number of input features per sample. grid size + spline order represents the support interval length of each spline basis function; Step 2.4: Optimize the residual activation function. The activation function φ(x) is the sum of the basis function b(x) and the spline function φ(x) = w(b(x) + spline(x)) Set b(x) to b(x) = silu(x) = x / (1 + e -x ) spline(x) is parameterized as a linear combination of B-splines, where c i is trainable Finally, w is combined with b(x) and the spline function, and the overall amplitude of the activation function is controlled by the w factor; The initial value of each activation function spline(x) ≈ 0, while w is initialized according to Xavier initialization. For each grid according to its input activation in real time, solve the problem that splines are defined on a bounded region; First, perform piecewise polynomial fitting on the input x through B-spline basis functions and the learned weights W spline , and then implement the above operations through matrix multiplication to form a tensor representation. Then, the input x is processed through an activation function, and the activated input is processed through a linear transformation. Finally, the basic linear features and the piecewise fitting features are fused to generate the final output; The process formula is as follows: y spline = B(x)·W spline [i] x act = SiLU(x) y base = W base · x act y = y base + y spl i ne where y spline is the spline fitting output, b i (x) represents the B-spline basis function, which is the eigenvalue of the input x on the grid i and represents the local feature fitting result of x. W spline [i] is the learnable weight corresponding to the i-th grid, representing the weight contribution of the feature under the action of the i-th spline basis function. grid_size is the number of segments of the spline grid, B(x) is the matrix of B-spline basis function values of the input x, SiLU(x) represents the activation function, and x act is the input feature after the non-linear transformation of the activation function, and y base represents the global feature extracted by the global linear transformation. y represents the final feature output, which is composed of the weighted sum of the global feature and the local feature.

4. The medical image depth segmentation method based on B-spline function and Transformer according to claim 1, wherein The said step 3 includes the following steps: 3.1: The input image I is first mapped into the discrete spline grid space through the grid process to generate the feature representation input spline encoder B(x) = B - spline(x, grid) where B(x) represents the B-spline basis function matrix, which represents the spline expansion of the input x on the grid grid. x is the input feature, and grid is the spline grid; Using the weight matrix W of the spline encoder spline , calculate the linear combination of spline basis functions to extract local features F spline = B(x) · W spline Among them, F spline is a local feature representation, and W spline represents the weight matrix corresponding to the B-spline At the same time, the input obtains the global feature through the basic linear transformation F base = SiLU(x) · W base Among them, F base represents the global feature, SiLU(x) is a non-linear activation of the input feature x, and W base is the weight matrix of the global feature; Sum the local feature F spline and the global feature F base for summation and fusion, and then re-enter them into the spline encoder to generate the final input feature F enc = F spline + F base The encoder extracts multi-level features in stages Among them, F enc represents the feature map input feature output by the i-th layer of the encoder, EncoderStage (i) represents the i-th layer in the encoder, which is used to extract multi-scale features, represents the feature map output by the i-th layer encoder; Step 3.2: The decoder at each stage uses the output features of the encoder and the upsampled features of the corresponding stage as inputs, and reconstructs the input features using a spline structure Denote the feature map output by the i-th layer of the encoder, Denote the result after upsampling the output features of the previous decoder stage, DecoderStage (i) Denote the i-th layer in the decoder, which is used to fuse the features of the encoder, The feature map output by the i-th layer of the decoder; Align the spline fitting feature and the upsampled feature in the channel dimension and add them element by element Among them represents the output feature of the i-th layer of the decoder, is the feature obtained by upsampling the output of the previous layer of the decoder, represents the fused feature; Enhance the attention of the original features Extract spatial attention weights Among them Perform global average pooling on the feature map to generate global information Perform global max pooling on the feature map to generate global information, Conv 1×1 Use 1×1 convolution to reduce the feature dimension. σ represents the activation function, which is used to normalize the weights, M s represents the spatial attention weight Perform global max pooling on the feature map to generate the global information extraction channel attention weight Among them Perform global average pooling on the feature map to generate global information Perform global max pooling on the feature map to generate global information. KAN represents the KAN neural network module based on the B-spline function, and M c represents the channel attention weight Use the attention weight to weight the feature Among them represents the spatially attention-weighted feature represents the channel attention-weighted feature represents the final feature after fusion; Step 3.3: During the training process, compare the error between the decoder output and the target image, the reconstruction loss Among them is the reconstruction loss, y pred is the predicted result output by the decoder, that is, the feature map output by the network, y true is the target true label, that is, the actual annotated image of image segmentation. DiceLoss is a loss form of the Dice coefficient, which is used to measure the similarity of two images; Perform regularization loss, including L1 regularization and entropy regularization where is the regularization loss, ‖W spline ‖1 is the L1 regularization of the spline weights, which constrains the sum of the absolute values of the weights, represents the entropy loss, which is used to measure the distribution complexity of the feature map, and λ entropy is the weight hyperparameter that controls the entropy regularization term and is used to balance the contributions of the entropy loss and the L1 regularization to the total loss Calculate the total loss Among them represents the total loss function, which combines the reconstruction loss and the regularization loss. λ is the weight hyperparameter used to balance the contributions of the reconstruction loss and the regularization loss; Calculate the gradient through backpropagation, update the network weights, and at the same time adjust the positions of the spline grid points according to the input data distribution to ensure that the spline function can adaptively distribute where W are all the weight parameters in the network, is the total loss is the gradient with respect to the weight parameter W, and η is the learning rate that controls the step size of weight updates; The trained encoder and decoder networks based on B-spline functions can accurately capture global and local features, achieve high-quality image feature extraction and reconstruction, and finally obtain the optimal model; Step 3.4 uses Precision, Accuracy, Balanced Accuracy, Recall, F1-score as the evaluation metrics of the model to evaluate the training and test results. The functional expressions of each evaluation metric are as follows: Among them, TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative respectively. Precision represents precision, Accuracy represents accuracy, which measures the proportion of samples correctly predicted by the model in the total samples. Recall represents recall, which measures the proportion of samples that are actually positive and correctly identified by the model. F1-score is the harmonic mean of precision and recall, used to balance between these two metrics.

5. The medical image depth segmentation method based on B-spline function and Transformer according to claim 1, characterized in that Step 4: After the model training is completed, use the optimal model during training to verify on the medical image validation set to detect the segmentation effect in medical images.

Images