Skin cancer focus segmentation method based on discrete neural memory ordinary differential equation

Abstract

The invention discloses a skin cancer focus segmentation method based on a discrete neural memory ordinary differential equation, and the method comprises the steps: determining the form of a basic differential equation in a model, and collecting an image data set; discretizing a neural memory ordinary differential equation by applying Taylor formula derivation, and preliminarily screening the collected skin disease data set; discretizing the neural memory ordinary differential equation by using a linear multistep method; according to a neural network structure, a neural memory ordinary differential equation which is derived by applying a Taylor formula and discretized by applying a linear multi-step method is changed into a proper form, and a second-order Taylor method decoder and a linear multi-step method decoder which are suitable for the U-shaped network are designed; and embedding the decoder into a U-shaped network to form a skin cancer focus segmentation model, and inputting the augmented data set into the model for training. According to the method, the problems of high complexity and high calculation cost of the existing medical image segmentation model are solved, the model performance is not damaged, and the method is a lightweight scheme with high compatibility and high transferability.

Description

Technical Field

[0001] The present invention relates to the field of skin cancer lesion image segmentation, and in particular to a skin cancer lesion segmentation method based on discrete neural memory ordinary differential equations. Background Art

[0002] Medical image segmentation is an important research area at the intersection of computer vision and medicine. Its main purpose is to demarcate different structures, tissues, or pathologies in medical images, enabling quantitative analysis and diagnosis of specific areas. UNet is an important milestone network architecture that demonstrated the effectiveness of encoder-decoder convolutional neural networks (CNNs) with skip connections in medical image segmentation. Today, UNet has become the underlying framework for most notable medical image segmentation methods. To achieve higher accuracy, many studies have introduced complex modules or increased the number of parameters. For example, ResUNet combines the UNet architecture with ResNet-inspired residual connections. Swin-UNet utilizes the Swin Transformer architecture. The TransFuse model uses a dual-path configuration that seamlessly combines CNN and ViT to capture both local and global information.

[0003] Existing U-networks improve the performance of UNet, but the computational cost of most networks is also greatly increased. In practical application scenarios, this may lead to deployment difficulties with limited computing resources. To solve this problem, researchers began to explore methods to lightweight models. In some of the latest studies, UNeXt combines UNet and MLP to propose a lightweight architecture that can achieve significant performance while reducing parameters and computational requirements. MALUNet successfully reduced the model size by reducing the number of model channels and introducing multiple attention modules. Efficient Group Enhanced UNet (EGE-UNet) reduces parameters and computational complexity by integrating enhanced attention mechanisms and feature fusion modules.

[0004] Although the above methods have demonstrated impressive capabilities in reducing model parameters and computational complexity, they often face challenges in adapting to existing frameworks and generality. In order to achieve lightweight, these networks have made a large reduction in the number of channels on their base networks, which inevitably damages the performance of the network while reducing the number of parameters and computation. After achieving the goal of lightweight, the performance of the network is improved through various carefully designed modules based on the low number of channels. Obviously, these networks pursue the two goals of lightweight and maintaining network performance separately, and lack a comprehensive solution. In addition, these networks are mostly composed of inseparable functional modules and network architectures, lacking compatibility and migration capabilities for different tasks. Summary of the invention

[0005] In order to overcome the shortcomings and deficiencies of the prior art, the present invention provides a skin cancer lesion segmentation method based on discrete neural memory ordinary differential equations.

[0006] The specific steps of the technical solution adopted by the present invention are:

[0007] Step S1: determine the form of the basic differential equation in the model and collect a dermatoscopic image dataset containing benign and malignant lesions;

[0008] Step S2: apply Taylor's formula to discretize the neural memory ordinary differential equation, preliminarily screen the collected skin disease data set, and screen out low-quality data to form a model training data set;

[0009] Step S3: Apply the linear multi-step method to discretize the neural memory ordinary differential equation, batch reshape the images in the model training data set to make their size adapt to the neural network input and normalize them;

[0010] Step S4: according to the neural network structure, the neural memory ordinary differential equation derived by Taylor formula and discretized by linear multi-step method is changed into a suitable form, and the image obtained in step S4 is augmented by flipping, rotating and cropping data enhancement methods to form an augmented data set;

[0011] Step S5: designing a second-order Taylor method decoder and a linear multi-step method decoder suitable for the U-type network according to the mathematical formula obtained in step S4;

[0012] Step S6: The U-shaped network decoder obtained in step S5 is embedded into the U-shaped network to form a skin cancer lesion segmentation model, and the augmented data set is input into the model for training. After training and verifying the model performance, the dermoscopic image is input into the trained model to obtain the predicted label.

[0013] Furthermore, in step S1, the form of the neural memory ordinary differential equation is:

[0014]

[0015] in, is the derivative of the output o(t), representing the hidden state of the network, o(t) is the current state of the network, A is the activation function, f is the convolution operation, i(t) is the current input of the network, and p t are the current parameters of the network.

[0016] Furthermore, in step S2, the neural memory ordinary differential equation is discretized by applying the Taylor formula, and the second-order Taylor formula is as follows:

[0017]

[0018] According to formula (1) and formula (2), we have:

[0019]

[0020] Among them, n+1 , o n , o n-1 are the outputs of the network at the next time step, the current time, and the previous time step, respectively. δ is the time step. and Represent the hidden states of the network at the current time step and the previous time step respectively.

[0021] Furthermore, in step S3, the linear multi-step method is applied to discretize the neural memory ordinary differential equation, and the linear multi-step method formula is:

[0022]

[0023] According to formula (1) and formula (4), we have:

[0024]

[0025] Furthermore, in step S4, the discretized neural memory ordinary differential equation is changed into a suitable form, and is rewritten according to equation (3) to be suitable for the U-type network decoder:

[0026]

[0027] in, and They are and l is the current number of layers in the network, o is l-1 , o l and l+1 are the outputs of the next layer, the current layer, and the previous layer of the network respectively, and the corresponding i i and i l+1 are the inputs of the current and previous layers of the network, respectively, l and p l+1 are the parameters of the current layer and the previous layer respectively, and δ is the discrete step length.

[0028] Furthermore, in step S4, the discretized neural memory ordinary differential equation is changed into a suitable form, and is rewritten according to equation (5) to be suitable for the U-type network decoder:

[0029]

[0030] Furthermore, in step S5, a lightweight decoder suitable for the U-type network is designed according to formula (6) and formula (7), and the mathematical formula is processed into a neural network structure.

[0031] Furthermore, in step S6, the decoder designed according to equations (6) and (7) is embedded in the U-type network, and δ is set to the inverse of the total number of network layers to form a skin cancer lesion segmentation method.

[0032] Beneficial effects:

[0033] The present invention proposes a skin cancer lesion segmentation method based on discrete neural memory ordinary differential equations. The present invention improves and optimizes discrete neural memory ordinary differential equations to perform skin cancer lesion segmentation, solving the problems of high complexity and high computational cost of existing high-performance medical image segmentation models. The proposed method does not damage the model performance, and is a lightweight solution with high compatibility and strong transferability. The model proposed by the present invention has high compatibility and strong transferability, and has the potential to be applied to any U-shaped network. The model can significantly reduce network complexity while ensuring network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of the overall steps of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other. The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0036] like Figure 1 As shown, a skin cancer lesion segmentation method based on discrete neural memory ordinary differential equations, the specific steps of the method are:

[0037] Step S1: determine the form of the basic differential equation in the model and collect a dermatoscopic image dataset containing benign and malignant lesions;

[0038] Step S2: apply Taylor's formula to discretize the neural memory ordinary differential equation, preliminarily screen the collected skin disease data set, and screen out low-quality data to form a model training data set;

[0039] Step S3: Apply the linear multi-step method to discretize the neural memory ordinary differential equation, batch reshape the images in the model training data set to make their size adapt to the neural network input and normalize them;

[0040] Step S4: according to the neural network structure, the neural memory ordinary differential equation derived by Taylor formula and discretized by linear multi-step method is changed into a suitable form, and the image obtained in step S4 is augmented by flipping, rotating and cropping data enhancement methods to form an augmented data set;

[0041] Step S5: designing a second-order Taylor method decoder and a linear multi-step method decoder suitable for the U-type network according to the mathematical formula obtained in step S4;

[0042] Step S6: The U-shaped network decoder obtained in step S5 is embedded into the U-shaped network to form a skin cancer lesion segmentation model, and the augmented data set is input into the model for training. After training and verifying the model performance, the dermoscopic image is input into the trained model to obtain the predicted label.

[0043] Specifically, in step S1, the form of the neural memory ordinary differential equation is:

[0044]

[0045] in, is the derivative of the output o(t), representing the hidden state of the network, o(t) is the current state of the network, A is the activation function, f is the convolution operation, i(t) is the current input of the network, and p t is the current parameter of the network. The neural memory ordinary differential equation imitates the human memory mechanism, separates the learning system from the memory system, and is different from the neural ordinary differential equation which directly maps input to input. The neural memory ordinary differential equation learns the input features with an independent learning system, and uses another initialized data in the independent memory system as a carrier of the learned features.

[0046] Specifically, in step S2, the neural memory ordinary differential equation is discretized by applying the Taylor formula, and the second-order Taylor formula is as follows:

[0047]

[0048] According to formula (1) and formula (2), we have:

[0049]

[0050]

[0051] Among them, n+1 , o n , o n-1 are the outputs of the network at the next time step, the current time, and the previous time step, respectively. δ is the time step. and Represent the hidden states of the network at the current time step and the previous time step respectively.

[0052] Specifically, in step S3, the linear multi-step method is used to discretize the neural memory ordinary differential equation, and the linear multi-step method formula is:

[0053]

[0054] According to formula (1) and formula (8), we have:

[0055]

[0056] Specifically, in step S4, the discretized neural memory ordinary differential equation is changed to a form in which the number of neural network layers is used as the number of derivation steps, and is rewritten according to formula (3) to be suitable for the U-type network decoder:

[0057]

[0058] in, and They are and l is the current number of layers in the network, o is l-1 , o l and l+1 are the outputs of the next layer, the current layer, and the previous layer of the network respectively, and the corresponding i i and i l+1 are the inputs of the current and previous layers of the network, respectively, l and p l+1 are the parameters of the current layer and the previous layer respectively, and δ is the discrete step length.

[0059] In step S4, the discretized neural memory ordinary differential equation is changed to a form in which the number of neural network layers is used as the number of derivation steps, and is rewritten according to equation (5) to be suitable for the U-type network decoder:

[0060]

[0061] Since equations (10) and (11) need to apply the information of the current layer l and the previous layer l+1 for derivation, there is a certain limit on the number of layers l, that is, 1≤l≤L-1, where L is the total number of layers in the network. The explicit Euler method needs to be used once in the bottom L layers to obtain the initial operation information. The explicit Euler method formula is Discretize equation (1) to get y n+1 =(1-δ)·y n +δ·f(y n +h(x n ,θ n )), rewritten as the number of neural network layers as the number of derivation steps, using only the information of the bottom layer, i.e. L-1 =(1-δ)·o L +δ·A(o L +fiL,θL.

[0062] Specifically, in step S5, a lightweight decoder suitable for U-type networks is designed according to equations (10) and (11). The decoder can significantly reduce the number of network parameters and computational complexity while maintaining the performance of the model, and due to its plug-and-play characteristics, it has the potential to be applied to all U-type networks. The decoder converts i in equations (10) and (11) into l Corresponding to the encoder output of the U-type network with corresponding layer number l, o l The number of layers corresponding to the U-type network corresponds to the decoder output of l in the formula, and the jump connection process of the U-type network is represented by f(i l+1 ,p l+1 ) and implement the code modules one by one according to the mathematical symbol operation process in the formula.

[0063] In step S6, the decoder designed according to formula (10) and formula (11) is embedded in the U-type network. The U-type network structure is a convolutional neural network for image segmentation, which has symmetrical encoder and decoder parts. The encoder extracts features and gradually reduces the image resolution through convolution and maximum pooling layers, while the decoder gradually restores the image resolution through deconvolution layers. The jump connection directly transfers the feature map of the corresponding layer in the encoder to the decoder, retaining the high-resolution feature information, thereby improving the segmentation accuracy. δ is set to the inverse of the total number of network layers to form the final skin cancer lesion segmentation method. The downlink path of this method adopts the traditional U-type network structure, downsamples the input data at each layer and extracts features at different levels; the uplink path adopts the decoder designed in the present invention, and each layer of the decoder receives data i of the current layer l and the previous layer l+1 l 、i l+1 and l ,o l+1 ,The extracted features are aggregated in the form of ordinary differential equations; the features are labeled and reconstructed at the top layer of the network, and finally the predicted pixel-level binary labels are output. This method benefits from the superiority of the upstream path. The number of channels in the derivation process is only 3, which greatly reduces the number of parameters and computational complexity of the U-type network while maintaining superior performance.

[0064] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Skin cancer lesion segmentation method based on discrete neural memory ordinary differential equations, It is characterized in that The method includes: Step S1: determine the form of the basic differential equation in the model and collect a dermatoscopic image dataset containing benign and malignant lesions; Step S2: apply Taylor's formula to discretize the neural memory ordinary differential equation, preliminarily screen the collected skin disease data set, and screen out low-quality data to form a model training data set; Step S3: Apply the linear multi-step method to discretize the neural memory ordinary differential equation, batch reshape the images in the model training data set to make their size adapt to the neural network input and normalize them; Step S4: according to the neural network structure, the neural memory ordinary differential equation derived by Taylor formula and discretized by linear multi-step method is changed to a form in which the number of neural network layers is used as the number of derivation steps, and the image obtained in this step is augmented by flipping, rotating, and cropping data enhancement methods to form an augmented data set; Step S5: designing a second-order Taylor method decoder and a linear multi-step method decoder suitable for the U-type network according to the mathematical formula obtained in step S4; Step S6: The U-type network decoder obtained in step S5 is embedded into the U-type network to form a skin cancer lesion segmentation model, and the augmented data set is input into the model for training. After training and verifying the model performance, the dermoscopic image is input into the trained model to obtain a predicted label, which is a pixel-level classification binary grayscale image of the lesion. At this time, the lesion segmentation of the input image is completed.

2. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, Features: In step S1, the form of the neural memory ordinary differential equation is: in, is the derivative of the output o(t), representing the hidden state of the network, o(t) is the current state of the network, A is the activation function, f is the convolution operation, i(t) is the current input of the network, and p t are the current parameters of the network.

3. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, It is characterized in that In step S2, the neural memory ordinary differential equation is discretized by applying the Taylor formula. The second-order Taylor formula is as follows: Among them, δ is the discrete step length, y t+δ ,y t are the predicted value of the next step length and the value of the current node, They are the first-order derivative and second-order derivative of the current node respectively; According to formula (1) and formula (2), we have: Among them, n+1 , o n , o n-1 are the outputs of the network at the next time step, the current time, and the previous time step, respectively. δ is the time step. and Represent the hidden states of the network at the current time step and the previous time step respectively.

4. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, It is characterized in that In step S3, a linear multi-step method is used to discretize the neural memory ordinary differential equation. The linear multi-step method formula is: in, is the first-order derivative of the previous node; According to formula (1) and formula (4), we have:

5. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, It is characterized in that In step S4, the discretized neural memory ordinary differential equation is changed into a form in which the number of neural network layers is used as the number of derivation steps, and is rewritten according to formula (3) to be suitable for the U-type network decoder: in, and They are and l is the current number of layers in the network, o is l-1 , o l and l+1 are the outputs of the next layer, the current layer, and the previous layer of the network respectively, and the corresponding i i and i l+1 are the inputs of the current and previous layers of the network, respectively, l and p l+1 are the parameters of the current layer and the previous layer respectively, and δ is the discrete step length.

6. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, It is characterized in that In step S4, the discretized neural memory ordinary differential equation is changed into a form in which the number of neural network layers is used as the number of derivation steps, and is rewritten according to equation (5) to be suitable for the U-type network decoder:

7. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, It is characterized in that In step S5, a lightweight decoder suitable for the U-type network is designed according to equations (6) and (7), and the mathematical formula is processed into a neural network structure.

8. The skin cancer lesion segmentation method based on discrete neural memory ordinary differential equation according to claim 1, It is characterized in that In step S6, the decoder designed according to equations (6) and (7) is embedded into a U-shaped network, and δ is set to the inverse of the total number of network layers to form a skin cancer lesion segmentation method.

Images