Image segmentation method and system based on adaptive pulse, medium and equipment

Abstract

The invention discloses an image segmentation method and system based on adaptive pulses, a medium and equipment, and belongs to the technical field of image segmentation. Training the image segmentation model by using the training image, and obtaining a trained image segmentation model after the training is completed; wherein when the image segmentation model is trained, gradient information in a back propagation process of the model is calculated; taking the gradient information and the parameter weight of the current model as a posterior synaptic membrane; calculating neurons in the forward propagation process of the model; updating the image segmentation model by using a posterior synaptic membrane and neurons; and utilizing the trained image segmentation model to segment similar images of the training image. The problem that the model weight cannot be updated when the current image segmentation model is trained is solved, and the training precision of the image segmentation model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to an image segmentation method, system, medium and equipment based on adaptive pulse. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The neural membrane system (neural P system, SNP system) is a membrane computing model that uses a spiking neural network to encode information over time. The neural membrane system combines the idea of ​​spiking neurons and can be regarded as a third-generation artificial neural network model. It can well simulate the processing mechanism of some biological neural systems and has a powerful distributed framework to complete various high-performance parallel computing. At present, the research on the neural membrane system is still mostly theoretical. How to use the joint advantages of different disciplines to propose a new membrane computing model, solve various practical problems, and broaden the application scope of membrane computing is a valuable and important topic in the field of membrane computing research.

[0004] Currently, image segmentation models are mainly constructed through neural membrane systems, and then images are segmented through the image segmentation models. However, when existing image segmentation models are trained, they only have neurons as computing units, and the models are updated only through the calculations of neurons.

[0005] The current training method of this image segmentation model cannot adaptively update the model weights, resulting in limited model training accuracy and, in turn, limited image segmentation accuracy. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes an image segmentation method, system, medium and device based on adaptive pulse, which improves the accuracy of image segmentation.

[0007] To achieve the above object, the present invention adopts the following technical solution: Firstly, an image segmentation method based on adaptive pulse is proposed, comprising: Get training images; The image segmentation model is trained using training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; Use the trained image segmentation model to segment the same images as the training images.

[0008] Furthermore, the image segmentation model includes a multi-layer downsampling layer, a multi-layer upsampling layer and a segmentation head. The multi-layer downsampling layer extracts multi-scale features of the image, and the multi-layer upsampling layer performs a fusion upsampling operation on the multi-scale features to obtain image features; the segmentation head is used to segment the image features to obtain an image segmentation result.

[0009] Furthermore, the multi-layer downsampling layer includes a large kernel convolution layer and multiple four-directional Mamba block layers connected in sequence; wherein each four-directional Mamba block layer includes a gated spatial convolution module, a first layer normalization, a four-directional Mamba module, a second layer normalization and a multi-layer perception layer connected in sequence.

[0010] Furthermore, a residual connection is set between the output of the gated spatial convolution module and the output of the four-directional Mamba module in each four-directional Mamba block layer; and a residual connection is set between the output of the four-directional Mamba module and the output of the multi-layer perception layer.

[0011] Furthermore, the gated spatial convolution module includes a first convolution block and a second convolution block in parallel; the output of the first convolution block is multiplied by the output of the second convolution block and input into the third convolution block, and the output of the third convolution block is added to the feature of the input gated spatial convolution module as the output of the gated spatial convolution module.

[0012] Furthermore, the neuron transmits neuromodulators to the postsynaptic membrane, which are used to update the weights in the postsynaptic membrane.

[0013] Furthermore, pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rules are: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rules are: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of the rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators.

[0014] Secondly, an adaptive pulse-based image segmentation system is proposed, including: An image acquisition module, used to acquire training images; The model training module is used to train the image segmentation model using the training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; The image segmentation module is used to segment the same type of training images using the trained image segmentation model.

[0015] In a third aspect, a computer device is provided, the device comprising: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the image segmentation method based on adaptive pulse proposed in the first aspect is implemented.

[0016] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the adaptive pulse-based image segmentation method proposed in the first aspect.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes an image segmentation method, system, medium and device based on adaptive pulses. When training an image segmentation model, the method calculates the gradient information of the model during the back propagation process; uses the gradient information and the parameter weights of the current model as the postsynaptic membrane; calculates the neurons during the forward propagation process of the model; uses the postsynaptic membrane and the neurons to update the image segmentation model, and uses the neurons and the postsynaptic membrane to transmit pulses. The postsynaptic membrane represents the parameters of various weights that need to be updated, thereby improving the diversity of communication between neurons; and thus improving the accuracy of image segmentation.

[0018] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0020] Figure 1 The figure is an overall block diagram of the image segmentation method based on adaptive pulse disclosed in the embodiment; Figure 2It is a block diagram of the FSMamba block layer structure disclosed in the embodiment; Figure 3 It is a structural block diagram of the gated spatial convolution module disclosed in the embodiment; Figure 4 A schematic diagram of a pulse transmission process disclosed in an embodiment; Figure 5 A diagram showing the segmentation effect of the image segmentation method based on adaptive pulse disclosed in the embodiment; Figure 6 These are various indicators in the image segmentation model training disclosed in the embodiment. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0023] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] Example 1 This embodiment will start from the spiking neuron mechanism of the spiking neural membrane system and build a new spiking neuron model with reference to the second-generation neural network model, called the adaptive spiking neural membrane system with neuromodulators (SSNNP), and introduce it into the recognition of pancreas and tumors in medical image processing.

[0025] The classic SNP system consists of neurons and synapses. There is only one computing unit, neurons, which limits the application capability of the SNP system. In order to improve the accuracy of image segmentation, this embodiment improves the classic SNP system. The improved adaptive spike neural membrane system (SSNNP) includes two computing units, neurons and postsynaptic membranes. The pixels of the image are encoded as pulses in neurons, and then the UNet operation is performed through rules to segment the image.

[0026] In this embodiment, an image segmentation method based on adaptive pulse is disclosed, such as Figure 1-Figure 6 As shown, including: Get training images; The image segmentation model is trained using training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; Use the trained image segmentation model to segment the same images as the training images.

[0027] Among them, the image segmentation model includes multi-layer downsampling layers, multi-layer upsampling layers and a segmentation head. The multi-layer downsampling layers extract multi-scale features of the image, and the multi-layer upsampling layers perform fusion upsampling operations on the multi-scale features to obtain image features; the segmentation head is used to segment the image features to obtain the image segmentation result.

[0028] The multi-layer downsampling layer includes a large kernel convolution layer and multiple four-directional Mamba block layers connected in sequence; wherein each four-directional Mamba block layer includes a gated spatial convolution module, a first layer normalization, a four-directional Mamba module, a second layer normalization and a multi-layer perception layer connected in sequence.

[0029] A residual connection is set between the output of the gated spatial convolution module and the output of the four-directional Mamba module in each four-directional Mamba block layer; and a residual connection is set between the output of the four-directional Mamba module and the output of the multi-layer perception layer.

[0030] The gated spatial convolution module includes a first convolution block and a second convolution block in parallel; the output of the first convolution block is multiplied by the output of the second convolution block and then input into the third convolution block; the output of the third convolution block is added to the features of the input gated spatial convolution module and output as the gated spatial convolution module.

[0031] Neurons transmit neuromodulators to the postsynaptic membrane, which are used to update the weights in the postsynaptic membrane.

[0032] Pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rule is: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of the rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators.

[0033] In order to adapt the SSNNP system to the segmentation task, this embodiment redefines the rules in neurons and postsynaptic membranes in the SSNN P, including pulse transmission rules, pulse communication rules, and pulse bias rules.

[0034] Among them, the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than the set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane.

[0035] Specifically, the pulse transmission rules are: (1) When the postsynaptic membrane m contains c pulses, c>= set value; the pulse transmission rule is activated. c pulses are consumed and generated at the same time pulses are sent to subsequent neurons, where is the weight of the postsynaptic membrane m. In addition, the neuron transmits neuromodulators to the postsynaptic membrane, which are used to update the weight of the postsynaptic membrane and increase or decrease the pulse intensity. After the pulse is generated by such a rule weighted in the postsynaptic membrane, it is sent to the neuron behind it. Weight Can be a vector or a single value. The setting value here can be 1.

[0036] The pulse communication rule is: when the number of pulses in a neuron is greater than the set value, a first set number of pulses is generated and sent to subsequent neurons.

[0037] The pulse communication rules are: (2) When neuron ijk contains s pulses, s>= set value; pulse communication rule is activated. S pulses are consumed, and v pulses are generated and sent to subsequent neurons.

[0038] The pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than the set value, and the postsynaptic membrane contains a set number of neuromodulators with a set efficacy that fits the rule, a third set number of pulses is generated and sent to subsequent neurons. The third set number is determined based on the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the neuromodulator efficacy and the number of neuromodulators.

[0039] The pulse bias rule is: (3) When the postsynaptic membrane m contains c (c>=1) pulses, c>=set value, and the number of suitable rules is Efficacy The pulse bias rule is activated. c pulses are consumed and generated A pulse is sent to the subsequent neuron, and the value is set to 1.

[0040] The topological structure of the spiking neural P system is composed of a directed graph, where each neuron in the system is represented by a node, the synapse connecting two adjacent neurons in the system is represented by an edge, the pulse is represented by the letter a, and the neuron sends pulses through directed arcs. Therefore, the spiking neural P system is an extended model of the tissue membrane system. The pulse is the information medium used by the object neurons of the general SNP system to communicate with each other. The definition of the SSNN P system of order m+n proposed in this embodiment is as follows:

[0041] in, It's the alphabet. Represented as a pulse, It is a neuromodulator; is a neuron in the form of , ; is the number of initial peaks; It is a finite set of pulse communication rules and pulse forgetting rules. The pulse forgetting rule is a rule for clearing unused or inapplicable pulses in neurons. It is used to prove the universality of the SSNN P system, but not used in image segmentation applications. is the postsynaptic membrane in the form of , ; is the weight of the postsynaptic membrane, which has two forms: fixed weight and adaptive weight; It is a neuromodulator contained in the postsynaptic membrane, which is defined as a permanent resource and will not be consumed. It is a finite set of pulse transmission rules, pulse bias rules, and neuromodulator decomposition rules. Neuromodulator decomposition rules are rules for removing unused or inapplicable neuromodulators from neurons. It is used to prove the universality of the SSNN P system, but not used in image segmentation applications. They are represented as the collection of connections between neurons and between neurons and postsynaptic membranes, respectively; and It is the input / output neuron or postsynaptic membrane.

[0042] The initial configuration C0 is the state before the model is initialized and the image is input. It is composed of the initial state of the neurons representing the image and the postsynaptic membrane representing the various parameters of the model. The neurons representing the image and the postsynaptic membrane representing the parameters of the model Composition; The transition from one structure to another is achieved through calculations between neurons and postsynaptic membranes, that is, the execution of pulse communication rules and forgetting rules in neurons, and the execution of pulse transmission, pulse bias, and neuromodulator decomposition rules in postsynaptic membranes. In particular, in each postsynaptic membrane, if the number of neuromodulators meets the adaptive weight conditions, the weights will be adaptively adjusted. Dropout is a technique that randomly ignores neurons during training. In each training iteration, each neuron has a certain probability of being "discarded". The discarded neurons will not participate in the forward propagation and backpropagation of the current training step. Therefore, the network will try to "self-learn" in each round of training, without relying on any specific neurons, thereby avoiding overfitting. Similarly, if the number of weighted pulses meets the rules in neurons or postsynaptic membranes, the rules in neurons or postsynaptic membranes will be executed. In any calculation step, the rules are executed in a maximally parallelized manner. All applicable rules can work at the same time. If the triggering conditions of several rules are the same, the SSNN P system randomly extracts a rule for processing. The SSNN P system maximizes the parallelization of transition steps until the output rule that satisfies the output neuron is reached. The network layer responsible for the segmentation head is represented by the postsynaptic membrane, and the output of the postsynaptic membrane goes to the last layer of neurons. The output of this layer of neurons is the result of image segmentation.

[0043] This embodiment uses an image segmentation model to identify the pulse sequence and obtain a segmentation result of the image to be segmented, wherein the image segmentation model (MSNet) includes multiple downsampling layers, multiple upsampling layers and a segmentation head.

[0044] like Figure 1As shown, multiple downsampling layers correspond to multiple upsampling layers one by one; multiple downsampling layers include the first convolution layer and three FSMamba block layers (four-directional Mamba block layers); the first convolution layer uses a large kernel convolution layer, and the large kernel convolution layer uses depthwise convolution, with a kernel size of 7×7×7, a padding of 3×3×3, and a stride of 2×2×2. Given a three-dimensional input volume I∈RC×D×H×W, where C represents the number of input channels, the large kernel convolution layer extracts the first scale feature z0∈R32×D / 2×H / 2×W / 2 from the image. A second scale feature is extracted from the first scale feature z0 through the first FSMamba block layer, a third scale feature is extracted from the second scale feature through the second FSMamba block layer, a fourth scale feature is extracted from the third scale feature through the third FSMamba block layer, and the fourth scale feature is input into the multi-layer upsampling layer. After the multi-layer upsampling layer performs an upsampling operation on the fourth scale feature, a fourth upsampling feature is obtained. The fourth upsampling feature and the third scale feature are fused and then an upsampling operation is performed to obtain the third upsampling feature; the third upsampling feature and the second scale feature are fused and then an upsampling operation is performed to obtain the second upsampling feature; the second upsampling feature and the first scale feature are fused and then an upsampling operation is performed to obtain the image feature.

[0045] For the mth FSMamba block layer, the calculation process can be defined as: (4) Among them, GSC and FoM represent the gated spatial convolution module and the four-directional Mamba module, respectively. L∈{0,1,…,Nm−1}, LN (LayerNom) represents layer normalization, and MLP represents a multi-layer perception layer to enrich feature representation.

[0046] The four-directional Mamba module models feature dependencies by flattening 3D features into 1D sequences. Therefore, in order to extract the spatial relationship before the four-directional Mamba module, this embodiment designs a gated spatial convolution module (GSC), such as Figure 3 As shown in the figure, the input 3D features are fed into the first convolutional block and the second convolutional block. Both the first convolutional block and the second convolutional block contain normalization, convolution and nonlinear layers. The convolution kernel sizes of the first convolutional block and the second convolutional block are 5×5×5 and 3×3×3 respectively. Then the two features are multiplied pixel by pixel to control the information transmission, similar to the gating mechanism. Finally, the convolutional blocks are used to further fuse the features, and the residual connection is used to reuse the input features.

[0047] (5) Among them, z represents the input 3D feature and C represents the convolution block.

[0048] The multiple upsampling layers consist of four layers, each of which has a deconvolution layer and several convolution layers with multiple residual connections. The output feature map of the encoding path is fed into a 1*1*1 convolution layer with a sigmoid activation function in the horizontal input branch.

[0049] For the image input to the image segmentation model, pulse encoding is performed. The size of the convolution kernel represents the receptive field. The pixels on the image are defined by neurons, and each neuron has a pulse value. Figure 4 As shown, multiple neurons are used to represent a three-dimensional vector, which represents the pixels in the feature map receptive field. Assuming the size of the receptive field is d*d*d, then the pulse set The definition is as follows: (6) Represents the pixels of all layers in the 1st row and nth column of the receptive field.

[0050] The receptive field is initialized to the initial set of spikes output by the neuron.

[0051] Assume that there is a convolutional layer l, the pulse set input of the convolutional layer l is formula (6), and the output pulse set is , Represents the pixels of all layers in the 1st row and nth column of the output feature, and the output feature size is t*t*t.

[0052] The input of convolutional layer l is initialized as the neuron set The output of the convolutional layer l is represented by a neuron set The output pulse collection. At the postsynaptic membrane Receive from Pulse When , the convolution process with the nonlinear activation function is performed by rule (a). The pulse generated by rule (a) is transmitted to the neuron set , the convolution process ends.

[0053] (a) In the formula, For neurons s pulses transmitted; Postsynaptic membrane The weight of To assist neuronal transmission A neuromodulator, a bias; NL represents a nonlinear function, and the nonlinear function is a ReLU or sigmoid activation function.

[0054] Postsynaptic membrane Receive from Pulse When , the convolution process with the nonlinear activation function is performed through rule (7). The FoMamba layer is implemented through rules (8) to (12) at the postsynaptic membrane arrive implemented in .

[0055] (7) In the formula For neurons s pulses transmitted; W is the postsynaptic membrane The weight of are b neuromodulators that assist neuron transmission, is a bias; NL represents a nonlinear function, which is a ReLU or sigmoid activation function.

[0056] (8) (9) (10) (11) (12) In the formula, , , , Represent the postsynaptic membrane of the forward, backward, upper and lower structured state space models (SSMs), respectively; Represents the postsynaptic membrane , , , Z pulses; Postsynaptic membrane , , , The postsynaptic membrane undergoes integration, Represents the postsynaptic membrane c pulses are generated and transmitted to the next layer of neurons or the postsynaptic membrane.

[0057] Then, in order to improve the feature reuse rate and reduce the loss of effective information, residual connections are applied multiple times in the GSC and FoMamba layers in the network. The pulses after rule (7) and the pulses after rule (12) are input into the residual connection postsynaptic membrane F. Here, the weight w of the postsynaptic membrane F is specified to be 1 (w is omitted), and the residual connections are executed in parallel through rules (13) and (14).

[0058] (13) (14) Residual connections are used in many places in the network. For simplicity, the neuron Mamba / MLP is used to represent the residual connection of the mamba layer or multi-layer perceptron (MLP), which consumes Z pulses, generates Z neuromodulators, and transmits them to the postsynaptic membrane F representing the residual connection.

[0059] Supervised learning through cross entropy loss, neurons In back propagation, the postsynaptic membrane m will adaptively update the weights, continuously optimize the calculations, and ultimately generate a segmentation result similar to the true value.

[0060] (15) Calculate the gradient of the loss function with respect to the output: First, calculate the loss function for the model output The gradient is: (16) The predicted category of the point (d, w, c) output by the model is c. L is the cross entropy loss function, N is the total number of image pixels, D, W, H are the depth, width and height of the image, respectively, indicating the size of the image (or volume data) in three dimensions. C is the category index, and the value range of each pixel is 0 to 2, where 0, 1, and 2 represent background, pancreas, and tumor, respectively. is the one-hot encoding of the true label, is the probability that the model predicts that the position (d,w,h) is of category c.

[0061] Pulse a' is used as the input of the model output layer, and pulse a is used as the output of this layer, which can be expressed as: (17) and They represent the weight and neural modulator of the output layer respectively, and NL is the nonlinear activation function.

[0062] The network is optimized using gradient descent to calculate the gradient and : (18) (19) Wm and b are the weights and neuromodulators of the postsynaptic membrane m, corresponding to the parameter weights and biases in the network. The postsynaptic membrane m represents a network layer m with parameters.

[0063] The update of parameters can be expressed as: (20) (twenty one) in, is the learning rate, and is the updated parameter.

[0064] Therefore, from the above analysis, and Can be applied to the back-propagation algorithm to back-propagate the updated parameters of the output layer Sends neuromodulators to the postsynaptic membrane out and : (twenty two) (twenty three) Then the adaptive weight is applied in the postsynaptic membrane out, and the weight update process is as follows: (twenty four) Neuromodulators It remains in the postsynaptic membrane out and is used to bias the pulse.

[0065] The SSNNP system terminates before reaching the maximum number of iteration steps. The system iteratively optimizes the spikes of the neurons. All weighted spikes of the output neurons are considered as the final segmentation result.

[0066] The image to be segmented in this embodiment can be a medical CT image. Taking the pancreatic CT image as an example, the image segmentation method based on adaptive pulse disclosed in this embodiment is verified. The segmentation task is the 7th subtask in the Medical Segmentation Decathlon (MSD), and the goal is to segment the pancreas and tumor in the CT image. In order to highlight the difficulty of the challenge, the labels of this dataset selected by MSD are extremely unbalanced, which is reflected in the large proportion of background and the small proportion of foreground (pancreas and tumor). Compared with the pancreas, the proportion of tumor is extremely small, which also makes this segmentation task one of the two most difficult tasks of MSD. The MSD pancreatic dataset contains 281 CT images with pancreas and tumor labels. The dataset contains three types of pancreatic tumors-intraductal mucinous tumors, pancreatic neuroendocrine tumors, and pancreatic ductal adenomas.

[0067] The image segmentation model proposed in this example is trained from scratch without using any pre-trained model. During preprocessing, all CT images are normalized to [0,1] by subtracting the mean obtained from the foreground of the entire training set and dividing by the standard deviation. All images are cropped into small patches of size 64 × 192 × 192 using a batch size of 2, and the initial learning rate is set to 0.01. The network is optimized using the Adam optimizer with a weight decay of 0.0003. During training, data augmentation techniques such as rotation, scaling, elastic deformation, gamma correction, mirroring, and brightness adjustment are used to maximize the dataset and enhance the results, and Gaussian weighting is used to aggregate the predicted patches back to the original resolution. Five-fold cross validation is followed in a random split of 281 cases for training and testing, with 57, 56, 56, 56, and 56 cases applied to each test. DSC shows the overall overlap and similarity between the true and predicted values.

[0068]

[0069] where y s and g represent the predicted voxels and the true voxels, respectively.

[0070] The experimental results are shown in Table 1, which shows the performance of the method disclosed in this embodiment and the most advanced methods in terms of the average similarity coefficient (DSC± std) of five-fold cross validation. The best score is shown in bold. The data outside the brackets in Table 1 are DSC, and the data in the brackets in Table 1 are the standard deviations of DSC. MSCFS in Table 1 is a multi-stage segmentation model from coarse to fine, nnUNet is a single-stage segmentation model, Swin UNETR is a pre-trained segmentation model based on transformer, DistillDSM is a 2D segmentation model, and Unetr++ is a segmentation model using a lightweight transformer. It can be seen from Table 1 that compared with the above five methods, the pancreas DSC of the model proposed in this embodiment is improved by 0.24% to 2.78%, and the tumor DSC is improved by 0.88% to 12.35%. We can also observe that compared with the above-mentioned most advanced methods, the method disclosed in this embodiment achieves the best performance in both pancreas and tumor segmentation, and the obtained prediction results are also highly similar to the basic facts.

[0071] Table 1 Performance table

[0072] The method disclosed in this embodiment is used to segment the pancreatic embedded tumor, pancreatic tail tumor and pancreatic surface tumor. The segmentation results are as follows: Figure 5 As shown, the pancreas is marked in red and the tumor is marked in green. Figure 5From top to bottom, they are the segmentation examples of pancreatic embedded tumors, pancreatic tail tumors, and pancreatic surface tumors; Figure 5 From the left column to the right column are CT images, 2D labels, 2D segmentation results, and 3D segmentation results; Figure 5 The 3D segmentation results from top to bottom are: pancreas DSC is 0.8763, tumor DSC is 0.6369; pancreas DSC is 0.8574, tumor DSC is 0.7449; pancreas DSC is 0.8826, tumor DSC is 0.7521. Figure 5 Three 3D visualization cases are shown, demonstrating the effectiveness of the disclosed method, which can accurately segment the pancreas and tumor in various situations such as when the tumor is embedded inside the pancreas and when the tumor is in different locations of the pancreas (tail, upper part).

[0073] like Figure 6 As shown in FIG. 1 , various indicators of the image segmentation model proposed in this embodiment during 1000 rounds of training are: Figure 6 The blue color in the middle is the pancreatic DSC, the green color is the tumor DSC, the red color is the training loss, and the orange color is the validation loss. Among them, since the tumor occupies a small proportion of the image, the validation DSC fluctuates greatly.

[0074] Example 2 In this embodiment, an adaptive pulse-based image segmentation system is disclosed, comprising: An image acquisition module, used to acquire training images; The model training module is used to train the image segmentation model using the training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; The image segmentation module is used to segment the same type of training images using the trained image segmentation model.

[0075] The present invention also discloses a computer device, which includes: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the image segmentation method based on adaptive pulse disclosed in Example 1 is implemented.

[0076] The present invention also discloses a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor and executing the image segmentation method based on adaptive pulse disclosed in Example 1.

[0077] The present invention also discloses a computer program product, which includes a computer program. When the computer program is executed by a processor, the image segmentation method based on adaptive pulse disclosed in Example 1 is implemented.

[0078] The method disclosed in Example 1 can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0079] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0080] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An image segmentation method based on adaptive pulse, characterized in that: include: Get training images; The image segmentation model is trained using training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; Use the trained image segmentation model to segment the same images as the training images.

2. The image segmentation method based on adaptive pulse according to claim 1, characterized in that: The image segmentation model includes multi-layer downsampling layers, multi-layer upsampling layers and a segmentation head. The multi-layer downsampling layers extract multi-scale features of the image, and the multi-layer upsampling layers perform fusion upsampling operations on the multi-scale features to obtain image features; the segmentation head is used to segment the image features to obtain the image segmentation result.

3. The image segmentation method based on adaptive pulse according to claim 2, characterized in that: The multi-layer downsampling layer includes a large kernel convolution layer and multiple four-directional Mamba block layers connected in sequence; wherein each four-directional Mamba block layer includes a gated spatial convolution module, a first layer normalization, a four-directional Mamba module, a second layer normalization and a multi-layer perception layer connected in sequence.

4. The image segmentation method based on adaptive pulse according to claim 3, characterized in that: A residual connection is set between the output of the gated spatial convolution module and the output of the four-directional Mamba module in each four-directional Mamba block layer; and a residual connection is set between the output of the four-directional Mamba module and the output of the multi-layer perception layer.

5. The image segmentation method based on adaptive pulse according to claim 3, characterized in that: The gated spatial convolution module includes a first convolution block and a second convolution block in parallel; the output of the first convolution block is multiplied by the output of the second convolution block and then input into the third convolution block; the output of the third convolution block is added to the features of the input gated spatial convolution module and output as the gated spatial convolution module.

6. The image segmentation method based on adaptive pulse according to claim 1, characterized in that: Neurons transmit neuromodulators to the postsynaptic membrane, which are used to update the weights in the postsynaptic membrane.

7. The image segmentation method based on adaptive pulse according to claim 1, characterized in that: Pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rule is: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of the rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators.

8. The image segmentation system based on adaptive pulse is characterized by: include: An image acquisition module, used to acquire training images; The model training module is used to train the image segmentation model using the training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; The image segmentation module is used to segment the same type of training images using the trained image segmentation model.

9. An electronic device, characterized in that: The device comprises: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the adaptive pulse-based image segmentation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the adaptive pulse-based image segmentation method according to any one of claims 1 to 7.

Images