Image segmentation method, system, medium and device based on adaptive pulse

Abstract

The image segmentation method, system, medium and device based on adaptive pulse disclosed by the present invention belong to the field of image segmentation technology. The method comprises obtaining a training image; training an image segmentation model using the training image, and obtaining a trained image segmentation model after the training is completed; wherein, when the image segmentation model is trained, the gradient information in the back propagation process of the model is calculated; the gradient information and the parameter weight of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; the image segmentation model is updated using the postsynaptic membrane and the neurons; and the image segmentation model is used to segment the same type of images of the training image. The problem that the model weight cannot be updated when the current image segmentation model is trained is solved, and the training accuracy of the image segmentation model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular, to an image segmentation method, system, medium and device based on adaptive pulses. Background Art

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] A neural membrane system (neural P system, SNP system) is a membrane computing model that encodes information over time using a spiking neural network. The neural membrane system combines the idea of spiking neurons and can be regarded as a third-generation artificial neural network model. It can well simulate the processing mechanisms of some biological nervous systems and has a powerful distributed framework to complete various high-performance parallel computations. Currently, most of the research on neural membrane systems is still theoretical research. How to utilize the combined advantages of different disciplines to propose new membrane computing models, solve various practical problems, and broaden the application scope of membrane computing is an important and valuable topic in the field of membrane computing research.

[0004] Currently, an image segmentation model is mainly constructed through a neural membrane system, and then the image is segmented through the image segmentation model. When the existing image segmentation model is trained, it only has one type of computing unit, namely neurons, and the model is updated only through the computation of neurons.

[0005] The current training method of this kind of image segmentation model cannot adaptively update the model weights, resulting in limited training accuracy of the model, and thus limited accuracy of image segmentation. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes an image segmentation method, system, medium and device based on adaptive pulses, which improves the accuracy of image segmentation.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] In the first aspect, an image segmentation method based on adaptive pulses is proposed, including:

[0009] Obtain training images;

[0010] Use the training images to train an image segmentation model. After training is completed, a trained image segmentation model is obtained. Among them, when training the image segmentation model, calculate the gradient information in the reverse propagation process of the model; use the gradient information and the parameter weights of the current model as the postsynaptic membrane; calculate the neurons in the forward propagation process of the model; use the postsynaptic membrane and the neurons to update the image segmentation model;

[0011] Use the trained image segmentation model to segment the same images as the training images.

[0012] Furthermore, the image segmentation model includes a multi-layer downsampling layer, a multi-layer upsampling layer and a segmentation head. The multi-layer downsampling layer extracts multi-scale features of the image, and the multi-layer upsampling layer performs a fusion upsampling operation on the multi-scale features to obtain image features; the segmentation head is used to segment the image features to obtain an image segmentation result.

[0013] Furthermore, the multi-layer downsampling layer includes a large kernel convolution layer and multiple four-directional Mamba block layers connected in sequence; wherein each four-directional Mamba block layer includes a gated spatial convolution module, a first layer normalization, a four-directional Mamba module, a second layer normalization and a multi-layer perception layer connected in sequence.

[0014] Furthermore, a residual connection is set between the output of the gated spatial convolution module and the output of the four-directional Mamba module in each four-directional Mamba block layer; and a residual connection is set between the output of the four-directional Mamba module and the output of the multi-layer perception layer.

[0015] Furthermore, the gated spatial convolution module includes a first convolution block and a second convolution block in parallel; the output of the first convolution block is multiplied by the output of the second convolution block and input into the third convolution block, and the output of the third convolution block is added to the feature of the input gated spatial convolution module as the output of the gated spatial convolution module.

[0016] Furthermore, the neuron transmits neuromodulators to the postsynaptic membrane, which are used to update the weights in the postsynaptic membrane.

[0017] Furthermore, pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rules are: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rules are: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of the rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators.

[0018] Secondly, an adaptive pulse-based image segmentation system is proposed, including:

[0019] An image acquisition module, used to acquire training images;

[0020] The model training module is used to train the image segmentation model using the training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons;

[0021] The image segmentation module is used to segment the same type of training images using the trained image segmentation model.

[0022] In a third aspect, a computer device is provided, the device comprising:

[0023] a processor adapted to execute a computer program;

[0024] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the image segmentation method based on adaptive pulse proposed in the first aspect is implemented.

[0025] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the adaptive pulse-based image segmentation method proposed in the first aspect.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention proposes an image segmentation method, system, medium and device based on adaptive pulses. When training an image segmentation model, the method calculates the gradient information of the model during the back propagation process; uses the gradient information and the parameter weights of the current model as the postsynaptic membrane; calculates the neurons during the forward propagation process of the model; uses the postsynaptic membrane and the neurons to update the image segmentation model, and uses the neurons and the postsynaptic membrane to transmit pulses. The postsynaptic membrane represents the parameters of various weights that need to be updated, thereby improving the diversity of communication between neurons; and thus improving the accuracy of image segmentation.

[0028] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0030] Figure 1The figure is an overall block diagram of the image segmentation method based on adaptive pulse disclosed in the embodiment;

[0031] Figure 2 It is a block diagram of the FSMamba block layer structure disclosed in the embodiment;

[0032] Figure 3 It is a structural block diagram of the gated spatial convolution module disclosed in the embodiment;

[0033] Figure 4 A schematic diagram of a pulse transmission process disclosed in an embodiment;

[0034] Figure 5 A diagram showing the segmentation effect of the image segmentation method based on adaptive pulse disclosed in the embodiment;

[0035] Figure 6 These are various indicators in the image segmentation model training disclosed in the embodiment. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0038] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0039] Example 1

[0040] This embodiment will start from the spiking neuron mechanism of the spiking neural membrane system and build a new spiking neuron model with reference to the second-generation neural network model, called the adaptive spiking neural membrane system with neuromodulators (SSNNP), and introduce it into the recognition of pancreas and tumors in medical image processing.

[0041] The classic SNP system consists of neurons and synapses. There is only one computing unit, neurons, which limits the application capability of the SNP system. In order to improve the accuracy of image segmentation, this embodiment improves the classic SNP system. The improved adaptive spike neural membrane system (SSNNP) includes two computing units, neurons and postsynaptic membranes. The pixels of the image are encoded as pulses in neurons, and then the UNet operation is performed through rules to segment the image.

[0042] In this embodiment, an image segmentation method based on adaptive pulse is disclosed, such as Figure 1-Figure 6 As shown, including:

[0043] Get training images;

[0044] The image segmentation model is trained using training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons;

[0045] Use the trained image segmentation model to segment the same images as the training images.

[0046] Among them, the image segmentation model includes multi-layer downsampling layers, multi-layer upsampling layers and a segmentation head. The multi-layer downsampling layers extract multi-scale features of the image, and the multi-layer upsampling layers perform fusion upsampling operations on the multi-scale features to obtain image features; the segmentation head is used to segment the image features to obtain the image segmentation result.

[0047] The multi-layer downsampling layer includes a large kernel convolution layer and multiple four-directional Mamba block layers connected in sequence; wherein each four-directional Mamba block layer includes a gated spatial convolution module, a first layer normalization, a four-directional Mamba module, a second layer normalization and a multi-layer perception layer connected in sequence.

[0048] A residual connection is set between the output of the gated spatial convolution module and the output of the four-directional Mamba module in each four-directional Mamba block layer; and a residual connection is set between the output of the four-directional Mamba module and the output of the multi-layer perception layer.

[0049] The gated spatial convolution module includes a first convolution block and a second convolution block in parallel; the output of the first convolution block is multiplied by the output of the second convolution block and then input into the third convolution block; the output of the third convolution block is added to the features of the input gated spatial convolution module and output as the gated spatial convolution module.

[0050] Neurons transmit neuromodulators to the postsynaptic membrane, which are used to update the weights in the postsynaptic membrane.

[0051] Pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rule is: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of the rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators.

[0052] In order to adapt the SSNNP system to the segmentation task, this embodiment redefines the rules in neurons and postsynaptic membranes in the SSNN P, including pulse transmission rules, pulse communication rules, and pulse bias rules.

[0053] Among them, the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than the set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane.

[0054] Specifically, the pulse transmission rules are:

[0055] (1)

[0056] When the postsynaptic membrane m contains c pulses, c>= set value; the pulse transmission rule is activated. c pulses are consumed and generated at the same time pulses are sent to subsequent neurons, where is the weight of the postsynaptic membrane m. In addition, the neuron transmits neuromodulators to the postsynaptic membrane, which are used to update the weight of the postsynaptic membrane and increase or decrease the pulse intensity. After the pulse is generated by such a rule weighted in the postsynaptic membrane, it is sent to the neuron behind it. Weight Can be a vector or a single value. The setting value here can be 1.

[0057] The pulse communication rule is: when the number of pulses in a neuron is greater than the set value, a first set number of pulses is generated and sent to subsequent neurons.

[0058] The pulse communication rules are:

[0059] (2)

[0060] When neuron ijk contains s pulses and s >= the set value, the pulse communication rule is activated. The s pulses are consumed, and at the same time, v pulses are generated and sent to the subsequent neurons.

[0061] The pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than the set value and the postsynaptic membrane contains the set number of neuromodulators with the set efficacy suitable for the rule, the third set number of pulses is generated and sent to the subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the neuromodulator efficacy, and the number of neuromodulators.

[0062] The pulse bias rule is:

[0063] (3)

[0064] When the postsynaptic membrane m contains c (c >= 1) pulses, c >= the set value, and there are neuromodulators with the number of and efficacy of suitable for the rule; the pulse bias rule is activated. The c pulses are consumed, and at the same time, pulses are generated and sent to the subsequent neurons, where the set value is 1.

[0065] The topological structure of the pulse neural P system is composed of a directed graph. Each neuron in the system is represented by a node, the synapse connecting two adjacent neurons in the system is represented by an edge, the pulse is represented by the letter a, and the neuron sends pulses through a directed arc. Therefore, the pulse neural P system is an extended model of the tissue-like membrane system. The pulse is the information medium used for communication between object neurons in the general SNP system. The definition of the SSNN P system of order m + n proposed in this embodiment is as follows:

[0066]

[0067] Among them, is the alphabet, represents the pulse, is the neuromodulator; is the neuron, in the form of , ; is the number of initial spikes; is a finite set of pulse communication rules and pulse forgetting rules. The pulse forgetting rule is the rule for clearing unused or inapplicable pulses in the neuron, which is used when proving the universality of the SSNN P system and not used in image segmentation applications; is the postsynaptic membrane, in the form of , ; is the weight of the postsynaptic membrane, which has two forms: fixed weight and adaptive weight; is the neuromodulator contained in the postsynaptic membrane, which is defined as a resident resource in this step and will not be consumed; is a finite set of pulse transmission rules, pulse bias rules, and neuromodulator decomposition rules. The neuromodulator decomposition rule is a rule for clearing unused or inapplicable neuromodulators in neurons. It is used when proving the universality of the SSNN P system and not used in image segmentation applications; are respectively represented as the sets of connections between neurons and between neurons and the postsynaptic membrane; and are input / output neurons or postsynaptic membranes.

[0068] The initial configuration C0 is the state before model initialization and image input, which is composed of the initial states of neurons representing the image and postsynaptic membranes representing the various parameters of the model. The initial configuration of the SSNN P system is composed of neurons representing the image and postsynaptic membranes representing the various parameters of the model ; The conversion from one structure to another is achieved through calculations between neurons and postsynaptic membranes, that is, the execution of pulse communication rules and forgetting rules in neurons and the execution of pulse transmission, pulse bias, and neuromodulator decomposition rules in postsynaptic membranes. In particular, in each postsynaptic membrane, if the number of neuromodulators satisfies the adaptive weight condition, the weight will be adaptively adjusted. Dropout is a technique that randomly ignores neurons during training. In each training iteration, each neuron has a certain probability of being "discarded". The discarded neurons will not participate in the forward propagation and backward propagation of the current training step. Therefore, the network will try to "self-learn" in each round of training, not relying on any specific neuron, thus avoiding overfitting. Similarly, if the number of weighted pulses satisfies the rules in neurons or postsynaptic membranes, the rules in neurons or postsynaptic membranes will be executed. In any computational step, the rules are executed in a maximally parallelized manner. All applicable rules can work simultaneously. If the triggering conditions of several rules are the same, the SSNN P system randomly selects one rule for processing. The SSNN P system maximally parallelizes the transition steps until the output rules of the output neurons are satisfied. The network layer responsible for the segmentation head is represented by the postsynaptic membrane, and the output of the postsynaptic membrane goes to the last layer of neurons, and the output of this layer of neurons is the result of image segmentation.

[0069] This embodiment uses an image segmentation model to identify pulse sequences and obtain the segmentation result of the image to be segmented. Among them, the image segmentation model (MSNet) includes multiple downsampling layers, multiple upsampling layers, and a segmentation head.

[0070] Such as Figure 1 As shown, multiple downsampling layers and multiple upsampling layers correspond one by one; the multiple downsampling layers include a first convolutional layer and three FSMamba block layers (four-direction Mamba block layers); the first convolutional layer uses a large-kernel convolutional layer, and the large-kernel convolutional layer uses depth convolution, with a kernel size of 7×7×7, a padding of 3×3×3, and a stride of 2×2×2. Given a three-dimensional input volume I∈RC×D×H×W, where C represents the number of input channels, the large-kernel convolutional layer extracts the first-scale feature z0∈R32×D / 2×H / 2×W / 2 from the image. The second-scale feature is extracted from the first-scale feature z0 through the first FSMamba block layer, the third-scale feature is extracted from the second-scale feature through the second FSMamba block layer, the fourth-scale feature is extracted from the third-scale feature through the third FSMamba block layer, and the fourth-scale feature is input into the multi-layer upsampling layer. After the multi-layer upsampling layer performs an upsampling operation on the fourth-scale feature, the fourth upsampled feature is obtained. After fusing the fourth upsampled feature and the third-scale feature, an upsampling operation is performed to obtain the third upsampled feature; after fusing the third upsampled feature and the second-scale feature, an upsampling operation is performed to obtain the second upsampled feature; after fusing the second upsampled feature with the first-scale feature, an upsampling operation is performed to obtain the image feature.

[0071] For the m-th layer of the FSMamba block layer, the calculation process can be defined as:

[0072] (4)

[0073] where GSC and FoM represent the gated spatial convolution module and the four-direction Mamba module respectively. L∈{0,1,..., Nm−1}, LN (LayerNom) represents layer normalization, and MLP represents the multi-layer perceptron layer to enrich the feature representation.

[0074] The four-direction Mamba module models the feature dependencies by flattening the 3D features into a 1D sequence. Therefore, in order to extract the spatial relationship before the four-direction Mamba module, this embodiment designs a gated spatial convolution module (GSC), as Figure 3 shown, the input 3D features are fed into the first convolutional block and the second convolutional block. Both the first convolutional block and the second convolutional block contain a normalization, a convolution, and a non-linear layer. The kernel sizes of the first convolutional block and the second convolutional block are 5×5×5 and 3×3×3 respectively. Then these two features are multiplied pixel by pixel to control the information transmission, similar to a gating mechanism. Finally, the convolutional blocks are used to further fuse the features, and the residual connection is used to reuse the input features.

[0075] (5)

[0076] Among them, z represents the input 3D feature and C represents the convolution block.

[0077] The multiple upsampling layers consist of four layers, each of which has a deconvolution layer and several convolution layers with multiple residual connections. The output feature map of the encoding path is fed into a 1*1*1 convolution layer with a sigmoid activation function in the horizontal input branch.

[0078] For the image input to the image segmentation model, pulse encoding is performed. The convolution kernel size represents the receptive field. The pixels on the image are defined by neurons, and each neuron has a pulse value. Figure 4 As shown, multiple neurons are used to represent a three-dimensional vector, which represents the pixels in the feature map receptive field. Assuming the size of the receptive field is d*d*d, then the pulse set The definition is as follows:

[0079] (6)

[0080] Represents the pixels of all layers in the 1st row and nth column of the receptive field.

[0081] The receptive field is initialized to the initial set of spikes output by the neuron.

[0082] Assume that there is a convolutional layer l, the pulse set input of the convolutional layer l is formula (6), and the output pulse set is , Represents the pixels of all layers in the 1st row and nth column of the output feature, and the output feature size is t*t*t.

[0083] The input of convolutional layer l is initialized as the neuron set The output of the convolutional layer l is represented by a neuron set The output pulse collection. At the postsynaptic membrane Receive from Pulse When , the convolution process with the nonlinear activation function is performed by rule (a). The pulse generated by rule (a) is transmitted to the neuron set , the convolution process ends.

[0084] (a)

[0085] In the formula, For neurons s pulses transmitted; Postsynaptic membrane The weight of To assist neuronal transmission A neuromodulator, a bias; NL represents a nonlinear function, and the nonlinear function is a ReLU or sigmoid activation function.

[0086] Postsynaptic membrane Receive from Pulse When , the convolution process with the nonlinear activation function is performed through rule (7). The FoMamba layer is implemented through rules (8) to (12) at the postsynaptic membrane arrive implemented in .

[0087] (7)

[0088] In the formula For neurons s pulses transmitted; W is the postsynaptic membrane The weight of are b neuromodulators that assist neuron transmission, is a bias; NL represents a nonlinear function, which is a ReLU or sigmoid activation function.

[0089] (8)

[0090] (9)

[0091] (10)

[0092] (11)

[0093] (12)

[0094] In the formula, , , , Represent the postsynaptic membrane of the forward, backward, upper and lower structured state space models (SSMs), respectively; Represents the postsynaptic membrane , , , Z pulses; Postsynaptic membrane , , , The postsynaptic membrane undergoes integration, Represents the postsynaptic membrane c pulses are generated and transmitted to the next layer of neurons or the postsynaptic membrane.

[0095] Then, in order to improve the feature reuse rate and reduce the loss of effective information, residual connections are applied multiple times in the GSC and FoMamba layers in the network. The pulses after rule (7) and the pulses after rule (12) are input into the postsynaptic membrane F of the residual connection. Here, it is stipulated that the weight w of the postsynaptic membrane F is 1 (w is omitted), and the residual connection is executed in parallel through rules (13) and (14).

[0096] (13)

[0097] (14)

[0098] The residual connection is used in many places in the network. For simplicity, the neuron Mamba / MLP is used to represent the residual connection of the mamba layer or the multi-layer perceptron (MLP). It consumes Z pulses, generates Z neuromodulators, and transmits them to the postsynaptic membrane F representing the residual connection.

[0099] Through supervised learning with cross-entropy loss, the neuron performs backpropagation, and the postsynaptic membrane m adaptively updates its weight, continuously optimizing the calculation, and finally generating a segmentation result similar to the true value.

[0100] (15)

[0101] Calculate the gradient of the loss function with respect to the output:

[0102] First, calculate the gradient of the loss function with respect to the model output :

[0103] (16)

[0104] The predicted class for the point (d, w, c) of the model output is c. L is the cross-entropy loss function, N is the total number of image pixels, and D, W, H are the depth, width, and height of the image, respectively, representing the sizes of the image (or volume data) in three dimensions. C is the class index, and the value range of each pixel is from 0 to 2, where 0, 1, and 2 represent the background, pancreas, and tumor, respectively. is the one-hot encoding of the true label, and is the probability that the model predicts the class c for the position (d, w, h).

[0105] The pulse a' is used as the input of the model output layer, and the pulse a is used as the output of this layer, which can be expressed as:

[0106] (17)

[0107] and They represent the weight and neural modulator of the output layer respectively, and NL is the nonlinear activation function.

[0108] The network is optimized using gradient descent to calculate the gradient and :

[0109] (18)

[0110] (19)

[0111] Wm and b are the weights and neuromodulators of the postsynaptic membrane m, corresponding to the parameter weights and biases in the network. The postsynaptic membrane m represents a network layer m with parameters.

[0112] The update of parameters can be expressed as:

[0113] (20)

[0114] (twenty one)

[0115] in, is the learning rate, and is the updated parameter.

[0116] Therefore, from the above analysis, and Can be applied to the back-propagation algorithm to back-propagate the updated parameters of the output layer Sends neuromodulators to the postsynaptic membrane out and :

[0117] (twenty two)

[0118] (twenty three)

[0119] Then the adaptive weight is applied in the postsynaptic membrane out, and the weight update process is as follows:

[0120] (twenty four)

[0121] Neuromodulators It remains in the postsynaptic membrane out and is used to bias the pulse.

[0122] The SSNNP system terminates before reaching the maximum number of iteration steps. The system iteratively optimizes the spikes of the neurons. All weighted spikes of the output neurons are considered as the final segmentation result.

[0123] The image to be segmented in this embodiment can be a medical CT image. Taking the pancreatic CT image as an example, the image segmentation method based on adaptive pulse disclosed in this embodiment is verified. The segmentation task is the 7th subtask in the Medical Segmentation Decathlon (MSD), and the goal is to segment the pancreas and tumor in the CT image. In order to highlight the difficulty of the challenge, the labels of this dataset selected by MSD are extremely unbalanced, which is reflected in the large proportion of background and the small proportion of foreground (pancreas and tumor). Compared with the pancreas, the proportion of tumor is extremely small, which also makes this segmentation task one of the two most difficult tasks of MSD. The MSD pancreatic dataset contains 281 CT images with pancreas and tumor labels. The dataset contains three types of pancreatic tumors-intraductal mucinous tumors, pancreatic neuroendocrine tumors, and pancreatic ductal adenomas.

[0124] The image segmentation model proposed in this example is trained from scratch without using any pre-trained model. During preprocessing, all CT images are normalized to [0,1] by subtracting the mean obtained from the foreground of the entire training set and dividing by the standard deviation. All images are cropped into small patches of size 64 × 192 × 192 using a batch size of 2, and the initial learning rate is set to 0.01. The network is optimized using the Adam optimizer with a weight decay of 0.0003. During training, data augmentation techniques such as rotation, scaling, elastic deformation, gamma correction, mirroring, and brightness adjustment are used to maximize the dataset and enhance the results, and Gaussian weighting is used to aggregate the predicted patches back to the original resolution. Five-fold cross validation is followed in a random split of 281 cases for training and testing, with 57, 56, 56, 56, and 56 cases applied to each test. DSC shows the overall overlap and similarity between the true and predicted values.

[0125]

[0126] where y s and g represent the predicted voxels and the true voxels, respectively.

[0127] The experimental results are shown in Table 1, which shows the performance of the method disclosed in this embodiment and the most advanced methods in terms of the average similarity coefficient (DSC± std) of five-fold cross validation. The best score is shown in bold. The data outside the brackets in Table 1 are DSC, and the data in the brackets in Table 1 are the standard deviations of DSC. MSCFS in Table 1 is a multi-stage segmentation model from coarse to fine, nnUNet is a single-stage segmentation model, Swin UNETR is a pre-trained segmentation model based on transformer, DistillDSM is a 2D segmentation model, and Unetr++ is a segmentation model using a lightweight transformer. It can be seen from Table 1 that compared with the above five methods, the pancreas DSC of the model proposed in this embodiment is improved by 0.24% to 2.78%, and the tumor DSC is improved by 0.88% to 12.35%. We can also observe that compared with the above-mentioned most advanced methods, the method disclosed in this embodiment achieves the best performance in both pancreas and tumor segmentation, and the obtained prediction results are also highly similar to the basic facts.

[0128] Table 1 Performance table

[0129]

[0130] The method disclosed in this embodiment is used to segment the pancreatic embedded tumor, pancreatic tail tumor and pancreatic surface tumor. The segmentation results are as follows: Figure 5 As shown, the pancreas is marked in red and the tumor is marked in green. Figure 5 From top to bottom, they are the segmentation examples of pancreatic embedded tumors, pancreatic tail tumors, and pancreatic surface tumors; Figure 5 From the left column to the right column are CT images, 2D labels, 2D segmentation results, and 3D segmentation results; Figure 5 The 3D segmentation results from top to bottom are: pancreas DSC is 0.8763, tumor DSC is 0.6369; pancreas DSC is 0.8574, tumor DSC is 0.7449; pancreas DSC is 0.8826, tumor DSC is 0.7521. Figure 5 Three 3D visualization cases are shown, demonstrating the effectiveness of the disclosed method, which can accurately segment the pancreas and tumor in various situations such as when the tumor is embedded inside the pancreas and when the tumor is in different locations of the pancreas (tail, upper part).

[0131] like Figure 6 As shown in FIG. 1 , various indicators of the image segmentation model proposed in this embodiment during 1000 rounds of training are: Figure 6 The blue color in the middle is the pancreatic DSC, the green color is the tumor DSC, the red color is the training loss, and the orange color is the validation loss. Among them, since the tumor occupies a small proportion of the image, the validation DSC fluctuates greatly.

[0132] Example 2

[0133] In this embodiment, an adaptive pulse-based image segmentation system is disclosed, comprising:

[0134] An image acquisition module, used to acquire training images;

[0135] The model training module is used to train the image segmentation model using the training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons;

[0136] The image segmentation module is used to segment the same type of training images using the trained image segmentation model.

[0137] The present invention also discloses a computer device, which includes:

[0138] a processor adapted to execute a computer program;

[0139] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the image segmentation method based on adaptive pulse disclosed in Example 1 is implemented.

[0140] The present invention also discloses a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor and executing the image segmentation method based on adaptive pulse disclosed in Example 1.

[0141] The present invention also discloses a computer program product, which includes a computer program. When the computer program is executed by a processor, the image segmentation method based on adaptive pulse disclosed in Example 1 is implemented.

[0142] The method disclosed in Example 1 can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0143] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0144] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An image segmentation method based on adaptive pulse, characterized in that: include: Get training images; The image segmentation model is trained using training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; Neurons transmit neuromodulators to the postsynaptic membrane, which are used to update weights in the postsynaptic membrane; Pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rule is: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators; Use the trained image segmentation model to segment the same images as the training images.

2. The image segmentation method based on adaptive pulse according to claim 1, characterized in that: The image segmentation model includes multi-layer downsampling layers, multi-layer upsampling layers and a segmentation head. The multi-layer downsampling layers extract multi-scale features of the image, and the multi-layer upsampling layers perform fusion upsampling operations on the multi-scale features to obtain image features; the segmentation head is used to segment the image features to obtain the image segmentation result.

3. The image segmentation method based on adaptive pulse according to claim 2, characterized in that: The multi-layer downsampling layer includes a large kernel convolution layer and multiple four-directional Mamba block layers connected in sequence; wherein each four-directional Mamba block layer includes a gated spatial convolution module, a first layer normalization, a four-directional Mamba module, a second layer normalization and a multi-layer perception layer connected in sequence.

4. The image segmentation method based on adaptive pulse according to claim 3, characterized in that: A residual connection is set between the output of the gated spatial convolution module and the output of the four-directional Mamba module in each four-directional Mamba block layer; and a residual connection is set between the output of the four-directional Mamba module and the output of the multi-layer perception layer.

5. The image segmentation method based on adaptive pulse according to claim 3, characterized in that: The gated spatial convolution module includes a first convolution block and a second convolution block in parallel; the output of the first convolution block is multiplied by the output of the second convolution block and then input into the third convolution block; the output of the third convolution block is added to the features of the input gated spatial convolution module and output as the gated spatial convolution module.

6. The image segmentation system based on adaptive pulse is characterized by: include: An image acquisition module, used to acquire training images; The model training module is used to train the image segmentation model using the training images, and after the training is completed, a trained image segmentation model is obtained; wherein, when the image segmentation model is trained, the gradient information of the model in the back propagation process is calculated; the gradient information and the parameter weights of the current model are used as the postsynaptic membrane; the neurons in the forward propagation process of the model are calculated; and the image segmentation model is updated using the postsynaptic membrane and the neurons; Neurons transmit neuromodulators to the postsynaptic membrane, which are used to update weights in the postsynaptic membrane; Pulse communication rules are set in neurons; pulse transmission rules and pulse bias rules are set in postsynaptic membranes; the pulse communication rule is: when the number of pulses in a neuron is greater than a set value, a first set number of pulses is generated and sent to subsequent neurons; the pulse transmission rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, a second set number of pulses is generated and sent to subsequent neurons, and the second set number is determined according to the number of pulses contained in the postsynaptic membrane and the weight of the postsynaptic membrane; the pulse bias rule is: when the number of pulses contained in the postsynaptic membrane is greater than a set value, and the postsynaptic membrane contains a neuromodulator with a set efficacy suitable for a set number of rules, a third set number of pulses is generated and sent to subsequent neurons, and the third set number is determined according to the number of pulses contained in the postsynaptic membrane, the weight of the postsynaptic membrane, the efficacy of the neuromodulator and the number of neuromodulators; The image segmentation module is used to segment the same type of training images using the trained image segmentation model.

7. An electronic device, characterized in that: The device comprises: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the adaptive pulse-based image segmentation method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the adaptive pulse-based image segmentation method according to any one of claims 1 to 5.

Images