Segmentation network model optimization method, image segmentation method, device, and medium

Abstract

A segmentation network model optimization method, an image segmentation method, a device, and a medium. The segmentation network model optimization method comprises: acquiring feature channels of a first segmentation network model and information transmission relationships between the feature channels; using the feature channels as nodes, and using the information transmission relationships as edges; changing the edges on the basis of a preset rule to form a second segmentation network model having small world characteristics; and training the second segmentation network model on the basis of image data.

Description

Optimization method of segmentation network model, image segmentation method, device and medium Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, the field of data processing technology, and specifically to a segmentation network model optimization method, image segmentation method, device, and medium. Background Art

[0002] The aorta, also known as the great artery, is the main conduit for blood flow throughout the body. Vascular segmentation and visualization play a crucial role in optimizing treatment. Three-dimensional vascular analysis and visualization, in particular, are crucial for clinical diagnosis and treatment. Vascular segmentation in 3D medical images is a prerequisite for this. However, manual segmentation of 3D vessels is time-consuming and laborious due to complex morphological variations and varying imaging modalities and protocols.

[0003] With the development of deep learning technology, its application in the medical field is becoming more and more extensive. The accuracy of segmentation is crucial to medical images and directly affects subsequent diagnosis and treatment. Deep learning tasks often rely on a large amount of labeled data. Currently, there is relatively little labeled data for medical images, which increases the difficulty of applying deep learning in the medical field and leads to poor model performance.

[0004] Summary of the Invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The embodiments of the present disclosure provide a method for optimizing a segmentation network model, an image segmentation method, a device, and a medium.

[0007] On the one hand, an embodiment of the present disclosure provides a segmentation network model optimization method, including: obtaining feature channels of a first segmentation network model and information transfer relationships between the feature channels; using the feature channels as nodes and the information transfer relationships as edges; changing the edges based on preset rules to form a second segmentation network model with small-world characteristics; and training the second segmentation network model based on image data.

[0008] On the other hand, an embodiment of the present disclosure provides an image segmentation method, including:

[0009] Acquire image data; input the image data into the segmentation network model obtained by using the above-mentioned segmentation network model optimization method to obtain a segmented image.

[0010] On the other hand, an embodiment of the present disclosure further provides a computer device, comprising a processor and a memory storing a computer program that can be run on the processor, wherein when the processor executes the program, it implements the aforementioned segmentation network model optimization method, or implements the aforementioned image segmentation method.

[0011] On the other hand, an embodiment of the present disclosure further provides a non-volatile computer-readable storage medium on which program instructions that can be executed on a processor are stored. When the program instructions are executed by the processor, the aforementioned segmentation network model optimization method or the aforementioned image segmentation method is implemented.

[0012] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0013] Summary of the Figures

[0014] The accompanying drawings are intended to provide a further understanding of the technical solutions of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation of the technical solutions of the present disclosure. The shapes and sizes of the components in the drawings do not reflect the actual scale and are intended only to illustrate the contents of the present disclosure.

[0015] FIG1 is a flow chart of a segmentation network model optimization method according to an embodiment of the present disclosure;

[0016] Figure 2 is a structural diagram of a UNet network;

[0017] FIG3 is a schematic diagram of the first convolutional block in the encoder of FIG2 ;

[0018] FIG4 is a flow chart of an image segmentation method according to an embodiment of the present disclosure;

[0019] FIG5 is an overall flow chart of an embodiment of the present disclosure for transforming a UNet network using an NW small-world model to obtain a final segmentation network model;

[0020] Figure 6 is a schematic diagram of the modified UNet network structure;

[0021] FIG7 is an overall flow chart of the embodiment of the present disclosure using the WS small-world model to transform the UNet network to obtain the final segmentation network model;

[0022] FIG8 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure.

[0023] Details

[0024] The present disclosure describes a plurality of embodiments, but the description is exemplary rather than restrictive, and it is apparent to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure. Although many possible feature combinations are shown in the drawings and discussed in the embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0025] The present disclosure includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features, and elements of the present disclosure may also be combined with any conventional features or elements to form a unique inventive solution defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other inventive solutions to form another unique inventive solution defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this disclosure may be implemented individually or in any appropriate combination. Therefore, the embodiments are not subject to other limitations except for the limitations set forth in the appended claims and their equivalents. In addition, various modifications and changes may be made within the scope of protection of the appended claims.

[0026] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation on the claims. In addition, the claims to the method and / or process should not be limited to performing their steps in the order written, and those skilled in the art can readily understand that these orders can be changed and still remain within the spirit and scope of the disclosed embodiments.

[0027] Unless otherwise defined, the technical or scientific terms used in this disclosure have the usual meanings understood by persons of ordinary skill in the art to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are merely used to distinguish different components. In this disclosure, "plurality" may refer to two or more numbers. Words such as "include" or "comprising" mean that the elements or objects preceding the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0028] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and components. The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure, and other structures can refer to the general design.

[0029] The inventors of the present application discovered that when using deep learning technology to perform aortic vessel segmentation on abdominal aortic aneurysm CTA (CT angiography) image data, the segmentation effect is poor.

[0030] The small-world property, also known as the six degrees of separation theory, states that the number of people between any member and any stranger in a social network is no more than six, resulting in a small characteristic path length and a high aggregation coefficient. In a network, the minimum number of edges connecting any two nodes is the path length. The average path length for all pairs of nodes in the network is defined as the characteristic path length of the network, which is a global characteristic of the network. Assuming a node has k edges, the maximum possible number of edges between the k nodes connected by these k edges is k(k-1) / 2. The fraction obtained by dividing the actual number of edges by the maximum possible number of edges is defined as the aggregation coefficient of the node. The average of the aggregation coefficients of all nodes is defined as the aggregation coefficient of the network, which is a local characteristic of the network.

[0031] To this end, the inventors of the present application considered giving the neural network structure for segmentation a small-world characteristic when constructing it, so that information propagation is more efficient and rapid, thereby improving the segmentation effect, and at the same time reducing the cost and complexity of the entire network.

[0032] To this end, an embodiment of the present disclosure provides an optimization method for a segmentation network model, as shown in FIG1 , comprising the following steps:

[0033] Step 11, obtaining the feature channels of the first segmentation network model and the information transfer relationship between the feature channels, with the feature channels as nodes and the information transfer relationship as edges;

[0034] Step 12: changing the edges based on preset rules to form a second segmentation network model with small-world characteristics;

[0035] Step 13: Train the second segmentation network model based on the image data to obtain a trained segmentation network model for segmenting the image data.

[0036] In the embodiment of the present disclosure, by using the characteristic channels in the segmentation network model as nodes in the network topology structure, it can be optimized (or transformed) according to the small-world model, so that the segmentation network model has small-world characteristics, that is, it has high aggregation and short average path length, which can improve the information transmission efficiency and fault tolerance of the segmentation network model, and ultimately improve the segmentation effect of the segmentation network model.

[0037] In an exemplary embodiment, the first segmentation network model adopts a UNet network, for example. The UNet network is a fully convolutional neural network with a symmetrical U-shaped structure, including a downsampling path and an upsampling path. The downsampling path includes an encoder (or downsampling block), which gradually reduces the image resolution by stacking convolution layers and pooling layers to capture feature map information at different scales. The upsampling path includes an encoder (or upsampling block), which gradually restores the resolution by stacking convolution layers and upsampling operations, and merges the features from the downsampling path with the corresponding high-resolution features to retain detail information. Figure 2 is a structural diagram of a UNet network. The left side of the figure is an encoder for feature extraction, using convolution (conv, represents convolution) and pooling ( The decoder is used to extract features. The decoder includes multiple deconvolution blocks (up-conv, Denotes deconvolution). The decoder concatenates the up-sampled feature map with the left feature map (concatenate, in the figure The encoder's pooling layer operation loses image information and reduces image resolution, which affects the image segmentation task. Upsampling allows a low-resolution image containing high-level abstract features to be converted to high resolution while retaining these features. The image is then spliced ​​with the high-resolution image with low-level surface features on the left. The resulting image undergoes two convolution operations to generate a feature map, which is then classified using two 1*1 convolutions to obtain the final heatmap. The first heatmap represents the score of the first category, and the second heatmap represents the score of the second category. These heatmaps serve as input to a normalization function (such as the softmax function), which calculates the probability distribution.

[0038] In an exemplary embodiment, the feature channel is used as a node, which means that each feature map in the UNet network is used as a node (or neuron). The numbers in Figure 2 represent the number of feature channels. Figure 3 is a schematic diagram of the first module in the encoder in Figure 2. Taking Figure 3 as an example, the module includes two convolution layers. An input image is convolved once to obtain 64 feature maps, that is, 64 feature channels. After a second convolution, 64 feature maps are obtained. For this convolution block, it can be regarded as a network structure with three layers, with 1 node in the first layer, 64 nodes in the second layer, and 64 nodes in the third layer. The amount of data in the feature channel depends on the convolution kernel. This is only an example and is not limited in this article.

[0039] In an exemplary embodiment, the method of modifying the edges to form a second segmentation network model with small-world characteristics based on a preset rule can be based on NW small-world model attributes, or based on WS small-world model attributes, i.e., the preset rule includes either the NW small-world model attributes or the WS small-world model attributes. Using either the NW small-world model or the WS small-world model for modification can improve the randomness of the segmentation network model, enhance the performance and generalization ability of the neural network, form a more optimal network topology, and enhance the segmentation effect.

[0040] Among them, the NW small-world model was proposed by Newman and Watts. By implementing "randomized edge addition" on the network, it is very easy to ensure the connectivity of the network. Randomized edge addition refers to adding an edge between a randomly selected pair of nodes with a certain probability. Among them, there can be at most one edge between any two different nodes, and each node cannot have an edge connected to itself. The WS small-world model was proposed by Watts and Strogatz. By implementing "randomized reconnection" on the network, it is very easy to ensure the dense connectivity of the network. Randomized reconnection refers to randomly reconnecting each edge in the network with a certain probability, that is, keeping one end point of the edge unchanged and taking the other end point to a randomly selected node in the network. Among them, there can be at most one edge between any two different nodes, and each node cannot have an edge connected to itself.

[0041] Exemplarily, altering the edges based on the NW small-world model attributes to form a second segmented network model with small-world properties can involve adding edges between randomly selected node pairs with a first probability p1 to obtain a second segmented network. Whether the second segmented network model exhibits small-world properties can be determined by calculating a performance metric of the network model, such as information transfer efficiency and / or fault tolerance. If the performance metric meets a preset threshold (small-world attribute characteristic metric), the second segmented network model is considered to exhibit small-world properties. If the performance metric does not meet the preset threshold, edges are further added between randomly selected node pairs with the first probability p1 to continue determining whether the second segmented network model exhibits small-world properties. This process is repeated until the performance metric meets the threshold. When modifying a segmented network using the NW small-world model, the modification can be performed on a module-by-module basis or on the entire network. If the modification is performed on a module-by-module basis, the module is treated as a subnetwork when calculating the performance metric, and the performance metric of the subnetwork is calculated to determine whether the module as the subnetwork exhibits small-world properties. When multiple subnetworks exhibit the small-world property, the network formed by these subnetworks also exhibits the small-world property. In an exemplary embodiment, if some subnetworks exhibit the small-world property, it is also possible to determine whether the entire network exhibits the small-world property. By optimizing each module individually, the complexity of the optimized network can be reduced, facilitating subsequent segmentation calculations.

[0042] Exemplarily, based on the WS small-world model attribute, the edges are changed to form a second segmented network model with small-world characteristics. This can be done by disconnecting the edges between randomly selected node pairs with a second probability p2, and connecting the disconnected nodes to nodes randomly selected with a third probability p3 while ensuring the direction of information propagation (or data transmission direction). The disconnected nodes can be randomly connected to the lower layer or cross-layer connected to obtain a second segmented network model. In order to achieve the purpose of pruning, p2>p3 is set. For example, randomly selecting node pairs AB and node pairs CD to disconnect edges, and the data transmission direction is A->B, C->D, disconnecting the connection between nodes A and B, and between nodes C and D, and randomly selecting node E, A connecting to E, thereby achieving pruning. Similarly, determining whether the second segmented network model exhibits the small-world property can be performed, for example, by calculating performance indicators of the network. These indicators can be, for example, information transfer efficiency and / or fault tolerance. If the performance indicators meet a preset threshold, the second segmented network model is considered to exhibit the small-world property. If the performance indicators do not meet the preset threshold, the edges between randomly selected node pairs are disconnected with a second probability p2, and the disconnected nodes are connected to randomly selected nodes with a third probability p3. The determination of whether the second segmented network model exhibits the small-world property is repeated until the performance indicators meet the threshold. When modifying the segmented network using the WS small-world model, the modification can be performed on a module-by-module basis, or on the entire network. If the modification is performed on a module-by-module basis, the module is treated as a subnetwork when calculating the performance indicators, and the performance indicators of the subnetwork are calculated to determine whether the module as the subnetwork exhibits the small-world property. When multiple subnetworks exhibit the small-world property, the network formed by these subnetworks also exhibits the small-world property. In an exemplary embodiment, if some subnetworks exhibit the small-world property, it is also possible to determine whether the entire network exhibits the small-world property. By optimizing each module individually, the complexity of the optimized network can be reduced, facilitating subsequent segmentation calculations.

[0043] In the above embodiment, information transfer efficiency and / or fault tolerance are used as network performance indicators to measure whether the second segmented network model has the small-world characteristic. In other examples, other performance indicators may be used. The information transfer efficiency is determined based on the number of nodes in the network and the shortest path between any two nodes. The fault tolerance is determined based on the number of nodes in the network, the number of nodes directly connected to any node, and the shortest path length between two adjacent nodes when the connection between the two nodes is disconnected.

[0044] In an exemplary embodiment, in order to make the second segmented network satisfy the small-world property, in the process of changing the edges based on preset rules to form the second segmented network model with the small-world property, there can be at most one edge between any two different nodes, and each node cannot have an edge connected to itself.

[0045] In an exemplary embodiment, the training of the second segmentation network model based on the image data can adopt an evolutionary training method to train the second segmentation network model. The evolutionary training process may, for example, include: removing the edge with the smallest weight each time, and randomly disconnecting any edge in the network with a fourth probability p4, and randomly connecting the disconnected nodes with a fifth probability p5, and repeating the above steps until a preferred segmentation network topology is formed. The weights of the above paths can be calculated using existing technologies. By removing the path with the smallest weight, the path that has no effect on the segmentation effect can be removed, thereby simplifying the network. By randomly disconnecting the path, randomness is increased, and the performance and generalization ability of the neural network can be improved. Experiments have found that the segmentation effect is better after adding the step of randomly disconnecting the path. The nodes disconnected during random connection can be randomly connected to the lower layer or cross-layer connection.

[0046] In an exemplary embodiment, in addition to adopting evolutionary training, the use of image data to train the transformed segmentation network model may be to adopt an iterative training method to train the transformed segmentation network model. By using the training data for training multiple times in iterations, the parameters of the model are updated in each iteration to gradually improve the performance. An iterative cycle includes the process of calculating the predicted value, calculating the loss function (Loss Function) to measure the difference between the predicted value and the true value, and updating the network model parameters. Iterative training gradually optimizes the model through multiple iterations until a preset stopping condition is reached, such as convergence or the maximum number of iterations is reached. In this example, the loss function can be obtained by combining the cross entropy loss function and the Dice loss function, that is, a cross entropy-Dice hybrid loss function can be used. The cross entropy loss function converges faster in the early stage of training, and the Dice loss function performs better for unbalanced samples.

[0047] Exemplarily, the above-mentioned evolutionary training and iterative training methods can be used in combination.

[0048] In an exemplary embodiment, the probability ranges from 0 to 1. When the probability is 0, it is equivalent to disconnection.

[0049] The disclosed embodiment proposes to optimize the UNet network structure based on the small-world property of a complex network, and uses an evolutionary training method to generate a solution for segmenting the network structure, which can improve the performance of network segmentation. The brain network has the small-world property of a complex network, and has the characteristics of high information transmission efficiency and fast processing speed. Based on the above characteristics, the small-world property is used to optimize the UNet network model from the channel level: based on the transformation of the NW small-world property, a UNet network topology structure with complex network characteristics is generated, which can improve the efficiency of information transmission between network nodes; based on the transformation of the WS small-world property, a UNet network topology structure with small-world characteristics is generated, which can improve the efficiency of information transmission in the network and reduce the number of parameters of the network model. Using the evolutionary training method during the training process to remove the edge with the smallest weight and randomly disconnect any edge in the network can further increase the randomness of the network structure, form a better network topology structure, and improve the effect of aorta segmentation.

[0050] The present disclosure also provides an image segmentation method, as shown in FIG4 , comprising the following steps:

[0051] Step 21, acquiring image data;

[0052] The image data may be, for example, medical image data. In other application scenarios, other image data may be used, as long as similar data is used for training.

[0053] Step 22: input the image data into the segmentation network model obtained by the method of the above embodiment to obtain a segmented image.

[0054] Taking the UNet network as an example, since the UNet network is a pixel-level classification network, its output is the category of each pixel. Pixels of different categories will display different colors. The UNet output is the segmented image, and the UNet network output result can also locate the position of the target category.

[0055] For example, the UNet network can also use a sliding window to provide the surrounding area (patch) of a pixel as input to predict the class label of each pixel. Since the input training data is the surrounding area, it is equivalent to data augmentation, thus solving the problem of the small number of biomedical images. As a result, the UNet structure allows the network to use fewer training images without reducing the accuracy of segmentation.

[0056] In an exemplary embodiment, the image data includes medical image data, including but not limited to CTA image data, X-ray image data, computed tomography (CT) image data, magnetic resonance imaging (MRI) image data, ultrasound image data, and positron emission tomography (PET) image data, and the segmented image includes one or more of the following: blood vessels, organs, lesions, and suspected lesions. The lesion can be an abnormal area or lesion in human tissue or organs that has been confirmed to be a disease; the suspected lesion refers to an abnormal area or lesion in human tissue or organs that is suspected to be a disease. Experiments have found that the segmentation network model optimized using this embodiment has a good segmentation effect on CTA images, and the segmentation of image edges is more accurate.

[0057] In an exemplary embodiment, an application scenario is provided, in which the optimized image segmentation model can be pre-installed in a computer device, which can be placed in a hospital or a research institute. To make it easier for users to use, an operating interface can be designed to receive medical image data input by the user and receive instructions for segmenting the medical image data input by the user in the operating interface. The medical image data input by the user is input into the pre-installed optimized segmentation network model, and the segmentation network model outputs one or more of the blood vessels, organs, lesions, and suspected lesions segmented from the medical image data for hospital doctors or researchers in research institutes to perform the next step. For example, it can be used for doctors to diagnose diseases. The images obtained using the image segmentation method of this embodiment, especially the image edges, are more accurate, making it easier for doctors or researchers to perform the next step.

[0058] The method of the above embodiment is described below by taking medical image data as an example through an application example.

[0059] The optimization process of the segmentation network model is described by taking the NW small-world model as an example to transform the segmentation network model. The overall flow chart can be seen in Figure 5, which may include the following steps:

[0060] Step 1: Modify the UNet segmentation model to construct a segmentation network model that is more suitable for medical image segmentation

[0061] Construct an initial UNet network, which can adopt an existing UNet structure model. Each feature channel in the UNet network is regarded as a neuron. A node pair is randomly selected with a first probability p1 in the module, and an edge (hereinafter referred to as a path) is added between the node pairs. There can be at most one path between any two different nodes, and each node cannot have a path connected to itself.

[0062] In this example, the modules used for feature extraction and feature fusion in the UNet network are modified as units. In other examples, you can customize which modules are optimized.

[0063] Determine whether the modified network (which can be the entire UNet network or a module or sub-network in the UNet network) satisfies the following formulas (1) and (2):

[0064] Where: D Global Related to L, L is the characteristic path length, D Global represents the information transmission efficiency between nodes i and j, N is the number of neurons (i.e. nodes) in the network, and d ij is the shortest path between any two neurons.

[0065] Where: D Local Related to 1 / C, C is the aggregation coefficient, D Local Represents the fault tolerance of the network, N is the number of neurons in the network, N i is the number of neurons directly connected to node i, d kl It is the shortest path length between node k and its adjacent node l after the two nodes are disconnected.

[0066] When D Global and D Local are all less than their corresponding preset thresholds (D Global The corresponding preset thresholds are the information transmission efficiency threshold and D Local When the corresponding preset threshold is the fault tolerance threshold), the UNet network satisfies the small-world property.

[0067] When the UNet network does not meet the small-world property, including the current information transfer efficiency value D Global Does not meet the information transmission efficiency threshold, or the current fault tolerance value D Local If the fault tolerance threshold is not met, an edge is added between randomly selected node pairs with the first probability p1. The modified UNet is then checked to see if both its information transfer efficiency and fault tolerance are below the preset threshold. This process continues until the modified UNet satisfies the small-world property. The final number of paths added is determined based on the experimental results of the current task.

[0068] The modified UNet network structure is shown in Figure 6. After the modification, cross-layer connections are added. Indicates the path added to the topology. For the first module of UNet, for example, a path from the first layer to the third layer is added. This is equivalent to accumulating the data of the first layer when calculating the feature map of the third layer by accumulating the data of the second layer. That is, the data of the first layer is included in the summation process of the final convolution result.

[0069] Step 2: Perform evolutionary training on the modified UNet network

[0070] Taking the application of UNet network to aortic vessel segmentation as an example, the UNet network obtained in step 1 is trained for aortic segmentation using the evolutionary training method.

[0071] The detailed operation is as follows: within a module, at each iteration, remove the path with the smallest weight, randomly disconnect any path in the network, and randomly connect the disconnected nodes with a fourth probability p4 (disconnected nodes can be randomly connected to the next layer or cross-layer connections). Then, use 5-fold cross-validation to train the model, and repeat these steps until the final optimal network topology is achieved. This iteration can be performed module by module to optimize each module.

[0072] In the 5-fold cross-validation, the abdominal aortic aneurysm CTA imaging dataset was divided into 5 subsets of equal size, 4 of which were used as training sets and the remaining 1 subset was used as a validation set. The 4 training sets were used for model training, and the validation set was used to evaluate the performance of the model. This process was repeated 5 times, each time using a different subset as the validation set and the other subsets as the training set. Finally, the results of the 5 validations were averaged to obtain the final performance index. Using the 5-fold cross-validation method, by randomly dividing the dataset multiple times and performing training and validation, the performance of the model can be evaluated more accurately and the dependence on specific data partitioning can be reduced. This method can also help detect the stability of the model, that is, whether the model performs consistently on different data subsets.

[0073] When using the validation set to evaluate the model characteristics, the following loss function L can be used to determine whether the UNet network model has converged. The loss function L in this embodiment is a cross entropy-Dice hybrid loss function: L = L1 + L2

[0074] Among them, L1 is the cross entropy loss function:

[0075] L2 is the Dice loss function:

[0076] In the above formula, N is the number of pixels, y is the ground truth, For the prediction results.

[0077] Repeat the above steps until the final UNet network topology for blood vessel segmentation is obtained.

[0078] After obtaining the UNet network topology, the trained model can be tested using a test set of abdominal aortic aneurysm CTA images.

[0079] This embodiment optimizes the UNet network topology based on the small-world properties of complex networks, constructing a network topology with high inter-node information transmission efficiency and fault tolerance, thereby improving the segmentation performance of the segmentation network model and making segmentation more accurate. Furthermore, this embodiment uses an evolutionary training method to optimize the network structure during training, thereby generating a network topology that best suits the current segmentation task. This can improve the segmentation performance of the segmentation network model and make segmentation more accurate.

[0080] The optimization process of the segmentation network model is described by taking the transformation of the segmentation network model using the WS small-world model as an example. The overall flow chart can be seen in Figure 7, which may include the following steps:

[0081] Step 1: Construct the initial UNet network

[0082] An initial UNet network is constructed. The network may adopt an existing UNet network model. Abdominal aorta CTA image data is input into the initial UNet network. The UNet network is preliminarily trained to obtain a trained UNet network structure.

[0083] The initial training of the UNet network can be achieved by using the existing UNet network training method.

[0084] Step 2: Transform the trained initial UNet network

[0085] In this example, the WS small-world model is used to transform the segmentation network model to obtain a segmentation network model more suitable for medical image segmentation. Since the WS small-world model is used to transform the segmentation network model, the UNet network model is pruned. Therefore, the initial UNet network is preliminarily trained before the transformation, and then the trained UNet network is transformed to obtain better segmentation accuracy.

[0086] The transformation operation of the trained UNet network is as follows: each feature channel in the UNet network (or the subnetwork or module in the UNet) is regarded as a neuron, and the paths with smaller weights are processed as follows in each module according to a preset ratio (the ratio value can be determined according to the task): a node pair is randomly selected with a second probability p2, the path between the node pair is disconnected, and the disconnected node and the randomly selected node are connected with a third probability p3. The disconnected node can be randomly connected to the lower layer or cross-layer connected, where p2>p3 to ensure that the number of disconnected paths is greater than the number of connected paths, there can be at most one path between any two different nodes, and each node cannot have a node connected to itself.

[0087] To determine whether the modified UNet network satisfies the small-world property, the information transmission efficiency and fault tolerance can be calculated according to the above formula (1) and / or formula (2), and then the preset information transmission efficiency threshold and fault tolerance threshold are used to determine whether the UNet network satisfies the small-world property. If not, the above modification operation is repeated until the modified UNet network satisfies the small-world property. The detailed calculation and judgment methods are the same as those in the previous embodiment and will not be repeated here.

[0088] Step 3: Iteratively train the modified UNet network

[0089] In this example, the model is trained using a 5-fold cross-validation method. In 5-fold cross-validation, the abdominal aortic aneurysm CTA image dataset is divided into five equally sized subsets, four of which serve as training sets, and the remaining subset serves as a validation set. The model is trained using the four training sets, and the validation set is used to evaluate model performance. This process is repeated five times, each time using a different subset as the validation set and the remaining subsets as the training set. Finally, the results from the five validation runs are averaged to obtain the final performance metric.

[0090] When using the validation set to evaluate model characteristics, the following loss function L can be used to determine whether the UNet network model converges: L = L1 + L2

[0091] Wherein, L1 is the cross entropy loss function, and L2 is the Dice loss function. The calculation formula is the same as that of the previous embodiment and will not be repeated here.

[0092] This embodiment prunes the UNet network based on the WS small-world property, thereby constructing a model with high inter-node information transmission efficiency and few network parameters, which can reduce the number of model parameters, shorten the calculation time, and improve the segmentation effect.

[0093] In an exemplary embodiment, Figure 8 is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. As shown in Figure 18, the device 80 includes: at least one processor 801; at least one memory 802 and a bus 803 connected to the processor 801; wherein the processor 801 and the memory 802 communicate with each other via the bus 803; the processor 801 is configured to call program instructions in the memory 802 to execute the steps of the classification model generation method or classification method in any of the above embodiments.

[0094] The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a transistor logic device, etc., and this disclosure does not limit this.

[0095] Memory can include read-only memory (ROM) and random access memory (RAM), and provides instructions and data to the processor. Some memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0096] In addition to the data bus, the bus may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are all labeled as buses in FIG8 .

[0097] During implementation, the processing performed by the processing device can be completed by hardware integrated logic circuits in the processor or by instructions in the form of software. That is, the method steps of the embodiments of the present disclosure can be embodied as being executed by a hardware processor, or by a combination of hardware and software modules in the processor. The software module can be located in a storage medium such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0098] In an exemplary embodiment, the present disclosure also provides a non-volatile computer-readable storage medium on which a computer program that can be run on a processor is stored. When the computer program is executed by the processor, the steps of the aforementioned smart branch application management method are implemented.

[0099] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the functional modules / units mentioned in the above description are not divided equally; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0100] Although the embodiments disclosed in this disclosure are as described above, the contents described are merely embodiments adopted to facilitate understanding of the disclosure and are not intended to limit the disclosure. Any person skilled in the art to which the disclosure belongs may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope of the disclosure. However, the scope of patent protection of the disclosure shall still be based on the scope defined by the attached claims.

Claims

1. An optimization method for segmentation network model, include: Acquire a feature channel of a first segmentation network model and an information transfer relationship between the feature channels; The characteristic channel is used as a node, and the information transmission relationship is used as an edge; Changing the edge based on a preset rule to form a second segmentation network model with a small-world characteristic; The second segmentation network model is trained based on the image data.

2. The optimization method according to claim 1, in, The preset rules include: NW small world model properties, or WS small world model properties.

3. The optimization method according to claim 2, in, The step of changing the edge based on the NW small-world model attribute to form a second segmentation network model with a small-world characteristic includes: Adding edges between randomly selected node pairs with a first probability, determining whether the second segmentation network model has the small-world characteristic, and if it does not have the small-world characteristic, continuing to add edges between randomly selected node pairs with the first probability until the second segmentation network model has the small-world characteristic.

4. The optimization method according to claim 2, in, The step of changing the edge based on the WS small-world model attribute to form a second segmentation network model with a small-world characteristic includes: Disconnect the edges between the randomly selected node pairs with the second probability, and connect the disconnected nodes with the nodes randomly selected with the third probability, and determine whether the second segmentation network model has the small-world characteristic. If it does not have the small-world characteristic, continue to disconnect the edges between the randomly selected node pairs with the second probability, and connect the disconnected nodes with the nodes randomly selected with the third probability until the second segmentation network model has the small-world characteristic.

5. The optimization method according to claim 3 or 4, in, In the process of changing the edge based on the preset rule to form the second segmentation network model with small-world characteristics, there can be at most one edge between any two different nodes, and each node cannot have an edge connected to itself.

6. The optimization method according to claim 3 or 4, in, The determining whether the second segmentation network model has the small-world characteristic comprises: It is determined whether the information transmission efficiency of the second segmentation network model meets the information transmission efficiency threshold, and / or whether the fault tolerance of the second segmentation network model meets the fault tolerance threshold. If both are satisfied, the second segmentation network model has the small-world characteristic.

7. The optimization method according to claim 6, in, The information transmission efficiency is determined based on the number of nodes in the network and the shortest path between any two nodes.

8. The optimization method according to claim 6, in, The fault tolerance is determined based on the number of nodes in the network, the number of nodes directly connected to any node, and the length of the shortest path between two adjacent nodes when the connection between the two nodes is disconnected.

9. The optimization method according to claim 1, in, The training of the second segmentation network model based on the image data includes: The second segmentation network model is trained using an evolutionary training method.

10. The optimization method according to claim 9, in, The method of adopting evolutionary training to train the second segmentation network model includes: Each time, the edge with the smallest weight is removed, and any edge in the network is randomly disconnected with the fourth probability, and the fifth probability is The disconnected nodes are randomly connected, the loss function of the second segmentation network model is calculated, and the above steps are repeated until the second segmentation network model converges.

11. The optimization method according to claim 10, in, The loss function is a cross entropy-Dice hybrid loss function.

12. An image segmentation method, include: Acquire image data; The image data is input into a segmentation network model obtained by the segmentation network model optimization method according to any one of claims 1 to 11 to obtain a segmented image.

13. The image segmentation method according to claim 12, in, The image data includes medical image data, and the segmented image includes one or more of the following: blood vessels, organs, lesions, and suspected lesions.

14. The image segmentation method according to claim 12, in, The acquiring of image data comprises: receiving medical image data input by a user; Before inputting the image data into the segmentation network model obtained by the segmentation network model optimization method according to any one of claims 1 to 11, the method further comprises: receiving an instruction for segmenting the medical image data input by a user in an operation interface; The obtaining of the segmented image includes: obtaining one or more of the blood vessels, organs, lesions and suspected lesions segmented from the medical image data.

15. A computer device comprising a processor and a memory storing a computer program executable on the processor, in, When the processor executes the program, the steps of the method according to any one of claims 1 to 11 or any one of claims 12 to 14 are implemented.

16. A non-transitory computer-readable storage medium storing program instructions, which can implement the method of any one of claims 1 to 11 or any one of claims 12 to 14 when executed.

Images