An Interactive Medical Image Segmentation Method and System Based on Graph Convolutional Neural Network

Abstract

The present invention discloses an interactive medical image segmentation method and system based on a graph convolutional neural network, which performs image segmentation through user interaction and algorithm collaboration, uses two stages of pre-segmentation and interactive segmentation to achieve satisfactory segmentation results, and realizes interactive segmentation of medical images by modeling the object contour through a graph convolutional neural network model. This method can solve the problems of medical image segmentation in the scenario of lack of training data, inflexibility of interaction during the interactive segmentation process, and the influence of medical images of different sizes on the segmentation results. The number of control points obtained by initializing curve sampling can dynamically adapt to different segmentation regions, which is more convenient for users to perform subsequent interactive segmentation, can adapt to different types of medical images, and can also adapt to user interaction during the interactive segmentation process.

Description

Technical Field

[0001] The present invention belongs to the field of medical image segmentation, and particularly relates to an interactive medical image segmentation method and system based on a graph convolutional neural network. Background Art

[0002] Accurate and robust segmentation of organs or lesions in medical images is widely used in clinical applications such as diagnosis and surgical planning. Currently, it is mainly processed by manual segmentation, which is not only time-consuming, cumbersome, and expensive, but also highly dependent on professional knowledge. Therefore, in recent years, many automatic segmentation methods have been proposed. However, the segmentation performance of fully automatic methods is difficult to meet clinical applications because medical images have some inherent characteristics different from natural images, such as low contrast, pathology-induced inhomogeneity, and patient differences.

[0003] To improve the robustness and accuracy of segmentation, more and more researchers are focusing on interactive segmentation methods for medical images. Interactive segmentation methods allow users to use their knowledge and experience to correct errors or verify results, thus obtaining more accurate results than automatic segmentation methods. For a desired interactive segmentation method, the user needs to use fewer interaction times to obtain more accurate results, and the user can also easily intervene in the results and correct errors. In addition, it would be better if the limited prior knowledge of the user can be fully utilized, and it can adapt to changes in the interaction scenario, while not relying on a large amount of labeled data for pre-training. However, most existing interactive segmentation methods are difficult to meet these expectations.

[0004] According to the methods adopted, interactive segmentation methods can be divided into traditional methods and deep learning-based methods. Traditional interactive segmentation methods mainly utilize the underlying features of images and ignore some high-level semantic features that are beneficial to image segmentation. Therefore, for images with low foreground-background contrast, it is difficult for them to obtain accurate results. In recent years, deep neural network structures have been widely applied to computer vision tasks, including image segmentation. However, deep learning-based interactive segmentation methods rely on the training process and are greatly affected by the differences brought about by scene changes.

[0005] Different from automatic segmentation methods, how to design human-computer interaction has an important impact on the performance of interactive segmentation methods. Most existing interactive methods require users to provide interaction information, such as rectangular boxes, scribbles, and clicks. Most rectangle-based methods require users to provide a rectangular box containing the target object. For scribble-based methods, two types of scribbles are supported to mark foreground and background information in the image. In the case of click-based methods, the user only needs to interact by clicking the mouse. Summary of the Invention

[0006] The object of the present invention is to provide an interactive medical image segmentation method and system based on a graph convolutional neural network, so as to overcome the problems of low automatic segmentation accuracy and low manual interactive segmentation efficiency in the prior art.

[0007] An interactive medical image segmentation method based on a graph convolutional neural network, comprising the following steps:

[0008] S1, loading and visualizing the original medical image to be segmented, and providing initial interaction points on the original image;

[0009] S2, using the B-spline curve interpolation method to process the initial interaction points of the target segmentation region to obtain a closed curve, and obtaining a set of control points of the segmentation region contour by sampling at equal step lengths on the curve, so as to obtain a pre-segmentation result;

[0010] S3, performing interactive segmentation on the regions where the pre-segmentation result is not satisfactory, and the neural network model based on graph convolution performs real-time learning based on the interaction information of the interactive segmentation and predicts the offsets of the surrounding control points, and continuous interaction is performed until the segmentation reaches a satisfactory result, that is, the final control points are obtained;

[0011] S4, interpolating the final control points into a closed curve using the B-spline curve, and extracting the final segmentation result of the target region through the closed curve.

[0012] Preferably, the initial interaction points are the boundary points or extreme points of the target segmentation region.

[0013] Preferably, the original medical image to be segmented is loaded into a visualization interface; by clicking on the extreme point positions of the image loaded into the interface, the coordinate information of the initial interaction points is obtained; the coordinates of the obtained initial interaction points are converted into the relative coordinates of the medical image.

[0014] Preferably, the initial interaction points include the four extreme points of the target segmentation region.

[0015] Preferably, the set composed of the four extreme points is defined as:

[0016] V = {v O , v1, v2, v3}

[0017] In the formula, v represents the coordinates of the points in the set, expressed as v = (x, y), and the first element in the set V is appended to the end of the set to obtain a new set V':

[0018] V' = {v0, v1, v2, v3, v0}

[0019] Based on the set V', using the B-spline interpolation algorithm to obtain a closed curve as the pre-segmentation result, and defining the initial segmentation contour as:

[0020] C = {v i | i = 0,..., n}

[0021] Based on the initial contour, control points are generated with the same sampling step size to obtain a set of contour control point sets:

[0022] P = {v s*i | i = 0,..., t, s * t ≤ n}

[0023] In the formula, s is the sampling step size, and t + 1 is the number of finally obtained control points.

[0024] Preferably, by clicking on the correct boundary position for interaction, at this time, the control point closest to the click point will be used as the point to be corrected, and the click point will be used as the correct movement end point, and then prediction is performed.

[0025] Preferably, by dragging the control point for interaction, at this time, the dragged control point will be updated to the dragged end position, and then prediction is performed.

[0026] Preferably, based on the user interaction information, labels are generated as the supervision information for model learning; the label of each control point is the coordinate offset:

[0027] Δx = x1 - x0

[0028] Δy = y1 - y0

[0029] In the formula, the offset is defined as the updated coordinate minus the original coordinate, and the offset (Δx, Δy) is obtained through the original coordinates (x0, y0) and the updated coordinates (x1, y1). The offset calculated by this formula for the interaction point provided by the user is used for model learning, and then the offsets of the four control points on the left and right around the interaction point are predicted. After obtaining the offset, the updated coordinates can be calculated by combining with the original coordinates:

[0030] x′ = x + Δx

[0031] y′ = y + Δy

[0032] In the formula, (Δx, Δy) is the offset predicted by the algorithm, and (x, y) is the original control point coordinates. Adding them together can obtain the updated coordinates (x′, y′).

[0033] Preferably, after the user performs interaction, the interaction information is converted into label information. After generating the label, the neural network model based on graph convolution will use the label of the interaction point for learning and predict the coordinate offsets of the four vertices on the left and right around the interaction point.

[0034] An interactive medical image segmentation system based on a graph convolutional neural network, including a preprocessing module and an interactive segmentation module;

[0035] A preprocessing module for loading and visualizing the original medical image to be segmented, providing initial interaction points on the original image, processing the initial interaction points of the target segmentation region using the B-spline curve interpolation method to obtain a closed curve, and obtaining a set of control points for the segmentation region contour by sampling at equal intervals on the curve, thereby obtaining a pre-segmentation result;

[0036] An interactive segmentation module for performing interactive segmentation on regions where the pre-segmentation result is unsatisfactory. The neural network model based on graph convolution performs real-time learning based on the interaction information of the interactive segmentation and predicts the offsets of the surrounding control points, continuously interacting until a satisfactory segmentation result is achieved, that is, obtaining the final control points. The final control points are interpolated into a closed curve using the B-spline curve, and the final segmentation result of the target region is obtained by extracting the closed curve.

[0037] Compared with the prior art, the present invention has the following beneficial technical effects:

[0038] The present invention provides an interactive medical image segmentation method based on a graph convolutional neural network, which performs image segmentation through user interaction and algorithm collaboration, uses two stages of pre-segmentation and interactive segmentation to achieve a satisfactory segmentation result, and models the object contour through a graph convolutional neural network model to realize the interactive segmentation of medical images. This method can solve the problems of medical image segmentation in the scenario of lack of training data, inflexibility of interaction during the interactive segmentation process, and the influence of medical images of different sizes on the segmentation result.

[0039] The number of control points obtained by initializing curve sampling can dynamically adapt to different segmentation regions, which is more convenient for users to perform subsequent interactive segmentation, can adapt to different types of medical images, and can also adapt to the user's interaction during the interactive segmentation process.

[0040] The neural network model based on graph convolution can real-time learn the user's interaction information, then learn through backpropagation, and finally output the prediction results of other surrounding control points, effectively reducing the user's interaction workload. Description of the Drawings

[0041] Figure 1 is the implementation flowchart of the interactive medical image segmentation method based on a graph convolutional neural network in an embodiment of the present invention.

[0042] Figure 2 is the structural diagram of the interactive medical image segmentation method based on a graph convolutional neural network in an embodiment of the present invention.

[0043] Figure 3 is the internal structural diagram of the neural network model based on graph convolution in an embodiment of the present invention.

[0044] Figure 4 It is a schematic diagram of real-time update of contour control points during interactive segmentation in an embodiment of the present invention.

[0045] Figure 5 It is the effect diagram of the interactive medical image segmentation method based on the graph convolutional neural network in an embodiment of the present invention. Detailed implementation manners

[0046] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] An interactive medical image segmentation method based on a graph convolutional neural network includes the following steps:

[0048] S1. Load and visualize the original medical image to be segmented, and provide initial interaction points on the original image; the initial interaction points in this application include four extreme points of the target segmentation region, that is, the four vertex points of the target segmentation region: the uppermost point, the rightmost point, the lowermost point, and the leftmost point; more initial boundary points can also be provided. Providing more boundary points requires more clicking time, but at the same time, a better initial segmentation result will be obtained.

[0049] Specifically, providing initial interaction points on the original image, that is, the user provides interaction operations, which specifically include the following steps:

[0050] (1.1) Load the original medical image to be segmented into the visualization interface;

[0051] (1.2) The user obtains the coordinate information of the initial interaction points by clicking on the extreme point positions of the image loaded into the interface;

[0052] (1.3) Convert the coordinates of the obtained initial interaction points into relative coordinates of the medical image.

[0053] S2. Use the B-spline curve interpolation method to process the initial interaction points of the target segmentation region to obtain a closed curve, and obtain a set of control points of the segmentation region contour by sampling at equal step lengths on the curve to obtain a pre-segmentation result;

[0054] Spline interpolation is a method for quickly obtaining an initial contour for further interaction, and it is very fast, and does not require a lot of labeled data for model training. Based on four extreme points (leftmost, rightmost, top, and bottom), a closed curve can be obtained as the initial contour line using B-spline interpolation.

[0055] Specifically, the set composed of the initial four extreme points can be defined as:

[0056] V = {v0, v1, v2, v3}

[0057] Where v represents the coordinates of the points in the set, which can be expressed as v = (x, y). In order to finally obtain a closed curve, the first element in the set V is appended to the end of the set, resulting in a new set V':

[0058] V' = {v0, v1, v2, v3, v0}

[0059] Based on the set V', the B-spline interpolation algorithm can be used to obtain a closed curve as the pre-segmentation result. The initially segmented contour is defined as:

[0060] C = {v i | i = 0,..., n}

[0061] Finally, based on the initial contour, control points can be generated with the same sampling step size, and finally a set of contour control point sets can be obtained:

[0062] P = {v s*i | i = 0,..., t, s * t ≤ n}

[0063] Where s is the sampling step size, and t + 1 is the number of finally obtained control points. Using the contour control point set, subsequent interactive segmentation can be conveniently carried out.

[0064] S3. For the areas where the pre-segmentation results are not satisfactory, perform interactive segmentation. The neural network model based on graph convolution learns in real time based on the interactive information of the interactive segmentation and predicts the offsets of the surrounding control points. The image will display the updated control point coordinates in real time, and continue to interact until the segmentation reaches a satisfactory result, that is, the final control points are obtained;

[0065] The initial segmentation result cannot reach the ideal accuracy. Therefore, further interactive segmentation needs to be carried out on the basis of the pre-segmentation result. According to the interactive information provided by the user, the user can interact in two ways:

[0066] The first way is to click, that is, the user interacts by clicking on the correct boundary position. At this time, the control point closest to the click point will be used as the point to be corrected, and the click point will be used as the correct moving end point, and then prediction is carried out. This method is simple and efficient;

[0067] The second way is to drag, that is, the user interacts by dragging the control points. At this time, the dragged control points will be updated to the end position of the drag, and then prediction is carried out. This method is accurate and flexible.

[0068] The graph convolutional neural network is a convolutional network built on a graph structure. In the present invention, interactive segmentation is defined as a regression problem, where the position of the control point is updated by predicting the coordinate offset of the control point. Based on the user interaction information, labels are generated as the supervision information for model learning; the label of each control point is the coordinate offset:

[0069] ΔX = x1 - x0

[0070] Δy = y1 - y0

[0071] In the formula, the offset is defined as the updated coordinate minus the original coordinate. Therefore, the offset (Δx, Δy) is obtained from the original coordinate (x0, y0) and the updated coordinate (x1, y1). For the interactive points provided by the user, the offset is calculated through this formula for model learning, and then the offsets of the four control points on the left and right around the interactive point are predicted. After obtaining the offsets, the updated coordinates can be calculated by combining with the original coordinates:

[0072] x′ = x + Δx

[0073] y′ = y + Δy

[0074] In the formula, (Δx, Δy) is the offset predicted by the algorithm, (x, y) is the original control point coordinate, and adding them can obtain the updated coordinate (x′, y′). The contour segmented from the medical image by the corresponding algorithm will be updated in real time.

[0075] After the user performs the interaction, the interaction information is converted into label information. After generating the labels, the graph convolutional neural network model will use the labels of the interactive points for learning and predict the coordinate offsets of the four vertices on the left and right around the interactive point.

[0076] Define the control points on the object boundary as the vertices in the graph G = (V, E), where V is the set of vertices:

[0077] V = {V i | i = 1, 2,..., N}

[0078] In the formula, V i is the i-th vertex in the graph structure, and there are a total of N vertices, corresponding to the vertices of the segmentation contour; E represents the set of edges in the graph structure, defined as:

[0079] E = {E i，j | i ∈ 1, 2,..., N, j ∈ 1, 2,..., N}

[0080] In the formula, E i,j represents the vertex V i and the vertex V jThe edges between them, adjacent vertices are connected, and all vertices are connected to the vertices corrected by the user.

[0081] S4, interpolate the final control points into a closed curve using B-spline curve, and obtain the final segmentation result of the target area by extracting through the closed curve.

[0082] The neural network model based on graph convolution in the present invention includes an information fusion module (IVIF) and a graph convolution module (GCN), as Figure 3 shown, where the information fusion module is used to fuse image features and vertex features and extract the eigenvalue of each vertex as the input of the graph convolution module. The graph convolution module finally predicts the coordinate offset of the vertex after three times of graph convolution decoding.

[0083] The image feature and vertex position feature information fusion module is used to learn the position relationship between the current vertex and adjacent vertices. For each vertex, the input size is a 32×32 area centered on the current vertex. The image feature channel and the vertex feature channel are concatenated to realize the fusion of image features and vertex features. After three times of convolution and pooling operations on the input N×C×32×32, a 1×1 convolution with a channel size of 64 is used to obtain the output N×256. Where N is the number of input vertices, and by default, the default interaction point and 4 control points around the interaction point are used as the input, so N is default to 9 here, and C is the number of input channels. The outputs of the three times of convolution and pooling operations are N×16×15×15, N×32×6×6, and N×32×2×2 respectively. Finally, the coordinate offset results of N control points are output using three layers of graph convolution layers.

[0084] The graph convolution module is used to predict the coordinate offset of the input N control points, that is, to obtain an output of N×2; the input feature N×256 obtains the coordinate offset (Δx, Δy) of N control points after three layers of graph convolution, and then the updated coordinates (x′, y′) are calculated. The feature propagation formula of each graph convolution layer is:

[0085]

[0086] In the formula, for the l-th layer of graph convolution network, the input feature is X l , and the obtained output is X l+1 , where is adding the identity matrix on the basis of the adjacency matrix A of graph G, is corresponding degree matrix, W l is the weight matrix of the l-th layer, and ReLU() is a non-linear activation function.

[0087] Figure 4This is the update process of control points during the interactive segmentation process. Taking the update of three points on each side of the interactive point as an example, the middle point is the interactive point provided by the user, and its offset is used for model learning. The control points (taking 3 as an example) within its surrounding area will update their coordinate positions using the results predicted by the algorithm. Therefore, the contour formed by the control points will also change accordingly.

[0088] The present invention is an interactive medical image segmentation method based on a graph convolutional neural network. It performs image segmentation through the cooperation of user interaction and algorithms, and uses two stages, pre-segmentation and interactive segmentation, to achieve satisfactory segmentation results. It models the object contour through a graph convolutional neural network model to realize the interactive segmentation of medical images. This method can solve the problems of medical image segmentation in the scenario of lack of training data, inflexibility of interaction during the interactive segmentation process, and the influence of medical images of different sizes on the segmentation results.

[0089] Different from the method of using fixed initialized control points, the number of control points obtained by initializing curve sampling can dynamically adapt to different segmentation regions, which is more convenient for users to perform subsequent interactive segmentation, can adapt to different types of medical images, and can also adapt to users' interactions during the interactive segmentation process.

[0090] The neural network model based on graph convolution can learn the user's interaction information in real time, then learn through backpropagation, and finally output the prediction results of other surrounding control points, effectively reducing the user's interaction workload.

[0091] The pre-segmentation stage aims to obtain a good initial segmentation result, and other automated segmentation algorithms can also be used to obtain the pre-segmentation result. The interactive segmentation stage aims to obtain a more accurate segmentation result. The user can continuously provide more interaction information, and the model can give real-time feedback results until the segmentation result meets the user's satisfaction and the segmentation is completed.

[0092] Embodiment

[0093] An interactive medical image segmentation method based on a graph convolutional neural network includes the following steps:

[0094] S1, load the original medical image to be segmented and provide initial interactive points; the specific workflow is as follows:

[0095] (1.1), load the original medical image to be segmented into the visualization interface;

[0096] (1.2), the user obtains the coordinate information of four initial interactive points by clicking on the extreme point positions of the image loaded into the interface;

[0097] (1.3), convert the obtained four extreme point coordinates into the relative coordinates of the medical image;

[0098] S2. Based on the four extreme points, perform pre-segmentation using B-spline curves. The specific workflow is as follows:

[0099] (2.1) Append the first point in the point set formed by the extreme points to the end of the set. The number of elements in the set changes from four to five, obtaining a new set S;

[0100] (2.2) Using the set S obtained in (2.1) as the input and the B-spline interpolation algorithm, a closed curve can be obtained;

[0101] (2.3) Visualize the closed curve obtained in (2.2) in the interface. This curve is the initial contour of the object to be segmented;

[0102] S3. Based on the initial contour, extract the control point set for interactive segmentation, as Figure 2 shown. The specific workflow is as follows:

[0103] (3.1) Based on the initial contour obtained in (2.3), perform equidistant sampling. The points obtained from the sampling form a set P, which forms the control point set;

[0104] (3.2) Visualize the control point set P obtained in (3.1) in the interface for subsequent interactive segmentation;

[0105] S4. In the segmentation area that needs to be optimized, the user provides interactive information by clicking or dragging. The specific workflow is as follows:

[0106] (4.1) The user can identify the segmentation area that can be improved in the pre-segmentation result;

[0107] (4.2) The user provides interactive information in the area that needs to be improved by clicking or dragging. Clicking is to provide interactive points on the correct contour, and dragging is to drag the original control point to the correct control point position;

[0108] (4.3) Adaptively identify the interactive information. If it is a click, the click is used as the updated control point position, and the point in the control point set P in S3 that is closest to the interactive point is used as the interactive point to be updated. If it is a drag, the dragged point is used as the interactive point, and the end point of the drag is the updated coordinate position;

[0109] (4.4) Based on the interactive information in (4.3), further calculate the offset of the interactive point, and thus obtain the label information of the interactive point;

[0110] After the interaction in S5, the neural network model based on graph convolution will automatically predict the control points in the area around the interaction point. The model will output the offsets of the surrounding control points, as Figure 3 shown. The specific workflow is as follows:

[0111] (5.1) First is the model input. The input of the model consists of an image channel and a vertex channel. For N control points, there are N input messages in total. For each control point, the image channel consists of a 32×32 area centered on the control point, and the vertex channel consists of a 32×32 single-channel image containing its own vertex information and the surrounding vertex information;

[0112] (5.2) The input information undergoes three convolutional pooling operations plus one 1×1 convolutional operation to extract N×256. Each vertex obtains 256 features, and the obtained features fuse the image features and the graph vertex features;

[0113] (5.2) Then use the GCN module. Pass the obtained features through three graph convolutional layers and finally obtain the output N×2, that is, obtain the coordinate offset information (Δx, Δy) of each vertex;

[0114] (5.3) Perform semi-supervised learning through backpropagation. The middle point of the N points input to the model is the interaction point. The interaction point can generate the corresponding label (Δx, Δy) through the user interaction information. Then the model uses the label information of the interaction point for backpropagation learning;

[0115] (5.4) After the model's semi-supervised learning is completed, through model prediction, obtain the coordinate offsets of the N control points after model prediction;

[0116] In S6, then use the coordinate offset information and the interaction information to calculate the updated coordinate positions of the N points and visualize the updated contour in real time. The specific workflow is as follows:

[0117] (6.1) Calculate the new coordinates of the control points. For the interaction point, use the interaction information to calculate the updated coordinate position. For the predicted points around the interaction point, use the coordinate offset obtained in (5.4) plus the original coordinates to obtain the new coordinate position;

[0118] (6.2) Visualize the updated control point coordinates on the interface in real time;

[0119] (6.3) Interactive segmentation can be carried out repeatedly. The steps of S4 to S6 can be executed multiple times according to the user's wishes until a satisfactory segmentation result is obtained;

[0120] In S7, finally, the segmentation result can be exported using the obtained control point set, as Figure 5 shown. The specific workflow is as follows:

[0121] (7.1) Use the B-spline curve interpolation algorithm on the set of control points after interaction in S6 to obtain a closed contour;

[0122] (7.2) Use the fillPoly method in the opencv library to extract the final segmentation result using the contour obtained in (7.1).

[0123] The present invention transforms the segmentation problem into a regression problem of contour control points, enabling the neural network model based on graph convolution to utilize the non-Euclidean structure information composed of control points, and making the model independent of the size information of the original image, so that the model has strong generalization ability and adaptability.

[0124] Adaptive user interaction can effectively improve the user's interaction efficiency. Users can provide interaction information by clicking or dragging. The whole process is adaptive and does not require function switching.

[0125] The model is composed of an information fusion module and a graph convolution module. Image features and graph vertex features are fused and extracted to obtain graph features. After prediction by the graph convolution module, the predicted coordinate offsets of each vertex are output.

[0126] By converting the interaction information into label information, the model can make full use of the limited user interaction information and perform semi-supervised learning through the label information. There is a corresponding relationship between the graph nodes and the contour control points of the segmentation region. The predicted results of the nodes obtained by semi-supervised learning are the coordinate offsets of the control points.

[0127] The present invention provides an interactive medical image segmentation method based on graph convolutional neural network, which performs image segmentation through the cooperation of user interaction and algorithms, uses two stages of pre-segmentation and interactive segmentation to achieve satisfactory segmentation results, and realizes the interactive segmentation of medical images by modeling the object contour through the graph convolutional neural network model.

[0128] Different from the method of using fixed initial control points, the number of control points obtained by initializing curve sampling can dynamically adapt to different segmentation regions, which is more convenient for users to perform subsequent interactive segmentation, can adapt to different types of medical images, and can also adapt to the user's interaction during the interactive segmentation process.

[0129] The neural network model based on graph convolution can learn the user's interaction information in real time, then learn through backpropagation, and finally output the predicted results of other surrounding control points, effectively reducing the user's interaction workload.

[0130] The method includes a pre-segmentation stage and an interactive segmentation stage. The pre-segmentation stage aims to obtain a good initial segmentation result, and other automated segmentation algorithms can also be used to obtain the pre-segmentation result. The interactive segmentation stage aims to obtain a more accurate segmentation result. The user can continuously provide more interactive information, and the model can provide real-time feedback results until the segmentation result satisfies the user and the segmentation is completed.

Claims

1. An interactive medical image segmentation method based on graph convolutional neural network, characterized in that It includes the following steps: S1, Load and visualize the original medical image to be segmented, and provide initial interaction points on the original image; S2, Use the B-spline curve interpolation method to process the initial interaction points of the target segmentation region to obtain a closed curve. By sampling at equal intervals on the curve, a set of control points for the segmentation region contour is obtained, and a pre-segmentation result is obtained; S3, Perform interactive segmentation on the areas where the pre-segmentation result is not satisfactory. The neural network model based on graph convolution performs real-time learning based on the interaction information of the interactive segmentation and predicts the offsets of the surrounding control points, and continues the interaction until the segmentation reaches a satisfactory result, that is, the final control points are obtained; S4, Interpolate the final control points into a closed curve using the B-spline curve, and extract the final segmentation result of the target region through the closed curve; The initial interaction points include the four extreme points of the target segmentation region; The set composed of the four extreme points is defined as: V = {v0, v1, v2, v3} In the formula, v represents the coordinates of the points in the set, expressed as v = (x, y). Append the first element in the set V to the end of the set to obtain a new set V': V' = {v0, v1, v2, v3, v0} Based on the set V', use the B-spline interpolation algorithm to obtain a closed curve as the pre-segmentation result, and define the initial segmentation contour as: C = {v i | i = 0, …, n} On the basis of the initial contour, generate control points through the same sampling step to obtain a set of contour control point sets: P = {v s*i | i = 0, …, t, s * t ≤ n} In the formula, s is the sampling step, and t + 1 is the number of control points finally obtained.

2. The interactive medical image segmentation method based on a graph convolutional neural network according to claim 1, wherein Load the original medical image to be segmented into the visualization interface; obtain the coordinate information of the initial interaction points by clicking on the extreme point positions of the image loaded into the interface; convert the coordinates of the obtained initial interaction points into the relative coordinates of the medical image.

3. The interactive medical image segmentation method based on a graph convolutional neural network according to claim 1, characterized in that Perform interaction by clicking on the correct boundary position. At this time, the control point closest to the click point will be used as the point to be corrected, and the click point will be used as the correct movement end point, and then prediction is performed.

4. The interactive medical image segmentation method based on a graph convolutional neural network according to claim 1, wherein Perform interaction by dragging the control point. At this time, the dragged control point will be updated to the end position of the drag, and then prediction is performed.

5. A method for interactive medical image segmentation based on a graph convolutional neural network according to claim 1, wherein, Based on the user interaction information, generate labels as the supervision information for model learning; the label of each control point is the coordinate offset: Δx = x1 - x0 Δy = y1 - y0 In the formula, the offset is defined as the updated coordinate minus the original coordinate. The offset (Δx, Δy) is obtained through the original coordinate (x0, y0) and the updated coordinate (x1, y1). The offset calculated by this formula for the interaction points provided by the user is used for model learning, and then the offsets of the four control points on the left and right of the interaction point are predicted. After obtaining the offset, the updated coordinate can be calculated by combining with the original coordinate: x' = x + Δx y' = y + Δy (x, y) is the original control point coordinate, and the updated coordinate (x', y') can be obtained by adding them.

6. The interactive medical image segmentation method based on graph convolutional neural network according to claim 5, characterized in that, After the user performs interaction, convert the interaction information into label information. After generating the label, the neural network model based on graph convolution will use the label of the interaction point for learning and predict the coordinate offsets of the four vertices on the left and right of the interaction point.

7. An interactive medical image segmentation system based on graph convolutional neural network, characterized in that, It includes a preprocessing module and an interactive segmentation module; A preprocessing module for loading and visualizing the original medical image to be segmented, providing initial interaction points on the original image, processing the initial interaction points of the target segmentation region using the B-spline curve interpolation method to obtain a closed curve, and obtaining a set of control points for the segmentation region contour by sampling at equal intervals on the curve to obtain a pre-segmentation result; An interactive segmentation module for performing interactive segmentation on regions where the pre-segmentation result is unsatisfactory. The neural network model based on graph convolution performs real-time learning based on the interaction information of the interactive segmentation and predicts the offsets of the surrounding control points, and continues the interaction until the segmentation reaches a satisfactory result, that is, obtaining the final control points. The final control points are interpolated into a closed curve using the B-spline curve, and the final segmentation result of the target region is obtained by extracting the closed curve; The initial interaction points include the four extreme points of the target segmentation region; The set composed of the four extreme points is defined as: V = {v0, v1, v2, v3} In the formula, v represents the coordinates of the points in the set, expressed as v = (x, y). Append the first element in the set V to the end of the set to obtain a new set V': V' = {v0, v1, v2, v3, v0} Based on the set V', use the B-spline interpolation algorithm to obtain a closed curve as the pre-segmentation result, and define the initial segmentation contour as: C = {v i | i = 0, …, n} On the basis of the initial contour, generate control points with the same sampling step to obtain a set of contour control point sets: P = {v s*i | i = 0, …, t, s * t ≤ n} In the formula, s is the sampling step, and t + 1 is the number of finally obtained control points.

Images